Qwen3.8-9B-Distill

Vision · Tools · Thinking · 262K context · Apache-2.0

Qwen3.8-9B-Distill is a dense 9B reasoning model: it brings the reasoning behaviour of a frontier-scale teacher (Qwen3.8, 2.4T-A95B) into a model that fits on a single consumer GPU. Every answer opens with a <think> block learned from real teacher traces, not synthetic self-generated reasoning.

It also keeps everything from the Qwen3.5 base: native function calling (no wrapper, no tool-specific fine-tune), vision, and a 262,144-token context.

Original weights and GGUF quants by empero-ai · base model Qwen/Qwen3.5-9B · this Ollama packaging is a community repack, all credit to empero-ai.

Tags

ollama run tobestyledintro/qwen3.8-9b-distill          # latest = Q4_K_M
ollama run tobestyledintro/qwen3.8-9b-distill:q6_k

Usage

# chat
ollama run tobestyledintro/qwen3.8-9b-distill

# long context (native 262144)
ollama run tobestyledintro/qwen3.8-9b-distill --ctx 131072

# vision: drag an image, or
ollama run tobestyledintro/qwen3.8-9b-distill "Extract the total from ./invoice.png"

API — thinking can be turned off (the model then answers directly, no <think> block):

curl http://localhost:11434/api/chat -d '{
  "model": "tobestyledintro/qwen3.8-9b-distill",
  "think": false,
  "messages": [{"role": "user", "content": "17*23 ?"}]
}'

Python (vision):

import ollama
r = ollama.chat(
    model='tobestyledintro/qwen3.8-9b-distill',
    messages=[{'role': 'user', 'content': 'What is in this image?',
               'images': ['screenshot.png']}],
)
print(r.message.content)

Tool calling (native, returns structured tool_calls):

tools = [{"type": "function", "function": {
    "name": "get_weather",
    "description": "Weather for a city",
    "parameters": {"type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"]}}}]
r = ollama.chat(model='tobestyledintro/qwen3.8-9b-distill',
                messages=[{'role': 'user', 'content': 'Weather in Paris? Use get_weather.'}],
                tools=tools)
print(r.message.tool_calls)   # -> get_weather(city='Paris')

Capabilities

• Thinking — every answer opens with a learned <think> block (real teacher traces). Disable it with think: false if you want short, direct answers. Strip <think>...</think> before showing output to end users.
• Tool calling — native, per the Qwen3.5 specification; Ollama parses it into real tool_calls (no XML to handle yourself).
• Vision — clip vision tower (27 blocks, 1152 → 4096), shipped as the projector layer.
• Context — 262,144 tokens natively.

Recommended sampling

Author’s recommendation: temperature=0.6, top_p=0.95, top_k=20 — these are the defaults baked into this model. Allow generous output length: reasoning needs room (16k tokens is a reasonable budget), and greedy decoding on long generations is a known repetition-loop failure mode for this class of model.

Performance

Measured on 1× RTX 5090 (32 GB), Q4_K_M fully offloaded:

Architecture

Credit & license

• Original model: empero-ai/Qwen3.8-9B-Distill — Apache-2.0
• GGUF quants: empero-ai/Qwen3.8-9B-Distill-GGUF
• Vision projector: repackaged from the Qwen3.8-9B-Distill vision tower (as published by mradermacher for the heretic variant)
• Base model: Qwen/Qwen3.5-9B (Alibaba Qwen team)