Qwen 3.5 9B (Unsloth Dynamic Q4_K_XL with MTP & Vision)

High-performance compact foundation model balancing high intelligence with lightweight hardware requirements. Features Unsloth Dynamic Q4_K_XL quantization, MTP Speculative Decoding, and integrated Multimodal Vision (mmproj-BF16).

The sweet spot for local reasoning, tool calling, document summarization, and vision-language tasks on consumer GPUs.

🚀 Key Highlights

• Unsloth Dynamic Q4_K_XL: Enhanced quantization maintaining critical matrix fidelity.
• High Throughput: Generates at ~60–70 tokens/s on 16GB GPUs and up to 24k context on 8GB GPUs.
• Extended Context: Out-of-the-box 64k context window with native RoPE scalability up to 256k.
• MTP Speculative Decoding: Pre-tuned draft_num_predict: 2 for faster generation.
• Vision Multimodal: Includes mmproj-BF16 (921 MB) for visual perception.

📦 Tags

• latest: Unsloth Dynamic Q4_K_XL (~6.1 GB model + 921 MB mmproj), 64k default context.

💡 Usage Recommendations

ollama run jetelain/Qwen3.5-9B

Hardware Recommendations

• 8 GB VRAM GPUs: Full GPU offload up to 24k context (~60 t/s). Maintains responsive flow (>22 t/s) up to 88k context with hybrid memory.
• 16 GB VRAM GPUs: 100% full GPU offload across 160k+ context at ~65 tokens/s.
• Unified Memory (32 GB+): Runs entirely in memory with vast context headroom.

💡 Hardware Alias Tip (Zero Disk Overhead)

Lock in the optimal context threshold for 8 GB or 16 GB GPUs:

# On 8 GB GPUs (88k context maintaining >22 t/s):
curl http://localhost:11434/api/create -d '{
  "model": "Qwen3.5-9B",
  "from": "jetelain/Qwen3.5-9B",
  "parameters": {
    "num_ctx": 90112
  }
}'

# On 16 GB GPUs (160k context, full GPU offload):
curl http://localhost:11434/api/create -d '{
  "model": "Qwen3.5-9B",
  "from": "jetelain/Qwen3.5-9B",
  "parameters": {
    "num_ctx": 163840
  }
}'