Qwen3.5-2B

A GGUF repackaging of Qwen/Qwen3.5-2B, the official 2B release from the Qwen team at Alibaba Cloud, made runnable in Ollama.

This model is not my work. All credit for the weights, training, and architecture belongs to the Qwen team. This page exists only to make their model one ollama run away.

Credits

The weights are byte-identical to the upstream Q4_K_M quantization. Nothing was retrained, merged, fine-tuned, or altered.

Quick start

ollama run treyleo16/qwen:3.5-2b

API:

curl http://localhost:11434/api/chat -d '{
  "model": "treyleo16/qwen:3.5-2b",
  "messages": [{"role": "user", "content": "Explain SSM layers in one paragraph."}]
}'

Model details

Architecture note

Qwen3.5 is a hybrid attention model rather than a plain transformer. Of its 24 layers, 18 use linear attention and every 4th layer uses full attention (full_attention_interval: 4), with Mamba-style state-space mixing in between. That design is what gives a 2B model a 262K context without quadratic blowup on memory.

Practical consequence: this needs a runtime that understands qwen35. Older Ollama builds and llama.cpp forks without hybrid-SSM support will reject the GGUF at load time. Use a current Ollama release.

Reasoning output

This is a reasoning model. It emits a thinking block before its answer, which Ollama surfaces in a separate thinking field rather than inline in content. Two things follow from that:

• Set a generous num_predict. A low cap will truncate mid-reasoning and return empty content.
• If you want the reasoning, read message.thinking. If you don’t, just read message.content and ignore it.

What’s not here

The upstream Qwen3.5-2B is multimodal — it ships a 24-layer vision tower for image and video input. This tag is text-only; the vision projector is not bundled. If you need image input, the mmproj files are in the unsloth repo.

Other quantizations, from UD-IQ2_XXS (768 MB) up to BF16 (3.8 GB), are also available there if Q4_K_M isn’t the tradeoff you want.

License

Apache-2.0, inherited from the upstream release. The full license and attribution ship inside the model itself:

ollama show --license treyleo16/qwen:3.5-2b

You are free to use, modify, and redistribute under Apache-2.0 terms. Please keep credit pointing at the Qwen team — they built this.

Qwen2.5 1.5B Instruct

A GGUF repackaging of Qwen/Qwen2.5-1.5B-Instruct, the 1.5B instruction-tuned model from the Qwen team at Alibaba Cloud, made runnable in Ollama.

This model is not my work. All credit for the weights, training, and architecture belongs to the Qwen team. This page exists only to make their model one ollama run away.

Credits

Worth noting: the quantization here is Qwen’s own, not a community conversion. The weights are byte-identical to the upstream Q4_K_M file. Nothing was retrained, merged, fine-tuned, or altered.

Quick start

ollama run treyleo16/qwen:2.5-1.5b

API:

curl http://localhost:11434/api/chat -d '{
  "model": "treyleo16/qwen:2.5-1.5b",
  "messages": [{"role": "user", "content": "Write a bash one-liner to count lines in every .py file."}]
}'

Model details

What to expect

This is a direct-answer model — no thinking block. It replies immediately, which makes it noticeably faster in practice than a reasoning model of similar size that burns hundreds of tokens on a scratchpad first.

It supports tool calling, which is unusual at 1.8B and the main reason to reach for this over other models in its class. Pair it with Ollama’s tools parameter for function calling in small agent loops.

The 32K context is generous for the size — 4× what Gemma 2 2B offers — so it handles longer documents and multi-turn chat without truncation.

Reasonable expectations at 1.8B: solid at short code, formatting, extraction, classification, and routine chat. Weak at multi-step math, long-horizon reasoning, and obscure factual recall. It is a fast worker, not a thinker.

Other quantizations

The official Qwen repo also publishes q2_k (753 MB) through q8_0 (1.9 GB) and fp16 (3.6 GB), if Q4_K_M isn’t the tradeoff you want.

License

Apache-2.0, inherited from the upstream release. Permissive — commercial use, modification, and redistribution are all allowed, with attribution. The full license and attribution ship inside the model itself:

ollama show --license treyleo16/qwen:2.5-1.5b

Please keep credit pointing at the Qwen team — they built this.

Qwen3.8-2B Distill

A GGUF repackaging of empero-ai/Qwen3.8-2B-Distill — Empero’s distillation of Qwen’s 2.4-trillion-parameter flagship into a 2B model — made runnable in Ollama.

This model is not my work. The distillation is Empero’s; the architecture and base weights are the Qwen team’s. This page exists only to make it one ollama run away.

Credits

Weights are byte-identical to Empero’s Q4_K_M upload, verified against their published SHA256SUMS before packaging. Nothing was retrained, merged, or altered.

What this actually is

Qwen3.8-2.4T-A95B is Qwen’s flagship — 2.4 trillion total parameters, 95 billion active per token. You are not running that on your hardware.

Empero performed a full-parameter distillation of it into the Qwen3.5-2B architecture, training on roughly 30,000 curated teacher traces. The result is a 1.3 GB file that inherits a meaningful slice of the teacher’s reasoning behavior.

Empero’s reported results against the unmodified Qwen3.5-2B base, using lm-evaluation-harness with CoT protocols and identical settings on both sides:

Those are the publisher’s own numbers, reproduced here as claims rather than as independently verified results. Roughly doubling MMLU at this size is a strong claim; run your own evals before betting anything important on it.

Quick start

ollama run treyleo16/qwen:3.8-2b

API:

curl http://localhost:11434/api/chat -d '{
  "model": "treyleo16/qwen:3.8-2b",
  "messages": [{"role": "user", "content": "If a shop sells pens at $3 and notebooks at $7, what do 4 pens and 3 notebooks cost?"}]
}'

Empero’s recommended sampling — temperature 0.6, top_p 0.95, top_k 20 — is baked into the params layer, so you get it by default without passing anything.

Model details

Runtime requirement

Qwen3.5-class models are hybrids — three Gated DeltaNet layers for every full-attention layer. A runtime with Qwen3.5 / Gated DeltaNet support is required. Older Ollama and llama.cpp builds fail to load the architecture outright rather than degrading gracefully. If it refuses to load, update Ollama first.

Reasoning output

This is a reasoning model. Every answer opens with a thinking block, which Ollama surfaces in a separate thinking field rather than inline in content. Two consequences:

• Set a generous num_predict. A low cap truncates mid-reasoning and returns empty content.
• Read message.content for the answer, message.thinking for the trace.

Other quantizations

Empero also publishes Q5_K_M (1.5 GB), Q6_K (1.6 GB), Q8_0 (2.1 GB), and BF16 (3.9 GB) in the source repo.

License

Apache-2.0, as published by Empero, inherited from the Qwen3.5-2B student base. Full attribution ships inside the model:

ollama show --license treyleo16/qwen:3.8-2b

One note for anyone building on this: the student base is Apache-2.0, but the teacher — Qwen3.8-2.4T-A95B — is distributed under Qwen’s own license, not Apache. Empero asserts Apache-2.0 for the distilled result. If you plan to redistribute or build commercially on top of a distilled model, read both licenses yourself rather than relying on the tag.

Credit belongs to Empero for the distillation and the Qwen team for the models on both ends of it.