Ternary-Bonsai 2 27B — GGUF

A 27B-class hybrid-attention reasoning model with ternary weights, for Prism ML’s llama.cpp fork (CUDA / Metal / CPU).

6.70 GiB for the full 27B in PQ2_0, 5.53 GiB in PTQ1_0 | 262K context 135.41 tok/s decode and 3802.74 tok/s prefill on an RTX 5090 (measured below)

This is a mirror. The weights are Prism ML’s Apache-2.0 packs from prism-ml/Ternary-Bonsai-2-27B-gguf, republished unchanged: each published blob’s sha256 is the upstream file’s sha256. Not affiliated with or endorsed by Prism ML. The model, the kernels and the upstream documentation are theirs.

⚠️ These files do not run in Ollama

The ternary kernels live in the PrismML-Eng/llama.cpp fork. Stock Ollama cannot run these packs. Verified against Ollama 0.34.2:

$ ollama create Ternary-Bonsai-2-27B:27b-pq2_0 -f Modelfile.pq2_0
Error: unsupported tensor "output.weight" size overflows

Ollama’s runner knows the architecture (qwen35) but has no pq2_0 or ptq1_0 anywhere in its libraries, so the GGUF parser rejects the file before anything is loaded. It also has no Hadamard activation runtime, and this family’s weights are stored in a rotated basis declared in prism.hadamard.* metadata — which is why the F16 pack of this family is a trap rather than a fallback: its tensors are ordinary F16/F32, so it parses and loads silently, and then decodes incoherent text. A runtime either applies the matching rotation or it produces garbage.

To actually run these files, use the fork: prebuilt binaries from PrismML-Eng/llama.cpp releases, and Bonsai-demo for setup, serving and tooling.

Tags

All five files of the upstream pack are mirrored. Every tag shares its blobs, so pulling several tags costs one download of the weights. Every tag carries the Q8_0 vision tower as a separate projector layer, loaded only when an image arrives; 27b-mmproj-bf16 carries the BF16 projector instead.

latest is PQ2_0; vision is an alias of 27b-mmproj-q8_0. Declared capabilities on every tag: completion, tools, vision, thinking, with renderer/parser pinned to qwen3.8 (the profile matching this model’s chat template). These declarations are metadata — nothing here makes the ternary packs loadable by stock Ollama.

Measured speed

Measured on the machine that published this mirror, with llama-bench from the fork (build d8f26eec7 / b10683), NVIDIA GeForce RTX 5090 32 GB, driver 616.56, every layer offloaded and flash attention on:

llama-bench -m <pack>.gguf -ngl 99 -fa on -p 512 -n 128 -r 3

Mean ± standard deviation over 3 runs, batch size 1, no draft model, no vision tower.

What the numbers say: PQ2_0 prefills 2.03x faster than PTQ1_0 (3803 vs 1876 tok/s), while decode is a tie — 135.41 vs 131.41 tok/s is a 3% gap against error bars of 5.47 and 0.76. So PQ2_0 is the default here, and PTQ1_0 is for when the last ~1.2 GiB matters.

Serving context for scale: with llama-server on the same card, a 65k-token context and -ngl 99 -fa on, decode measured 136.5 tok/s on a single streaming request. Context length and concurrency change the decode number; the table above is the reproducible configuration.

Not measured here: the F16 pack is 50.1 GiB, so it cannot be fully offloaded to a 32 GB card and no full-GPU number exists for it on this machine.

Model overview

Read from the GGUF metadata of these packs:

Base model: Qwen/Qwen3.8-27B. Upstream publishes the quality evaluations, the packing internals and the per-platform throughput; those are Prism ML’s measurements, not repeated here.

Running it

# binaries for the fork, then point a server at a pack
llama-server -m Ternary-Bonsai-2-27B-PQ2_0.gguf \
    -ngl 99 -fa on -c 65536 --host 127.0.0.1 --port 8080 \
    --temp 1.0 --top-p 0.95 --top-k 20

curl http://127.0.0.1:8080/v1/chat/completions -H "Content-Type: application/json" \
  -d '{"messages":[{"role":"user","content":"Explain quantum computing in simple terms."}]}'

Add --mmproj Ternary-Bonsai-2-27B-mmproj-Q8_0.gguf for image input. -ngl 99 offloads every layer, 0 is CPU-only; -c goes up to 262144.

License and attribution

Apache 2.0. Copyright Prism ML; base model Qwen/Qwen3.8-27B.

• Weights and GGUF pack: prism-ml/Ternary-Bonsai-2-27B-gguf
• Kernels: PrismML-Eng/llama.cpp · PrismML-Eng/mlx
• Demo, docs, run scripts: PrismML-Eng/Bonsai-demo
• Contact (upstream): contact@prismml.com

@misc{bonsai2_27b,
    title   = {Bonsai 2 27B: A 27B Ternary Reasoning Model},
    author  = {Prism ML},
    year    = {2026},
    month   = {September},
    url     = {https://prismml.com}
}