Ternary Bonsai 2 27B — a 27B reasoning model in 6 to 8 GB

~27B dense hybrid · 64 layers (~75% linear attention / ~25% full attention) · ternary weights at 1.72 bits · 262k native context · optional MTP head + vision tower

PrismML's Ternary Bonsai 2 27B, packaged for xOllama. The model is theirs: Qwen3.8-27B's architecture with every language weight — embeddings, attention and MLP projections, LM head — stored as {−1, 0, +1} with one FP16 scale per group of 128. Nothing here is re-quantized or re-trained; the tags are PrismML's published files, byte for byte, plus two things added around them: an optional MTP draft head for speculative decoding and the vision tower as a tag choice.

Full model card, whitepaper & benchmarks: prism-ml/Ternary-Bonsai-2-27B-gguf · MTP bundles: ManniX-ITA/Ternary-Bonsai-2-27B-MTP-GGUF on Hugging Face.

These tags do not run on stock Ollama. The ternary weight types (PQ2_0, PTQ1_0) need kernels that upstream llama.cpp does not have. They run on xOllama, whose opencoti engine carries them from its c11 cut; everything on this page was measured on NVIDIA cards (CUDA). Use an xOllama release newer than v0.40.2-xollama: that one has the engine but its model reader does not know the two weight types yet. If a load fails on stock Ollama, nothing is wrong with the download.

How it works

Ternary g128. A ternary value carries log₂3 ≈ 1.585 bits; with the 16-bit scale spread over 128 weights the format costs ~1.71 bits per weight, and 1.72 for the model as a whole. That is ~5.8 GB for a language model that is ~54 GB in FP16. The weights are stored in a rotated basis (a blockwise Hadamard rotation folded in offline), and the runtime applies the matching transform to activations; the file declares its rotation, so an engine either applies it or refuses the file.

Two packings of the same weights. PTQ1_0 packs five trits to the byte and lands on the information-theoretic size (1.75 bits/weight, 5.95 GB). PQ2_0 gives each trit a 2-bit slot (2.13 bits/weight, 7.21 GB), which is cheaper to unpack. Same model, same answers at the same sampler; which one is faster depends on the card, so both are published.

The MTP head is added, not merged. PrismML's files carry no next-token-prediction layer. The mtp- tags bundle the upstream Qwen3.8-27B MTP head (q8_0, ~0.45 GB) as one extra layer, blk.64.*; every other key and tensor is the PrismML file unchanged. The head stays in the normal basis, and its embedding lookup on the rotated table gets the inverse transform in the engine. The bundle is reproducible: the two sha256 on the Hugging Face card are identical across machines.

The vision tower is PrismML's own projector, attached by tag: vision- carries the BF16 reference file (0.93 GB), lvision- the Q8_0 one (0.63 GB). It is loaded at the first image request; text-only use never pays for it.

Quality — PrismML's published numbers (thinking mode, 14 benchmarks)

These are PrismML's measurements, not ours: EvalScope + vLLM on an H100, thinking mode, identical decoding and scoring for every row. They are reproduced here so the table travels with the tags. Nothing on this page re-ran them.

Read the two tables together. The headline is that a 5.95 GB file sits within 0.4 points of a 17.6 GB 4-bit build, and twelve points above the conventional 2-bit build of about its own size. The second table says where the remaining gap is: math, coding and instruction following are at the baseline; knowledge-and-reasoning (−5.7) and vision (−5.2) are where the ternary model gives ground. If your work is MuSR-shaped multi-step deduction or OCR, that is the cost.

Speed — what the MTP head buys

The draft head predicts ahead and the model verifies in one batch. On the ternary files the engine drafts two tokens a step (a measured ladder, no flag needed). Two measurements, different prompts, so read each column on its own:

Draft acceptance on the coding prompt was 0.71, 2.4 tokens accepted per step. The bottom two rows are one prompt each, not a benchmark: code is more predictable than chat, which is why the gain is larger there than on the twelve-prompt medians. Expect something between 1.25× and 1.5× depending on what you generate. On this 3090 the dense packing (PTQ1_0) was also the faster one, with and without the head; PrismML's own table shows that reverses on some architectures, so measure on yours.

Speculation preserves the model's distribution, not the argmax at near-ties: with greedy sampling an answer can differ from the unspeculated one where the top two tokens are within a few hundredths of a nat. No degeneration was measured (distinct 4-gram ratio identical on and off).

Which tag to pull

Twelve tags plus :latest (= mtp-lvision-ptq1_0): two packings × with or without the MTP head × no vision, vision- or lvision-. The tag reads left to right as [mtp-][vision-|lvision-]<packing>.

vision- or lvision-? Both read the same test image correctly on both packings. The Q8_0 projector saves 0.3 GB; the BF16 one is PrismML's reference. Take lvision- when the 0.3 GB matters and vision- otherwise.

Pull

xollama run mannix/bonsai-2-27b                     # :latest = mtp-lvision-ptq1_0, ~7.0 GB
xollama run mannix/bonsai-2-27b:mtp-vision-ptq1_0   # MTP + vision, dense packing, ~7.3 GB
xollama run mannix/bonsai-2-27b:mtp-vision-pq2_0    # the same in 2-bit slots, ~8.6 GB
xollama run mannix/bonsai-2-27b:mtp-ptq1_0          # text only, with the draft head
xollama run mannix/bonsai-2-27b:ptq1_0              # PrismML's file alone, smallest

What the tags ship — requires xOllama newer than v0.40.2

Every tag carries the same serving identity. The renderer and parser are those of the official qwen3.8:27b; the sampling values are PrismML's recommendation for thinking mode, not Qwen's defaults:

RENDERER  qwen3.8
PARSER    qwen3.5          # there is no "qwen3.8" parser
PARAMETER temperature 1
PARAMETER top_p 0.95
PARAMETER top_k 20
PARAMETER min_p 0.05       # PrismML: at least as accurate as 0, follows instructions more reliably
PARAMETER presence_penalty 0
PARAMETER repeat_penalty 1
PARAMETER think_budget high
PARAMETER think_budget_message "…I have used my thinking budget. I must stop analysing now and act…"

The thinking budget is on by default. This is a reasoning model that thinks at length, and on its own it can spend an entire response inside the thinking block. think_budget high closes the block once the model has used that share of what it may emit, and the message is written into the block just before, so the model reads that it must act instead of being cut off mid-thought. A budget is a ceiling, not a target: reasoning that finishes inside it is unaffected. Override it per request (think, or OpenAI's reasoning_effort), or remove it:

xollama run mannix/bonsai-2-27b:mtp-vision-ptq1_0
>>> /set parameter think_budget max

Two parameters are deliberately not pinned. num_ctx: the architecture is good for 262k natively, and only a quarter of the layers carry a KV cache, so context is unusually cheap here — but the right value depends on your VRAM, and the default is far below what a long thinking trace wants. The draft depth: the engine measures it at load and picked 2 on every card we ran; pinning a number would only stop it adapting.

>>> /set parameter num_ctx 65536

Good to know

• Which engine. The ternary kernels are opencoti's port of PrismML's llama.cpp fork. PrismML's own fork runs the plain files; only opencoti runs the bundled MTP head, because the head's lookup on the rotated embedding table needs the inverse transform.
• Low reasoning effort is not a thing on this model. PrismML: low is unsupported and behaves close to the maximum. Use the budget above to bound thinking rather than asking for less of it.
• Short output caps cut the answer off. A cap of a few hundred tokens ends generation inside the thinking block, before any answer. Leave room, or let the budget do the bounding.
• Vision is the weakest category in PrismML's table (66.19 against 71.36 for FP16), and OCR the weakest part of it. The tower works; do not expect FP16 OCR from it.
• Files are verified, not re-made. The four language files and both projectors match their Hugging Face sha256; each kind of tag generated through xOllama before it was pushed.

Links: HF · PrismML GGUF + full card · HF · MTP bundles · whitepaper · Qwen3.8-27B (architecture and MTP head) · xOllama

Apache-2.0 · Bonsai 2 by Prism ML, built from Qwen3.8-27B (Alibaba Cloud). Created using Bonsai by Prism ML.