JiRackTernaryGemma4-26B — CPU-friendly GGUF build (CMS Manhattan)

Local reasoning / tool-calling model from CMS Manhattan, packaged for Ollama. Standard llama.cpp quantizations of google/gemma-4-26B-A4B-it.

Mixture-of-Experts: ~25.2B total parameters, only ~3.8B active per token (128 experts + 1 shared expert, 8 active per token, 30 layers). Because so few weights are touched per token, it runs usably even without a GPU.

• 25.2B total / 3.8B active parameters (MoE)
• Native context: 256K (Modelfile default num_ctx 8192, raise it if you have the RAM)
• Vocabulary: 262K tokens, 140+ languages
• Reasoning / thinking model, tool calling
• Input in this build: text (vision projector not included)
• Variants: Q6, Q5, Q4, Q3 for different memory budgets

More models: https://huggingface.co/CMSManhattan

Run

ollama run cmsmanhattan/JiRackTernaryGemma4-26b-q4:latest

Smaller:

ollama run cmsmanhattan/JiRackTernaryGemma4-26b-q3:latest

Q5 and Q6 planned; not yet published for this repo.

API

curl http://localhost:11434/api/chat -d '{
  "model": "cmsmanhattan/JiRackTernaryGemma4-26b-q4:latest",
  "messages": [{"role": "user", "content": "Plan a 3-step tool-using workflow to reconcile two ledgers."}]
}'

Python

from ollama import chat

response = chat(
    model="cmsmanhattan/JiRackTernaryGemma4-26b-q4:latest",
    messages=[{"role": "user", "content": "Explain why local tool calling matters in finance agents."}],
)
print(response.message.content)

Measured on CPU only

Intel Xeon E5-2697 v2 (Ivy Bridge, 2013, no AVX2), 16 threads, no GPU, Q4_K_M:

A decade-old server CPU gives interactive speed — the MoE design keeps the per-token compute close to a 4B dense model.

Why this model

Hosted APIs can change or retire a model overnight. If your agent posts, closes, or certifies money, a broken tool call is not a chatbot mistake — it is books, controls, and audit trail. This build is for teams that want: local inference, stable tool calling, reasoning traces you can keep on-prem.

Treat cloud models as a prototype surface. Own the inference for anything that moves or certifies money.

Variants

Q5 and Q6 not yet published for this repo.

Base model

• Source: google/gemma-4-26B-A4B-it
• Architecture: Gemma 4 MoE, hybrid sliding-window + global attention
• License of the base model: Apache 2.0 (see Gemma Terms of Use and Prohibited Use Policy)

This release is a standard post-training quantization (llama.cpp k-quants) of the original weights, without additional training. It is not an official Google checkpoint. A JiRack ternary (BitNet b1.58) QAT build of this model is in progress and will be published separately.

Publisher

CMS Manhattan https://huggingface.co/CMSManhattan https://ollama.com/cmsmanhattan