dna5rm/gemma4:12b-8k is a context-optimized derivative of gemma4:12b that caps the default context window at 8,192 tokens. The base Gemma 4 12B ships with a 262K-token context that requires ~38 GB of KV cache alone -impossible on any consumer GPU. This variant sets a sane default so the model loads and runs on a 12 GB GPU without requiring clients to pass options.num_ctx on every request.

No weights are modified. The model retains the full 11.9B parameters (Q4_K_M quantization, 7.6 GB on disk), the CLIP vision projector (52.38M parameters), and all base model capabilities: completion, vision, audio, tools, and thinking.

Model Details

Capabilities

• Vision: text and image input, text output
• Thinking: reasoning mode with thinking tokens
• Tools: function calling / tool use
• Audio: audio input support (native architecture)
• Completion: standard text completion

Parameters

{
  "num_ctx": 8192,
  "temperature": 1,
  "top_k": 64,
  "top_p": 0.95
}

These match Google’s recommended sampling configuration for Gemma 4.

Why This Variant Exists

The upstream gemma4:12b defaults to 262K context. On a 12 GB GPU (e.g. RTX A2000):

Without an explicit num_ctx override on every API call, the model loads with the full 262K context and immediately overflows VRAM. This variant sets 8K as the default so it just works.

Clients can still request a larger context per-call via options.num_ctx if VRAM allows.

Quick Start

# Pull
ollama pull dna5rm/gemma4:12b-8k

# Run
ollama run dna5rm/gemma4:12b-8k

# API (OpenAI-compatible)
curl http://localhost:11434/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "dna5rm/gemma4:12b-8k",
    "messages": [{"role": "user", "content": "Describe this image."}],
    "images": ["<base64-encoded-image>"]
  }'

Best Practices

• Sampling: Use temperature 1, top_k 64, top_p 0.95 (already set as defaults).
• Thinking mode: In multi-turn conversations, include only the final response from previous assistant turns - strip thinking tokens from history.
• Vision: Images precede text in the input. Variable image resolution is supported via configurable visual token budgets.
• Context: If you need more than 8K context, pass options.num_ctx in your API request. Ensure your GPU has enough VRAM for the larger KV cache.

Intended Use

General-purpose multimodal assistant tasks: reasoning, agentic workflows, coding, and image understanding. Suitable for single-GPU homelab and edge deployments where VRAM is the bottleneck.

Base Model

Built from gemma4:12b by Google DeepMind. See the upstream model card for full benchmark results, training details, and architecture specifications.