dna5rm/granite4.1:3b-8k is a context-optimized derivative of granite4.1:3b that caps the default context window at 8,192 tokens. The base Granite 4.1 3B supports up to 128K context (extended to 512K during training), but loading at that default on a 12 GB GPU alongside other models causes VRAM pressure. This variant sets a conservative 8K default so the model loads predictably without requiring clients to pass options.num_ctx on every request.

No weights are modified. The model retains the full 3.4B parameters (Q4_K_M quantization, 2.1 GB on disk) and all base model capabilities: completion and tool use. The full IBM Granite chat template - including document-grounded RAG and function calling with XML-tagged tool call syntax - is preserved.

Model Details

Capabilities

• Completion: standard text completion
• Tools: function calling with XML-tagged <tool_call> syntax

Template Features

The Granite chat template includes built-in support for:

• Tool use: system message injection with tool schemas in <tools> XML tags; model responds with <tool_call> JSON blocks
• Document-grounded RAG: documents passed via the document role are formatted into <documents> XML and the system prompt instructs the model to answer strictly from provided documents
• Structured output: JSON output support for programmatic use

Parameters

{
  "num_ctx": 8192,
  "temperature": 0.7
}

Temperature 0.7 is set lower than Gemma’s 1.0, reflecting Granite’s enterprise-tuned calibration for more deterministic, less creative responses.

Why This Variant Exists

The upstream granite4.1:3b defaults to 128K context. While the 3B model is small enough (2.1 GB) that the full context won’t overflow VRAM on its own, loading at 128K still allocates a large KV cache that competes with other models in multi-model environments. On a 12 GB GPU:

The 8K default keeps the footprint minimal, leaving VRAM headroom for a larger primary model on the same GPU. Clients can still request larger contexts per-call via options.num_ctx.

Quick Start

# Pull
ollama pull dna5rm/granite4.1:3b-8k

# Run
ollama run dna5rm/granite4.1:3b-8k

# API - simple chat
curl http://localhost:11434/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "dna5rm/granite4.1:3b-8k",
    "messages": [{"role": "user", "content": "Write a Python function to check if a number is prime."}]
  }'

# API - tool use
curl http://localhost:11434/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "dna5rm/granite4.1:3b-8k",
    "messages": [{"role": "user", "content": "What is the weather in Charlotte NC?"}],
    "tools": [{
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Get current weather for a location",
        "parameters": {
          "type": "object",
          "properties": {
            "location": {"type": "string", "description": "City and state"}
          },
          "required": ["location"]
        }
      }
    }]
  }'

Intended Use

Lightweight agentic and tool-use workloads where a fast, small model is preferred over a large one:

• Tool calling / function routing: classification and dispatch to external tools
• RAG: document-grounded Q&A with the built-in document template
• Code assistance: Granite 4.1 is trained on a wide range of coding tasks
• Multi-language: supports English, German, Spanish, French, Japanese, Portuguese, Arabic, Czech, Italian, Korean, Dutch, and Chinese

Supported Languages

English, German, Spanish, French, Japanese, Portuguese, Arabic, Czech, Italian, Korean, Dutch, Chinese.

Base Model

Built from granite4.1:3b by IBM. Granite 4.1 models are trained from scratch on approximately 15 trillion tokens through a five-phase strategy (pre-training, mid-training with data annealing, and long-context extension up to 512K). Released April 28, 2026. See the upstream model card for full training details and capabilities.