edtorre/gemma4-26b-a4b-it-qat-agent

edtorre/ gemma4-26b-a4b-it-qat-agent

55 Downloads Updated 4 days ago

Gemma 4 26B tuned for agentic use. 64K context window, flash attention + Q8 KV cache quantization for reduced VRAM. Temperature 0.7, output capped at 8192 tokens.

vision tools thinking

ollama run edtorre/gemma4-26b-a4b-it-qat-agent

curl http://localhost:11434/api/chat \
  -d '{
    "model": "edtorre/gemma4-26b-a4b-it-qat-agent",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

from ollama import chat

response = chat(
    model='edtorre/gemma4-26b-a4b-it-qat-agent',
    messages=[{'role': 'user', 'content': 'Hello!'}],
)
print(response.message.content)

import ollama from 'ollama'

const response = await ollama.chat({
  model: 'edtorre/gemma4-26b-a4b-it-qat-agent',
  messages: [{role: 'user', content: 'Hello!'}],
})
console.log(response.message.content)

Applications

Claude Code ollama launch claude --model edtorre/gemma4-26b-a4b-it-qat-agent

Codex App ollama launch codex-app --model edtorre/gemma4-26b-a4b-it-qat-agent

OpenClaw ollama launch openclaw --model edtorre/gemma4-26b-a4b-it-qat-agent

Hermes Agent ollama launch hermes --model edtorre/gemma4-26b-a4b-it-qat-agent

Codex ollama launch codex --model edtorre/gemma4-26b-a4b-it-qat-agent

OpenCode ollama launch opencode --model edtorre/gemma4-26b-a4b-it-qat-agent

Models

View all →

Name

1 model

Size / Usage

Context

Input

gemma4-26b-a4b-it-qat-agent:latest

16GB · 256K context window · Text, Image · 4 days ago

gemma4-26b-a4b-it-qat-agent:latest

16GB

256K

Text, Image

Readme

Gemma 4 26B (A4B, QAT Q4_0) tuned for agentic use with a 64K context window.

Modelfile parameters:
- num_ctx 65536 — 64K context window, enough for large system prompts, tool schemas, and conversation history
- temperature 0.7 — tighter sampling for reliable tool-use and structured output
- top_p 0.95, top_k 64 — standard Gemma defaults
- num_predict 8192 — caps output generation length

Server-side settings (set via OLLAMA_ env vars, not the Modelfile):*
- OLLAMA_FLASH_ATTENTION=1 — reduces KV cache memory
- OLLAMA_KV_CACHE_TYPE=q8_0 — halves KV cache VRAM vs f16

VRAM: ~16.8 GB on a 20 GB GPU at 64K context (model 15 GB + ~1.8 GB KV cache). Flash attention + Q8 KV cache saves ~1 GB compared to f16 defaults, leaving ~3.2 GB headroom.

Built from gemma4:26b-a4b-it-qat with parameter overrides only — model weights are unchanged.

Gemma 4 26B (A4B, QAT Q4_0) tuned for agentic use with a 64K context window.                                                                                                                 
                                                                                                                                                                                              
 Modelfile parameters:                                                                                                                                                                        
 - num_ctx 65536 — 64K context window, enough for large system prompts, tool schemas, and conversation history                                                                                
 - temperature 0.7 — tighter sampling for reliable tool-use and structured output                                                                                                             
 - top_p 0.95, top_k 64 — standard Gemma defaults                                                                                                                                             
 - num_predict 8192 — caps output generation length                                                                                                                                           
                                                                                                                                                                                              
 Server-side settings (set via OLLAMA_ env vars, not the Modelfile):*                                                                                                                         
 - OLLAMA_FLASH_ATTENTION=1 — reduces KV cache memory                                                                                                                                         
 - OLLAMA_KV_CACHE_TYPE=q8_0 — halves KV cache VRAM vs f16                                                                                                                                    
                                                                                                                                                                                              
 VRAM: ~16.8 GB on a 20 GB GPU at 64K context (model 15 GB + ~1.8 GB KV cache). Flash attention + Q8 KV cache saves ~1 GB compared to f16 defaults, leaving ~3.2 GB headroom.                 
                                                                                                                                                                                              
 Built from gemma4:26b-a4b-it-qat with parameter overrides only — model weights are unchanged.![Create_lovable_AI_robot_mascot_202607081906.jpeg](/assets/edtorre/gemma4-26b-a4b-it-qat-agent/51c44756-1096-4478-8c82-148d53104cee)

Paste, drop or click to upload images (.png, .jpeg, .jpg, .svg, .gif)