edtorre/gemma4-26b-a4b-it-qat-agent

edtorre/ gemma4-26b-a4b-it-qat-agent:latest

55 Downloads Updated 4 days ago

Gemma 4 26B tuned for agentic use. 64K context window, flash attention + Q8 KV cache quantization for reduced VRAM. Temperature 0.7, output capped at 8192 tokens.

vision tools thinking

ollama run edtorre/gemma4-26b-a4b-it-qat-agent

curl http://localhost:11434/api/chat \
  -d '{
    "model": "edtorre/gemma4-26b-a4b-it-qat-agent",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

from ollama import chat

response = chat(
    model='edtorre/gemma4-26b-a4b-it-qat-agent',
    messages=[{'role': 'user', 'content': 'Hello!'}],
)
print(response.message.content)

import ollama from 'ollama'

const response = await ollama.chat({
  model: 'edtorre/gemma4-26b-a4b-it-qat-agent',
  messages: [{role: 'user', content: 'Hello!'}],
})
console.log(response.message.content)

Details

Updated 4 days ago

4 days ago

8c22d78567c9 · 16GB ·

model

archgemma4

parameters25.2B

quantizationQ4_0

14GB

projector

archclip

parameters573M

quantizationBF16

1.2GB

license

Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ TERMS AND CONDITIONS FOR US

10kB

params

{ "num_ctx": 65536, "num_predict": 4096, "temperature": 0.7, "top_k": 64, "top_p

79B

Readme

Gemma 4 26B (A4B, QAT Q4_0) tuned for agentic use with a 64K context window.

Modelfile parameters:
- num_ctx 65536 — 64K context window, enough for large system prompts, tool schemas, and conversation history
- temperature 0.7 — tighter sampling for reliable tool-use and structured output
- top_p 0.95, top_k 64 — standard Gemma defaults
- num_predict 8192 — caps output generation length

Server-side settings (set via OLLAMA_ env vars, not the Modelfile):*
- OLLAMA_FLASH_ATTENTION=1 — reduces KV cache memory
- OLLAMA_KV_CACHE_TYPE=q8_0 — halves KV cache VRAM vs f16

VRAM: ~16.8 GB on a 20 GB GPU at 64K context (model 15 GB + ~1.8 GB KV cache). Flash attention + Q8 KV cache saves ~1 GB compared to f16 defaults, leaving ~3.2 GB headroom.

Built from gemma4:26b-a4b-it-qat with parameter overrides only — model weights are unchanged.

Gemma 4 26B (A4B, QAT Q4_0) tuned for agentic use with a 64K context window.                                                                                                                 
                                                                                                                                                                                              
 Modelfile parameters:                                                                                                                                                                        
 - num_ctx 65536 — 64K context window, enough for large system prompts, tool schemas, and conversation history                                                                                
 - temperature 0.7 — tighter sampling for reliable tool-use and structured output                                                                                                             
 - top_p 0.95, top_k 64 — standard Gemma defaults                                                                                                                                             
 - num_predict 8192 — caps output generation length                                                                                                                                           
                                                                                                                                                                                              
 Server-side settings (set via OLLAMA_ env vars, not the Modelfile):*                                                                                                                         
 - OLLAMA_FLASH_ATTENTION=1 — reduces KV cache memory                                                                                                                                         
 - OLLAMA_KV_CACHE_TYPE=q8_0 — halves KV cache VRAM vs f16                                                                                                                                    
                                                                                                                                                                                              
 VRAM: ~16.8 GB on a 20 GB GPU at 64K context (model 15 GB + ~1.8 GB KV cache). Flash attention + Q8 KV cache saves ~1 GB compared to f16 defaults, leaving ~3.2 GB headroom.                 
                                                                                                                                                                                              
 Built from gemma4:26b-a4b-it-qat with parameter overrides only — model weights are unchanged.![Create_lovable_AI_robot_mascot_202607081906.jpeg](/assets/edtorre/gemma4-26b-a4b-it-qat-agent/51c44756-1096-4478-8c82-148d53104cee)

Paste, drop or click to upload images (.png, .jpeg, .jpg, .svg, .gif)