edtorre/ gemma4:12qat-hermes

20 yesterday

Gemma 4 12B agent-tuned for Hermes Agent and other local agent harnesses. One built for 20GB GPUs the other for 8GB GPUs.

vision tools thinking 
ollama run edtorre/gemma4:12qat-hermes

Details

yesterday

dccc988b22a4 · 7.2GB ·

model
gemma4
·
11.9B
·
Q4_0
projector
clip
·
52.4M
·
BF16
license
Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ TERMS AND CONDITIONS FOR US
params
{ "num_ctx": 65536, "num_predict": 8192, "temperature": 0.7, "top_k": 40, "top_p
template
{{ .Prompt }}

Readme

edtorre/gemma4:12b-agent-20gbGPU

Gemma 4 12B agent-tuned for Hermes Agent and other local agent harnesses. Built for 20GB GPUs.

Based on Google’s Gemma 4 12B IT (Unsloth Q8_K_XL quantization) with agent-optimized sampling parameters, a 64K context window, and an 8K output cap. Designed for single-GPU setups where
VRAM is tight — 13GB weights leave room for a healthy KV cache on a 16-20GB card.

What it’s for
- Driving agent harnesses locally: Hermes Agent, Claude Code, Codex, OpenCode
- Agentic coding with native tool/function calling
- Full privacy and offline operation — no cloud, no API tokens

Specs

┌───────────────┬─────────────────────────────────────┐
│ │ │
├───────────────┼─────────────────────────────────────┤
│ Base │ Gemma 4 12B IT (Unsloth UD-Q8_K_XL) │
├───────────────┼─────────────────────────────────────┤
│ Parameters │ 11.9B │
├───────────────┼─────────────────────────────────────┤
│ Quantization │ Q8_0 (near-lossless) │
├───────────────┼─────────────────────────────────────┤
│ Size │ 13 GB │
├───────────────┼─────────────────────────────────────┤
│ Context │ 65,536 (64K) │
├───────────────┼─────────────────────────────────────┤
│ Max output │ 8,192 tokens │
├───────────────┼─────────────────────────────────────┤
│ Temperature │ 0.4 │
├───────────────┼─────────────────────────────────────┤
│ Top-k / Top-p │ 64 / 0.9 │
└───────────────┴─────────────────────────────────────┘

Capabilities

  • ✅ Tools (native function calling)
  • ✅ Vision (CLIP projector)
  • ✅ Audio
  • ✅ Thinking (disabled by default via Hermes reasoning_effort: none — sends think: false to Ollama)

Optimized for constrained hardware

Pair with Ollama server-level settings for best performance on a single GPU:

   OLLAMA_FLASH_ATTENTION=1                                                                                                                                                                   
   OLLAMA_KV_CACHE_TYPE=q8_0

Flash attention reduces memory overhead. Q8_0 KV cache roughly halves context memory vs FP16. Combined, these bring total VRAM usage to ~16GB at 64K context — comfortable on a single 20GB
card with room to spare.

Quick start

   ollama pull edtorre/gemma4:12b-agent-20gbGPU                                                                                                                                               
   ollama run edtorre/gemma4:12b-agent-20gbGPU

With Hermes Agent:

   ollama launch hermes --model edtorre/gemma4:12b-agent-20gbGPU

For direct Ollama use with thinking disabled:

   ollama run edtorre/gemma4:12b-agent-20gbGPU --think false

License

Apache 2.0 (inherited from Gemma 4 / Unsloth).

or

edtorre/gemma4:12qat-hermes - For smaller GPUs 8gb and up

A lightweight, agent-tuned Gemma 4 12B model built for local agent harnesses like Hermes Agent. Based on gemma4:12b-it-qat with Q4_0 quantization for a compact 7.2 GB footprint.

## Specs

| Parameter | Value |
|—————–|—————————————-|
| Base Model | gemma4:12b-it-qat |
| Parameters | 11.9B |
| Quantization | Q4_0 |
| Size | 7.2 GB |
| Context Window | 65,536 (64K) |
| Max Output | 8,192 tokens |
| Temperature | 0.7 |
| Top-k / Top-p | 40 / 0.9 |

## Capabilities

  • ✅ Tools / function calling
  • ✅ Vision (CLIP projector)
  • ✅ Audio
  • ✅ Thinking

## Performance

Pair with these Ollama server settings for best results:

                                                                                                                                                                                              
 OLLAMA_FLASH_ATTENTION=1                                                                                                                                                                     
 OLLAMA_KV_CACHE_TYPE=q8_0

At 64K context with flash attention + Q8_0 KV cache, total VRAM usage is ~13-14 GB — comfortable on any 16-20 GB GPU with room to spare.

## Quick Start

   ollama run edtorre/gemma4:12qat-hermes

With Hermes Agent:

   hermes config set model.default edtorre/gemma4:12qat-hermes                                                                                                                                
   hermes chat

License

Apache 2.0 (inherited from Gemma 4)