edtorre/ gemma4-31b-hermes:latest

39 Downloads Updated 2 weeks ago

Gemma 4 31B (Q4_0) optimized for Hermes Agent — 64K context, 8192 max tokens, flash attention + Q8 KV cache. Fits 24 GB VRAM with optimizations.

vision tools thinking

ollama run edtorre/gemma4-31b-hermes

curl http://localhost:11434/api/chat \
  -d '{
    "model": "edtorre/gemma4-31b-hermes",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

from ollama import chat

response = chat(
    model='edtorre/gemma4-31b-hermes',
    messages=[{'role': 'user', 'content': 'Hello!'}],
)
print(response.message.content)

import ollama from 'ollama'

const response = await ollama.chat({
  model: 'edtorre/gemma4-31b-hermes',
  messages: [{role: 'user', content: 'Hello!'}],
})
console.log(response.message.content)

Details

Updated 2 weeks ago

2 weeks ago

615f1a4c56f6 · 19GB ·

model

archgemma4

·

parameters30.7B

·

quantizationQ4_0

18GB

projector

archclip

·

parameters576M

·

quantizationBF16

1.2GB

license

Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ TERMS AND CONDITIONS FOR US

10kB

params

{ "num_ctx": 65536, "num_predict": 8192, "temperature": 0.7, "top_k": 64, "top_p

78B

template

{{ .Prompt }}

13B

Readme

edtorre/gemma4-31b-hermes - Best for 24+ gb vram setups.

A max-size, agent-tuned Gemma 4 31B model for local agent harnesses like Hermes Agent. Based on gemma4:31b-it-qat with Q4_0 quantization at 18 GB.

## Specs

| Parameter | Value |
|—————–|————————————|
| Base Model | gemma4:31b-it-qat |
| Parameters | 30.7B |
| Quantization | Q4_0 |
| Size | 18 GB |
| Context Window | 65,536 (64K) |
| Max Output | 8,192 tokens |
| Temperature | 0.7 |
| Top-k / Top-p | 64 / 0.9 |

## Capabilities

✅ Tools / function calling
✅ Vision (CLIP projector)
✅ Audio
✅ Thinking

## Performance Notes

At 18 GB + 64K context, this model is tight on 20 GB VRAM. Requires these Ollama server settings:

                                                                                                                                                                                              
 OLLAMA_FLASH_ATTENTION=1                                                                                                                                                                     
 OLLAMA_KV_CACHE_TYPE=q8_0

With flash attention and Q8_0 KV cache, total VRAM usage is approximately 18-19 GB — fits on a 20 GB card with very little headroom.

## Quick Start

   ollama run edtorre/gemma4-31b-hermes

With Hermes Agent:

   hermes config set model.default edtorre/gemma4-31b-hermes                                                                                                                                  
   hermes chat

License

Apache 2.0

# edtorre/gemma4-31b-hermes - Best for 24+ gb vram setups.                                                                                                                                                        
                                                                                                                                                                                              
   A max-size, agent-tuned Gemma 4 31B model for local agent harnesses like Hermes Agent. Based on `gemma4:31b-it-qat` with Q4_0 quantization at 18 GB.                                       
                                                                                                                                                                                              
   ## Specs                                                                                                                                                                                   
                                                                                                                                                                                              
   | Parameter       | Value                              |                                                                                                                                   
   |-----------------|------------------------------------|                                                                                                                                   
   | Base Model      | gemma4:31b-it-qat                  |                                                                                                                                   
   | Parameters      | 30.7B                              |                                                                                                                                   
   | Quantization    | Q4_0                               |                                                                                                                                   
   | Size            | 18 GB                              |                                                                                                                                   
   | Context Window  | 65,536 (64K)                       |                                                                                                                                   
   | Max Output      | 8,192 tokens                       |                                                                                                                                   
   | Temperature     | 0.7                                |                                                                                                                                   
   | Top-k / Top-p   | 64 / 0.9                           |                                                                                                                                   
                                                                                                                                                                                              
   ## Capabilities                                                                                                                                                                            
                                                                                                                                                                                              
   - ✅ Tools / function calling                                                                                                                                                              
   - ✅ Vision (CLIP projector)                                                                                                                                                               
   - ✅ Audio                                                                                                                                                                                 
   - ✅ Thinking                                                                                                                                                                              
                                                                                                                                                                                              
   ## Performance Notes                                                                                                                                                                       
                                                                                                                                                                                              
   At 18 GB + 64K context, this model is tight on 20 GB VRAM. Requires these Ollama server settings:                                                                                          
                                                                                                                                                                                              
 ```                                                                                                                                                                                          
                                                                                                                                                                                              
 OLLAMA_FLASH_ATTENTION=1                                                                                                                                                                     
 OLLAMA_KV_CACHE_TYPE=q8_0                                                                                                                                                                    
                                                                                                                                                                                              
 ```                                                                                                                                                                                          
                                                                                                                                                                                              
   With flash attention and Q8_0 KV cache, total VRAM usage is approximately 18-19 GB — fits on a 20 GB card with very little headroom.                                                       
                                                                                                                                                                                              
   ## Quick Start                                                                                                                                                                             
                                                                                                                                                                                              
   ```bash                                                                                                                                                                                    
   ollama run edtorre/gemma4-31b-hermes                                                                                                                                                       
 ```                                                                                                                                                                                          
                                                                                                                                                                                              
 With Hermes Agent:                                                                                                                                                                           
                                                                                                                                                                                              
 ```bash                                                                                                                                                                                      
   hermes config set model.default edtorre/gemma4-31b-hermes                                                                                                                                  
   hermes chat                                                                                                                                                                                
 ```                                                                                                                                                                                          
                                                                                                                                                                                              
 License                                                                                                                                                                                      
                                                                                                                                                                                              
 Apache 2.0![gemma4-31b-hermes.jpeg](/assets/edtorre/gemma4-31b-hermes/b7d173bc-bfc0-4cb6-a354-219edefd9db0)

Paste, drop or click to upload images (.png, .jpeg, .jpg, .svg, .gif)