batiai/ gemma4-26b:q3

284 Downloads Updated yesterday

Gemma 4 26B MoE quantized by BatiAI. 77 t/s on M4 Max. Requires 24GB+ Mac.

tools thinking

ollama run batiai/gemma4-26b:q3

curl http://localhost:11434/api/chat \
  -d '{
    "model": "batiai/gemma4-26b:q3",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

from ollama import chat

response = chat(
    model='batiai/gemma4-26b:q3',
    messages=[{'role': 'user', 'content': 'Hello!'}],
)
print(response.message.content)

import ollama from 'ollama'

const response = await ollama.chat({
  model: 'batiai/gemma4-26b:q3',
  messages: [{role: 'user', content: 'Hello!'}],
})
console.log(response.message.content)

Details

Updated yesterday

yesterday

27d470c7352e · 13GB ·

model

archgemma4

·

parameters25.2B

·

quantizationQ3_K_M

13GB

system

You are a helpful AI assistant.

31B

params

{ "num_ctx": 131072, "stop": [ "<turn|>" ], "temperature": 0.7 }

66B

Readme

Gemma 4 26B MoE — Quantized by BatiAI

Quantized from official Google weights. Verified on real Mac hardware.

Models

Tag	Size	VRAM	M4 Max (128GB)	Use Case
iq4	13GB	22GB	85.8 t/s	32GB Mac, fastest 4-bit
iq3	12GB	19GB	77 t/s	24GB Mac, imatrix optimized
q3 (latest)	13GB	20GB	70.7 t/s	24GB Mac, standard
q4	16GB	23GB	74.9 t/s	32GB+ Mac
q6	21GB	31GB	74.8 t/s	36GB+ Mac, highest quality

Quick Start

ollama run batiai/gemma4-26b:iq4

Why IQ4? — Fastest AND Smartest

IQ4 uses importance-matrix quantization: calibration data tells which weights matter most, compressing aggressively where it doesn’t matter.

	IQ4_XS (new)	Q4_K_M (standard)
Size	13GB	16GB
VRAM	22GB	23GB
Speed	85.8 t/s	74.9 t/s
Quality	4-bit imatrix	4-bit standard

Same 4-bit quality, 3GB smaller file, 15% faster. Verified with translation, tool calling, and math reasoning — identical output quality.

RAM Requirements — Be Honest

Your Mac RAM	IQ3 (12GB)	IQ4 (13GB)	Q3 (13GB)	Q4 (16GB)	Q6 (21GB)
16GB	❌ swap	❌ swap	❌ swap	❌ Won’t fit	❌ Won’t fit
24GB	✅ Fast	✅ Fits	⚠️ Tight	❌ Barely	❌ No
32GB	✅ Fast	✅ Fast	✅ Fast	✅ OK	❌ No
36GB+	✅ Fast	✅ Fast	✅ Fast	✅ Fast	✅ Fits
128GB	77 t/s	85.8 t/s	70.7 t/s	74.9 t/s	74.8 t/s

16GB Mac Users

26B models don’t work on 16GB Mac. Use these instead:

ollama run batiai/gemma4-e4b    # 57.1 t/s on 16GB Mac ✅
ollama run batiai/qwen3.5-9b    # 12.5 t/s on 16GB Mac ✅

Why BatiAI?

Quantized directly from official Google weights (not third-party)
imatrix optimized (IQ3, IQ4) for best quality at each size
Third-party GGUFs (unsloth) fail on Ollama 0.20+ — ours work
Verified on Mac mini M4 (16GB) + MacBook Pro M4 Max (128GB)
Vision support included (mmproj) — describe images in chat
Korean, tool calling, JSON generation all tested

Built for BatiFlow

Free, on-device AI automation for Mac. 5MB app, 100% local, unlimited.

https://flow.bati.ai

# Gemma 4 26B MoE — Quantized by BatiAI

Quantized from official Google weights. Verified on real Mac hardware.

## Models

| Tag | Size | VRAM | M4 Max (128GB) | Use Case |
|-----|------|------|----------------|----------|
| **iq4** | **13GB** | **22GB** | **85.8 t/s** | **32GB Mac, fastest 4-bit** |
| iq3 | 12GB | 19GB | 77 t/s | 24GB Mac, imatrix optimized |
| q3 (latest) | 13GB | 20GB | 70.7 t/s | 24GB Mac, standard |
| q4 | 16GB | 23GB | 74.9 t/s | 32GB+ Mac |
| q6 | 21GB | 31GB | 74.8 t/s | 36GB+ Mac, highest quality |

## Quick Start

```
ollama run batiai/gemma4-26b:iq4
```

## Why IQ4? — Fastest AND Smartest

IQ4 uses importance-matrix quantization: calibration data tells which weights matter most, compressing aggressively where it doesn't matter.

| | IQ4_XS (new) | Q4_K_M (standard) |
|---|---|---|
| Size | **13GB** | 16GB |
| VRAM | **22GB** | 23GB |
| Speed | **85.8 t/s** | 74.9 t/s |
| Quality | 4-bit imatrix | 4-bit standard |

Same 4-bit quality, 3GB smaller file, 15% faster. Verified with translation, tool calling, and math reasoning — identical output quality.

## RAM Requirements — Be Honest

| Your Mac RAM | IQ3 (12GB) | IQ4 (13GB) | Q3 (13GB) | Q4 (16GB) | Q6 (21GB) |
|-------------|-----------|-----------|-----------|-----------|-----------|
| 16GB | ❌ swap | ❌ swap | ❌ swap | ❌ Won't fit | ❌ Won't fit |
| 24GB | ✅ Fast | ✅ Fits | ⚠️ Tight | ❌ Barely | ❌ No |
| 32GB | ✅ Fast | ✅ Fast | ✅ Fast | ✅ OK | ❌ No |
| 36GB+ | ✅ Fast | ✅ Fast | ✅ Fast | ✅ Fast | ✅ Fits |
| 128GB | 77 t/s | **85.8 t/s** | 70.7 t/s | 74.9 t/s | 74.8 t/s |

## 16GB Mac Users

26B models don't work on 16GB Mac. Use these instead:

```
ollama run batiai/gemma4-e4b    # 57.1 t/s on 16GB Mac ✅
ollama run batiai/qwen3.5-9b    # 12.5 t/s on 16GB Mac ✅
```

## Why BatiAI?

- Quantized directly from official Google weights (not third-party)
- imatrix optimized (IQ3, IQ4) for best quality at each size
- Third-party GGUFs (unsloth) fail on Ollama 0.20+ — ours work
- Verified on Mac mini M4 (16GB) + MacBook Pro M4 Max (128GB)
- Vision support included (mmproj) — describe images in chat
- Korean, tool calling, JSON generation all tested

## Built for BatiFlow

Free, on-device AI automation for Mac. 5MB app, 100% local, unlimited.

https://flow.bati.ai

Paste, drop or click to upload images (.png, .jpeg, .jpg, .svg, .gif)