decadence/ gemma-4:12b-it-qat-heretic-q4_0

655 1 week ago

12B — QAT — GGUF — MLX — uncensored — sampling | ~7-13 GB

vision tools thinking audio
ollama run decadence/gemma-4:12b-it-qat-heretic-q4_0

Details

1 week ago

6261d55ce274 · 6.9GB

gemma4
·
11.9B
·
Q4_0
clip
·
52.4M
·
BF16
{ "temperature": 1, "top_k": 64, "top_p": 0.95 }

Readme

Gemma 4 12B

QAT model is a fine-tuned Large Language Model (LLM) that has undergone Quantization-Aware Training, specifically optimized during its training phase to tolerate low-bit precision (such as 4-bit)

model type quant unc size source
gemma-4:12b-it-abliterated-mlx-8bit MLX mxfp8 ✓ 13 GB AEON-7
gemma-4:12b-it-mlx-8bit MLX int8 - 12 GB lmstudio
gemma-4:12b-it-qat-heretic-q4_0 GGUF Q4_0 ✓ 6.9 GB SC117
gemma-4:12b-it-qat-mlx-4bit MLX mxfp4 - 7.9 GB OsaurusAI
gemma-4:12b-it-qat-q4_0 GGUF Q4_0 - 7.2 GB google

Sampling Parameters

Following Gemma 4 recommended sampling parameters:

parameter value
temperature 1.0
top_p 0.95
top_k 64