655 Downloads Updated 1 week ago
ollama run decadence/gemma-4:12b-it-qat-heretic-q4_0
QAT model is a fine-tuned Large Language Model (LLM) that has undergone Quantization-Aware Training, specifically optimized during its training phase to tolerate low-bit precision (such as 4-bit)
| model | type | quant | unc | size | source |
|---|---|---|---|---|---|
| gemma-4:12b-it-abliterated-mlx-8bit | MLX | mxfp8 | ✓ | 13 GB | AEON-7 |
| gemma-4:12b-it-mlx-8bit | MLX | int8 | - | 12 GB | lmstudio |
| gemma-4:12b-it-qat-heretic-q4_0 | GGUF | Q4_0 | ✓ | 6.9 GB | SC117 |
| gemma-4:12b-it-qat-mlx-4bit | MLX | mxfp4 | - | 7.9 GB | OsaurusAI |
| gemma-4:12b-it-qat-q4_0 | GGUF | Q4_0 | - | 7.2 GB |
Following Gemma 4 recommended sampling parameters:
| parameter | value |
|---|---|
| temperature | 1.0 |
| top_p | 0.95 |
| top_k | 64 |