655 1 week ago

12B — QAT — GGUF — MLX — uncensored — sampling | ~7-13 GB

vision tools thinking audio
ollama run decadence/gemma-4:12b-it-abliterated-mlx-8bit

Applications

Claude Code
Claude Code ollama launch claude --model decadence/gemma-4:12b-it-abliterated-mlx-8bit
OpenCode
OpenCode ollama launch opencode --model decadence/gemma-4:12b-it-abliterated-mlx-8bit
Hermes Agent
Hermes Agent ollama launch hermes --model decadence/gemma-4:12b-it-abliterated-mlx-8bit
OpenClaw
OpenClaw ollama launch openclaw --model decadence/gemma-4:12b-it-abliterated-mlx-8bit

Models

View all →

Readme

Gemma 4 12B

QAT model is a fine-tuned Large Language Model (LLM) that has undergone Quantization-Aware Training, specifically optimized during its training phase to tolerate low-bit precision (such as 4-bit)

model type quant unc size source
gemma-4:12b-it-abliterated-mlx-8bit MLX mxfp8 ✓ 13 GB AEON-7
gemma-4:12b-it-mlx-8bit MLX int8 - 12 GB lmstudio
gemma-4:12b-it-qat-heretic-q4_0 GGUF Q4_0 ✓ 6.9 GB SC117
gemma-4:12b-it-qat-mlx-4bit MLX mxfp4 - 7.9 GB OsaurusAI
gemma-4:12b-it-qat-q4_0 GGUF Q4_0 - 7.2 GB google

Sampling Parameters

Following Gemma 4 recommended sampling parameters:

parameter value
temperature 1.0
top_p 0.95
top_k 64