richardyoung/ gemma-4-12b-coder-abliterated:Q4_K_M

522 2 weeks ago

a Gemma-4 12B coding model with refusals dramatically reduced via the Heretic library.

tools thinking 
ollama run richardyoung/gemma-4-12b-coder-abliterated:Q4_K_M

Details

2 weeks ago

5bdb6af1c778 · 7.4GB ·

model
gemma4
·
11.9B
·
Q4_K_M
params
{ "num_ctx": 8192, "stop": [ "<end_of_turn>", "<start_of_turn>" ] }
template
{{ if .System }}{{ .System }} {{ end }}<start_of_turn>user {{ .Prompt }}<end_of_turn> <start_of_turn

Readme

Gemma-4-12B-Coder-Abliterated

Abliterated build of gemma-4-12B-coder, a Gemma-4 12B coding model with refusals dramatically reduced via the Heretic library.

🚀 Overview

Abliterated version of yuxinlu1/gemma-4-12B-coder-fable5-composer2.5-v1, a Gemma-4 12B model fine-tuned for verified-chain-of-thought Python coding. Refusal behavior was reduced with Heretic. The base refused nearly every “unsafe” prompt; this build engages with a far wider range.

📊 Abliteration Results

Metric Before After
Refusals (harmful-prompt eval) 100⁄100 7⁄100
Reduction – 93%
KL Divergence – 0.058

A 93% refusal reduction at low KL (0.058), among the strongest reductions in this collection (the base was extremely refusal-heavy).

🎯 Key Features

  • Strong coding, verified chain-of-thought Python focus (Gemma-4 12B base)
  • 93% fewer refusals via Heretic, low KL
  • 5 GGUF quant tiers

🏷️ Available Versions

Tag Size BPW Notes
IQ3_M ~5.5 GB 3.66 Smallest
IQ4_XS ~6.5 GB 4.25 Great quality/size
latest / Q4_K_M ~7.5 GB 4.85 Recommended
Q5_K_M ~8.7 GB 5.68 Higher quality
Q8_0 ~13 GB 8.5 Near-lossless

💻 Quick Start

ollama run richardyoung/gemma-4-12b-coder-abliterated

⚠️ Format note: this finetune uses a custom <channel|>-based output format (the composer2.5 scheme). The packaged Modelfile uses the standard Gemma <start_of_turn> template; for best results you may need to adapt the chat template to the channel format, or call the API with raw: true and the model’s native format.

🛠️ Use Cases

  • Python code generation, chain-of-thought coding
  • Research on coding-model refusal behavior

🔧 Technical Details

⚠️ Disclaimer

This model has reduced safety guardrails and will engage with a wider range of prompts, including generating code that a stock model would refuse. Use responsibly and in accordance with applicable laws.

🙏 Acknowledgments

  • Base Model: yuxinlu1 (gemma-4-12B-coder) / Google (Gemma 4)
  • Abliteration: Heretic by p-e-w
  • Quantization: llama.cpp

Built & maintained by Richard Young · DeepNeuro