Reduced-refusal IBM Granite 4.2 3B via Heretic: 27/100 refusals (from 98), KL 0.085. Apache-2.0, multilingual.

Granite-4.2-3B-Heretic

Reduced-refusal IBM Granite 4.2 3B via Heretic: ²⁷⁄₁₀₀ refusals (from 98), KL 0.085. Apache-2.0, multilingual.

🚀 Overview

This is an abliterated build of ibm-granite/granite-4.2-3b, IBM’s Granite 4.2 3B instruct model, multilingual (12 languages) with tool calling and a thinking mode. Refusal behavior was reduced using the Heretic library with KL-targeted parameters that preserve the model’s coherence.

Note on refusals: this build still refuses 27 of 100 harmful prompts, more than our other Heretic releases. That is deliberate: the trials that refused less also degraded the model’s normal behavior (KL divergence above 0.10), and we chose to keep the original model’s capabilities intact rather than trade them for fewer refusals.

📊 Abliteration Results

Metric	Before	After
Refusals	⁹⁸⁄₁₀₀	²⁷⁄₁₀₀
Reduction	–	72%
KL Divergence	–	0.085

The very low KL divergence (0.085, far below the 0.5 “damage” threshold) means the model retains essentially all of its original capabilities and coherence.

🎯 Key Features

Reduced censorship: 72% fewer refusals on typical “unsafe” prompts
Near-zero quality loss: KL 0.085
Multilingual: 12 languages
Tool / function calling
128K-token context
Apache-2.0 license
Reproducible: full reproduction data on Hugging Face

🏷️ Available Versions

Tag	Size	BPW	Notes
latest / Q4_K_M	2.2 GB	4.85	Recommended
Q5_K_M	2.6 GB	5.68	Higher quality
Q6_K	3.0 GB	6.56	Very high quality
Q8_0	3.9 GB	8.5	Near-lossless

💻 Quick Start

ollama run richardyoung/granite-4.2-3b-heretic           # recommended (Q4_K_M)
ollama run richardyoung/granite-4.2-3b-heretic:Q8_0      # near-lossless

🛠️ Use Cases

Business and agentic workflows without stock refusals
Multilingual assistants
Research and red-teaming

📋 System Requirements

VRAM	Recommended tier
4 GB+	Q4_K_M (2.2 GB file)
5 GB+	Q8_0 (3.9 GB file)

🔧 Technical Details

Base Model: ibm-granite/granite-4.2-3b
Parameters: 3.7B (granite architecture)
Context Length: 128K tokens
Quantization: GGUF via llama.cpp (text generation)
Abliteration: Heretic v2.0.0.dev0 by p-e-w (Trial 155: 98→27 refusals @ KL 0.085)
License: Apache-2.0 (inherited from the base model)

⚠️ Disclaimer

This model has reduced safety guardrails. The removal of refusal behavior means it will engage with a wider range of prompts. Use responsibly and in accordance with applicable laws and regulations.

🙏 Acknowledgments

Base Model: IBM
Abliteration: Heretic by p-e-w
Quantization: llama.cpp

Built & maintained by Richard Young · DeepNeuro