richardyoung/ granite-4.2-8b-heretic:Q6_K

176 2 weeks ago

Uncensored IBM Granite 4.2 8B via Heretic: 12/100 refusals (from 99), KL 0.080. Apache-2.0, multilingual. 

ollama run richardyoung/granite-4.2-8b-heretic:Q6_K

Details

2 weeks ago

04a0a30b64cc · 7.2GB

model
granite
·
8.79B
·
Q6_K

Readme

Granite-4.2-8B-Heretic

Uncensored IBM Granite 4.2 8B via Heretic: 12⁄100 refusals (from 99), KL 0.080. Apache-2.0, multilingual.

🚀 Overview

This is an abliterated build of ibm-granite/granite-4.2-8b, IBM’s Granite 4.2 8B instruct model, multilingual (12 languages) with tool calling and a thinking mode. Refusal behavior was reduced using the Heretic library with KL-targeted parameters that preserve the model’s coherence.

📊 Abliteration Results

Metric Before After
Refusals 99⁄100 12⁄100
Reduction – 88%
KL Divergence – 0.080

The very low KL divergence (0.080, far below the 0.5 “damage” threshold) means the model retains essentially all of its original capabilities and coherence.

🎯 Key Features

  • Reduced censorship: 88% fewer refusals on typical “unsafe” prompts
  • Near-zero quality loss: KL 0.080
  • Multilingual: 12 languages
  • Tool / function calling
  • 128K-token context
  • Apache-2.0 license
  • Reproducible: full reproduction data on Hugging Face

🏷️ Available Versions

Tag Size BPW Notes
latest / Q4_K_M 5.3 GB 4.85 Recommended
Q5_K_M 6.3 GB 5.68 Higher quality
Q6_K 7.2 GB 6.56 Very high quality
Q8_0 9.3 GB 8.5 Near-lossless

💻 Quick Start

ollama run richardyoung/granite-4.2-8b-heretic           # recommended (Q4_K_M)
ollama run richardyoung/granite-4.2-8b-heretic:Q8_0      # near-lossless

🛠️ Use Cases

  • Business and agentic workflows without stock refusals
  • Multilingual assistants
  • Research and red-teaming

📋 System Requirements

VRAM Recommended tier
7 GB+ Q4_K_M (5.3 GB file)
11 GB+ Q8_0 (9.3 GB file)

🔧 Technical Details

  • Base Model: ibm-granite/granite-4.2-8b
  • Parameters: 8.8B (granite architecture)
  • Context Length: 128K tokens
  • Quantization: GGUF via llama.cpp (text generation)
  • Abliteration: Heretic v2.0.0.dev0 by p-e-w (Trial 158: 99→12 refusals @ KL 0.080)
  • License: Apache-2.0 (inherited from the base model)

⚠️ Disclaimer

This model has reduced safety guardrails. The removal of refusal behavior means it will engage with a wider range of prompts. Use responsibly and in accordance with applicable laws and regulations.

🙏 Acknowledgments

  • Base Model: IBM
  • Abliteration: Heretic by p-e-w
  • Quantization: llama.cpp

Built & maintained by Richard Young · DeepNeuro