richardyoung/ phi-4-mini-instruct-heretic:Q4_K_M

147 1 week ago

Uncensored Phi-4-mini-instruct via Heretic: 5/100 refusals (from 99), KL 0.038. 3.8B with 128K context, MIT. 

ollama run richardyoung/phi-4-mini-instruct-heretic:Q4_K_M

Details

1 week ago

778c7720d65d · 2.5GB

model
phi3
·
3.84B
·
Q4_K_M

Readme

Phi-4-mini-instruct-Heretic

Uncensored Phi-4-mini-instruct via Heretic: 5⁄100 refusals (from 99), KL 0.038. 3.8B with 128K context, MIT.

🚀 Overview

This is an abliterated build of microsoft/Phi-4-mini-instruct, Microsoft’s 3.8B Phi-4-mini, a compact model strong at reasoning and math for its size. Refusal behavior was reduced using the Heretic library with KL-targeted parameters that preserve the model’s coherence.

📊 Abliteration Results

Metric Before After
Refusals 99⁄100 5⁄100
Reduction – 95%
KL Divergence – 0.038

The very low KL divergence (0.038, far below the 0.5 “damage” threshold) means the model retains essentially all of its original capabilities and coherence.

🎯 Key Features

  • Reduced censorship: 95% fewer refusals on typical “unsafe” prompts
  • Near-zero quality loss: KL 0.038
  • Strong reasoning and math for 3.8B
  • 128K-token context
  • MIT license
  • Reproducible: full reproduction data on Hugging Face

🏷️ Available Versions

Tag Size BPW Notes
latest / Q4_K_M 2.5 GB 4.85 Recommended
Q5_K_M 2.8 GB 5.68 Higher quality
Q6_K 3.2 GB 6.56 Very high quality
Q8_0 4.1 GB 8.5 Near-lossless

💻 Quick Start

ollama run richardyoung/phi-4-mini-instruct-heretic           # recommended (Q4_K_M)
ollama run richardyoung/phi-4-mini-instruct-heretic:Q8_0      # near-lossless

🛠️ Use Cases

  • Low-VRAM assistants without stock refusals
  • Reasoning and math
  • Research and red-teaming

📋 System Requirements

VRAM Recommended tier
4 GB+ Q4_K_M (2.5 GB file)
6 GB+ Q8_0 (4.1 GB file)

🔧 Technical Details

  • Base Model: microsoft/Phi-4-mini-instruct
  • Parameters: 3.8B (phi3 architecture)
  • Context Length: 128K tokens
  • Quantization: GGUF via llama.cpp (text generation)
  • Abliteration: Heretic v2.0.0.dev0 by p-e-w (Trial 184: 99→5 refusals @ KL 0.038)
  • License: MIT (inherited from the base model)

⚠️ Disclaimer

This model has reduced safety guardrails. The removal of refusal behavior means it will engage with a wider range of prompts. Use responsibly and in accordance with applicable laws and regulations.

🙏 Acknowledgments

  • Base Model: Microsoft
  • Abliteration: Heretic by p-e-w
  • Quantization: llama.cpp

Built & maintained by Richard Young · DeepNeuro