Qwen2.5-7B-Instruct-Abliterated

An abliterated (uncensored) build of Qwen2.5-7B-Instruct with reduced refusal behavior and preserved general capability.

🚀 Overview

This is an abliterated version of Qwen/Qwen2.5-7B-Instruct, Alibaba’s instruction-tuned 7.6B-parameter language model. Abliteration was performed with Heretic, an automated tool that suppresses the directions in activation space most associated with refusals.

The goal of this build is a conservative abliteration: it meaningfully reduces unwarranted refusals while doing as little damage as possible to the base model’s reasoning, coding, and multilingual abilities. The model is distributed as multiple GGUF quantizations for use with Ollama and llama.cpp.

📊 Abliteration Results

Refusal metrics pending re-measurement.

🎯 Key Features

• Reduced refusals, declines far fewer benign requests than the stock instruct model.
• Preserved quality, conservative abliteration aims to retain the base model’s coherence and instruction-following.
• Strong general capability, inherits Qwen2.5’s multilingual support, coding, and reasoning strengths.
• Multiple GGUF quants, pick the size/quality trade-off that fits your hardware.

🏷️ Available Versions

Quantization / BPW reference

Approximate bits-per-weight for common GGUF quant levels (✅ = published for this model):

Higher BPW means higher fidelity to the original weights at the cost of size and memory. Q4_K_M is the recommended starting point for most users; Q5_K_M offers a quality bump, and q8_0 is near-lossless.

💻 Quick Start

Run the model directly with Ollama:

ollama run richardyoung/qwen2.5-7b-instruct-abliterated

Pull a specific quantization:

ollama pull richardyoung/qwen2.5-7b-instruct-abliterated:Q5_K_M

Use it from the API:

curl http://localhost:11434/api/chat -d '{
  "model": "richardyoung/qwen2.5-7b-instruct-abliterated",
  "messages": [{ "role": "user", "content": "Explain abliteration in one paragraph." }]
}'

🛠️ Use Cases

• Open-ended assistant and chat applications where stock refusal behavior is too restrictive.
• Red-teaming, safety research, and evaluation of guardrail behavior.
• Coding assistance and code explanation.
• Multilingual text generation, translation, and summarization.
• Reasoning, drafting, and creative writing.

📋 System Requirements

Approximate VRAM / unified-memory needed to run this ~7.6B model (weights only; add headroom for context):

CPU-only inference is possible but slower; 16 GB of system RAM is a comfortable minimum, and the long-context (YaRN) modes require additional memory for the KV cache.

🔧 Technical Details

• Base Model: Qwen/Qwen2.5-7B-Instruct
• Parameters: 7.61B total (6.53B non-embedding)
• Context Length: 32,768 tokens native, extendable to 131,072 (128K) with YaRN; up to 8,192 tokens generation
• Architecture: Transformer decoder with RoPE, SwiGLU, RMSNorm, and QKV attention bias (28 layers, GQA: 28 Q heads / 4 KV heads)
• Quantization: GGUF (llama.cpp), available as Q4_K_M, Q5_K_M, and q8_0
• Abliteration: Performed with Heretic by p-e-w

⚠️ Disclaimer

This model has had its built-in refusal guardrails substantially reduced through abliteration. It will comply with a much wider range of requests than the original instruct model, including some the base model would refuse. You are solely responsible for how you use it. Use it responsibly and lawfully, comply with all applicable laws and the base model’s license, and do not deploy it to produce harmful, illegal, or abusive content.

🙏 Acknowledgments

• Base model: Qwen2.5-7B-Instruct by Alibaba’s Qwen team.
• Abliteration: Heretic by p-e-w.
• Quantization: llama.cpp (GGUF format).

Built & maintained by Richard Young · DeepNeuro