Zephyr-7B-β-Abliterated

An abliterated build of HuggingFaceH4’s Zephyr-7B-β, a DPO-aligned chat model built on Mistral-7B, with reduced refusals while preserving its helpful, conversational instruction-following style.

🚀 Overview

This is an uncensored (abliterated) build of HuggingFaceH4/zephyr-7b-beta, a ~7.24B-parameter chat model fine-tuned by the Hugging Face H4 team from mistralai/Mistral-7B-v0.1 using Direct Preference Optimization (DPO). Refusal directions were suppressed using the Heretic library, lowering refusals while preserving the base model’s friendly, conversational instruction-following quality. Distributed as a GGUF quant for efficient local inference via Ollama / llama.cpp.

📊 Abliteration Results

Metric	Before	After
Refusals	TBD	TBD
Reduction	–	TBD
KL Divergence	–	TBD

Refusal metrics pending re-measurement.

🎯 Key Features

Reduced refusals, engages with a far wider range of prompts than the stock Zephyr chat model.
Preserved quality, retains Zephyr’s DPO-aligned, helpful conversational style and reasoning ability.
Mistral foundation, built on Mistral-7B-v0.1, inheriting its efficient architecture.
GGUF quantization, distributed as a quantized GGUF for efficient local inference via Ollama / llama.cpp.

🏷️ Available Versions

Tag	Size	BPW	Notes
`Q4_K_M`	4.4GB	~4.85 (Q4_K_M)	Default 4-bit quant; good quality/size balance.

Only the Q4_K_M tag is currently published. Additional tiers can be added on request.

BPW reference (bits per weight): IQ3_M ≈ 3.66 · IQ4_XS ≈ 4.25 · Q4_K_M ≈ 4.85 · Q5_K_M ≈ 5.68 · Q6_K ≈ 6.56 · Q8_0 ≈ 8.5

💻 Quick Start

ollama run richardyoung/zephyr-7b-beta-abliterated

🛠️ Use Cases

General-purpose chat and assistant-style conversation without stock refusal behavior.
Open-ended research, writing, and analysis.
Code generation, explanation, and debugging.
Creative writing and brainstorming.

📋 System Requirements

Quant	Approx. VRAM	Performance
Q4_K_M (`~4.4GB`)	~6–8GB	Fits any modern 8GB GPU (RTX ³⁰⁶⁰⁄₄₀₆₀+); fast
Q4_K_M (offloaded)	4GB GPU + system RAM	Usable with partial CPU offload; slower
Q4_K_M (CPU-only)	8GB+ system RAM	Functional and reasonably usable on CPU

🔧 Technical Details

Base Model: HuggingFaceH4/zephyr-7b-beta (DPO-tuned from mistralai/Mistral-7B-v0.1)
Parameters: ~7.24B (MistralForCausalLM / mistral)
Context Length: 32,768 tokens (32K), with 4096-token sliding-window attention
License: MIT
Quantization: Q4_K_M (GGUF), single published tag
Abliteration: Heretic (p-e-w), targeted weight modification

⚠️ Disclaimer

This model has reduced safety guardrails and will engage with a wider range of prompts. Use responsibly and in accordance with applicable laws.

🙏 Acknowledgments

Base Model: Hugging Face H4 (built on Mistral AI’s Mistral-7B-v0.1), MIT license
Abliteration: Heretic by p-e-w
Quantization: llama.cpp

Built & maintained by Richard Young · DeepNeuro

An abliterated build of HuggingFaceH4's Zephyr-7B-β

Models

Readme