Qwen2.5-14B-1M-Heretic

A long-context (up to ~1M token) Qwen2.5-14B-Instruct model with conservative Heretic abliteration to reduce refusals while preserving capability.

🚀 Overview

qwen2.5-14b-1m-heretic is built on Qwen/Qwen2.5-14B-Instruct-1M, Alibaba’s long-context member of the Qwen2.5 family. It pairs that ultra-long-context base with a conservative Heretic abliteration pass, which suppresses refusal behavior using directional ablation while keeping perplexity and general quality close to the original weights.

The result is a 14B-class instruct model that retains Qwen2.5-1M’s standout feature, a native context window measured in the hundreds of thousands of tokens, with reduced guardrails for research, red-teaming, and long-document work.

📊 Abliteration Results

Metric	Before	After	Reduction	KL Divergence
Refusals (harmful prompt set)	TBD	TBD	TBD	TBD

Refusal metrics pending re-measurement.

Heretic optimizes ablation strength to minimize refusals while bounding the KL divergence from the original model, so quality degradation is intended to be small. Exact before/after counts and KL for this build have not yet been re-measured.

🎯 Key Features

• Very long context, native context up to ~1,010,000 tokens (Qwen2.5-1M), marketed as “1M”; this Ollama build reports a 986K usable context window.
• 14.7B parameters (13.1B non-embedding), 48 transformer layers, 40 query / 8 KV heads with grouped-query attention (GQA).
• Conservative abliteration via Heretic, refusal reduction with bounded divergence from the base weights.
• Full-precision weights, the published tag ships 16-bit (FP16/BF16) weights (~30GB), not a low-bit quant.
• Strong instruction following and multilingual capability inherited from Qwen2.5-Instruct.

🏷️ Available Versions

Tag	Size	Precision	Context
latest	30GB	FP16/BF16 (16-bit)	986K

The 30GB size for a 14.7B-parameter model corresponds to ~16 bits per weight, i.e. full-precision weights rather than a quantized GGUF.

Bits-per-weight reference (for sizing alternative quants, not all published here):

Quant	BPW
IQ3_M	3.66
IQ4_XS	4.25
Q4_K_M	4.85
Q5_K_M	5.68
Q6_K	6.56
Q8_0	8.5

💻 Quick Start

ollama run richardyoung/qwen2.5-14b-1m-heretic

To run with an explicit large context window:

ollama run richardyoung/qwen2.5-14b-1m-heretic
# then, in a Modelfile or via /set parameter:
# PARAMETER num_ctx 131072

Note: very large context windows require correspondingly large KV-cache VRAM. Start with a modest num_ctx and increase as your hardware allows.

🛠️ Use Cases

• Long-document analysis, summarizing or querying entire books, contracts, or research corpora in a single context.
• Large codebase comprehension, reasoning across many files without chunking.
• Multi-document synthesis, combining several long sources into one analysis.
• Extended-context creative writing, maintaining coherence across very long passages.
• Research and red-teaming, evaluating model behavior with reduced refusal guardrails.

📋 System Requirements

Sizing is for the published 16-bit, ~30GB weights. Weights alone need ~30GB; KV cache grows with context length, so longer contexts need more VRAM.

Scenario	Context	VRAM (approx.)	Example hardware
Weights only	short	~32GB	1× RTX 5090 32GB (tight) / A100 40GB
Moderate context	32K–64K	~40–56GB	1× A100 80GB / RTX PRO 6000 96GB
Long context	128K+	64–96GB+	1× RTX PRO 6000 96GB / A100 80GB
Maximum context	up to 986K	multi-GPU / offload	2×+ 80–96GB GPUs or CPU/RAM offload

CPU/RAM offload (via Ollama/llama.cpp) lets the model run on smaller GPUs at reduced speed. A single 80–96GB GPU comfortably serves the weights plus a substantial context window.

🔧 Technical Details

• Base Model: Qwen/Qwen2.5-14B-Instruct-1M (Alibaba / Qwen)
• Parameters: 14.7B total (13.1B non-embedding), 48 layers, 40 Q-heads / 8 KV-heads (GQA)
• Context Length: native up to ~1,010,000 tokens (Qwen2.5-1M); this build reports a 986K window. Max generation length 8,192 tokens.
• Quantization: published tag is 16-bit (FP16/BF16), ~30GB; lower-bit GGUF quants can be produced with llama.cpp.
• Abliteration: Heretic by p-e-w (conservative / refusal-reduction configuration).

⚠️ Disclaimer

This model has reduced refusal guardrails via abliteration. It is more likely than the base model to comply with requests the original would decline. You are responsible for using it lawfully, ethically, and in compliance with applicable policies and regulations. Outputs may be inaccurate; verify before relying on them for any consequential use. No warranty is provided.

🙏 Acknowledgments

• Base model: Alibaba / Qwen team, Qwen2.5-14B-Instruct-1M
• Abliteration: Heretic by p-e-w
• Quantization / inference tooling: llama.cpp and Ollama

---

Built & maintained by Richard Young · DeepNeuro