Qwen3.8-27B Heretic (abliterated, no-refusal)

GGUF quantizations of sss22213/Qwen3.8-27B-Heretic-NoRefusal — Qwen/Qwen3.8-27B with its refusal direction removed by Heretic (directional ablation). No fine-tuning, no new data: the weights are the original model with the ablation merged in, so it behaves like Qwen3.8-27B except that it no longer refuses.

Unquantized BF16 weights: sss22213/Qwen3.8-27B-Heretic-NoRefusal on Hugging Face (safetensors, ~55 GB, includes the vision encoder).

Tags

Sizes are model weights only; the KV cache for your context length comes on top.

ollama run n0404n0404/qwen3.8-27b-heretic-6fcab5:q4_k_m

Usage notes

Thinking is on by default, exactly as in Qwen3.8. To turn it off:

>>> /set nothink

or from the API:

curl http://localhost:11434/api/chat -d '{
  "model": "n0404n0404/qwen3.8-27b-heretic-6fcab5:q4_k_m",
  "think": false,
  "messages": [{"role": "user", "content": "Hello"}]
}'

Context length. The model supports up to 262,144 tokens, but Ollama starts with a small default context. Raise it when you need it (memory use grows with the context):

>>> /set parameter num_ctx 32768

Text only. Qwen3.8-27B is a vision-language model, but these GGUF files contain only the language model — image and video input are not available in Ollama. Use the Hugging Face safetensors checkpoint if you need vision.

Tool calling works as in the base model (Ollama reports the tools capability).

How it was made

• Heretic master (commit bedb94e) via Heretic WebUI, on a single RTX 5090.
• Refusal direction from mlabonne/harmful_behaviors vs mlabonne/harmless_alpaca (train[:400] each), orthogonalized against the harmless direction, subtracted from attn.o_proj and mlp.down_proj with a per-layer weight profile.
• 350 Optuna trials minimizing refusals and KL divergence at the same time; scoring on held-out test[:100] prompts with English and Chinese refusal markers, thinking disabled during the search.
• Exported trial 198 (4 / 100 refusals, KL 0.0796) as a rank-3 LoRA and merged into the original bf16 weights, then converted to GGUF and quantized with llama.cpp.

Full details, the Pareto front and reproduction parameters are on the Hugging Face model card.

Caveats

• The refusal number is a keyword-based proxy measured on English prompts with thinking off. Thinking-mode behavior was not separately evaluated; some borderline prompts may still be declined.
• Quantization changes the model further; the numbers above are for the unquantized run. Expect Q2_K in particular to be weaker than the base model at the same quant.
• No downstream benchmarks were run on this checkpoint.
• This model will answer requests the base model refuses. It is meant for research on refusal behavior, red-teaming, creative writing and similar uses. You are responsible for how you use its outputs. Apache-2.0, same as the base model.

中文說明

這是 Qwen3.8-27B 用 Heretic 做方向消融（abliteration）去除拒答後的 GGUF 量化版，沒有額外微調。在 100 題保留測試的有害提示上，拒答從 ⁹⁹⁄₁₀₀ 降到 4/100，與原模型的 KL divergence 為 0.0796（拒絕詞包含中英文，所以用中文拒答也算在內）。

• 建議用 q4_k_m（約 16.5 GB，24 GB 顯卡可跑）；16 GB 顯卡選 q3_k_m；q6_k 最接近原始精度。
• 預設開啟思考模式，關閉請用 /set nothink（API 用 "think": false）。
• 需要長上下文時用 /set parameter num_ctx 32768 調整。
• 未量化的 BF16 原始權重（safetensors，約 55 GB，含視覺編碼器）在 Hugging Face：sss22213/Qwen3.8-27B-Heretic-NoRefusal。
• GGUF 只含語言模型，不支援圖片／影片輸入；需要視覺請用 Hugging Face 上的 safetensors 版本。
• 模型會回答原模型拒絕的問題，請自行負責使用方式。