Qwen3.6-27B Multi-Teacher Distill (Qwen3.8 Max · GLM 5.2 · Kimi K3) — Heretic (Abliterated)

Text-only GGUF build of a multi-teacher distillation finetune of Qwen/Qwen3.6-27B, decensored with Heretic (automatic abliteration via directional ablation with per-model parameter optimization).

The finetune is a LoRA (r=32, α=64, all attention + MLP projections, ~1 epoch SFT) trained on r0b0tlab/qwen3.8-max-glm5.2-kimi-k3-distillation — multilingual reasoning and tool-use traces distilled from three teacher models: Qwen 3.8 Max, GLM 5.2 and Kimi K3 — then merged into the full weights with PEFT before abliteration. The goal is to blend the reasoning styles of three frontier teachers onto the 27B base. This build removes the refusal direction on top of that — the model answers questions the original would decline. Reasoning (<think>) behavior is preserved and parsed natively by Ollama.

Tags

ollama run n0404n0404/qwen3.6-finetune-qwen3.8-max-glm5.2-kimi-k3-distillation-a56-1168cb-heretic:Q4_K_M

Long context is unusually cheap on this architecture: 48 of the 64 layers use linear attention, so the KV cache only costs ~64 KiB per token (about 2 GiB at 32k context). A 32 GB GPU (e.g. RTX 5090) runs q6_k fully offloaded at 32k–64k context; 24 GB cards are comfortable with Q4_K_M; 16 GB cards should pick q2_k/q3_k_m.

Model details

• Base: Qwen/Qwen3.6-27B → multi-teacher distillation LoRA (merged with PEFT) → Heretic abliteration
• Training data: r0b0tlab/qwen3.8-max-glm5.2-kimi-k3-distillation (multi-teacher reasoning / tool-use / multi-turn traces, en·zh·es·fr·de·ja)
• Architecture: qwen35 hybrid — 64 layers (48 linear attention + 16 full attention), GQA with 4 KV heads × 256 head dim, up to 262,144 context
• Thinking: reasoning model; served with Ollama’s native qwen3.5 renderer/parser, so thinking content is separated from the final answer
• Requires: Ollama 0.31.2 or newer

What was changed vs. the original

1. Distillation finetune — a LoRA trained on reasoning traces from three teacher models (Qwen 3.8 Max, GLM 5.2, Kimi K3) was merged into the base weights (PEFT merge_and_unload).
2. Abliteration — refusal behavior removed with Heretic (run 1168cb, 200 optimization trials): refusals on a 100-prompt harmful set dropped from ⁹⁷⁄₁₀₀ to ⁶⁄₁₀₀, with a KL divergence of only 0.0084 vs. the pre-abliteration model — output distribution on normal prompts is essentially unchanged.
3. Text-only — the original is multimodal (vision-language). The vision tower is not included in this GGUF; image input is not supported.
4. No MTP head — the multi-token-prediction (speculative decoding) head was dropped during conversion (--no-mtp); it is unused by Ollama.

Converted with llama.cpp convert_hf_to_gguf.py + llama-quantize.

Responsible use

Safety guardrails have been intentionally weakened. This model will comply with requests the original model would refuse. You are responsible for how you use it and for complying with applicable laws and the upstream license. Not recommended for user-facing deployments without your own moderation layer.

License & credits

• License: Apache-2.0 (inherited from the original model)
• Credits: Qwen team (base model), r0b0tlab (distillation dataset), p-e-w/heretic (abliteration), PEFT (LoRA merge), llama.cpp (GGUF conversion)

繁體中文說明

Qwen/Qwen3.6-27B 的多教師蒸餾微調 + 去審查（abliterated）版本，使用 Heretic 自動化處理，轉為純文字 GGUF 供 Ollama 使用。

• 微調資料集：r0b0tlab/qwen3.8-max-glm5.2-kimi-k3-distillation（自 Qwen 3.8 Max、GLM 5.2、Kimi K3 三個教師模型蒸餾的推理／工具使用軌跡，涵蓋六種語言），以 LoRA（r=32, α=64，約 1 epoch SFT）訓練後用 PEFT 合併為完整權重，目標是把三家旗艦模型的推理風格融合到 27B 基底上
• 本版本移除了拒答行為（Heretic run 1168cb，200 次參數優化）：拒答率 ⁹⁷⁄₁₀₀ → ⁶⁄₁₀₀，KL 散度僅 0.0084，一般提示的輸出分布幾乎不變
• <think> 推理功能完整保留，由 Ollama 原生解析
• 混合線性注意力架構，KV cache 僅約 64 KiB/token，長上下文非常省 VRAM
• 建議：16 GB 顯卡用 q2_k/q3_k_m，24 GB 用 Q4_K_M，32 GB（如 RTX 5090）可用 q6_k 開 32k–64k 上下文
• 不含視覺功能（原模型為多模態，本 GGUF 僅文字）
• 需要 Ollama 0.31.2 以上版本

負責任使用：本模型已刻意弱化安全防護，會回應原模型拒絕的請求。使用者須自行承擔使用責任並遵守相關法律與授權條款。不建議在未加自有審核層的情況下對外部署。