richardyoung/ stablelm-2-12b-chat-abliterated:Q4_K_M

79 7 months ago

This is an abliterated version of [stabilityai/stablelm-2-12b-chat](https://huggingface.co/stabilityai/stablelm-2-12b-chat) that reduces refusal behavior through targeted weight modification. 

ollama run richardyoung/stablelm-2-12b-chat-abliterated:Q4_K_M

Details

7 months ago

33ece1835a73 · 7.4GB ·

model
stablelm
·
12.1B
·
Q4_K_M
params
{ "stop": [ "<|im_start|>", "<|im_end|>" ] }
template
{{- range .Messages }}<|im_start|>{{ .Role }} {{ .Content }}<|im_end|> {{ end }}<|im_start|>assistan

Readme

StableLM-2-12B-Chat-Abliterated

🚀 Overview

This is an abliterated version of stabilityai/stablelm-2-12b-chat that reduces refusal behavior through targeted weight modification using the Heretic library while maintaining model coherence.

📊 Abliteration Results

Metric Before After
Refusals TBD TBD
Reduction - TBD
KL Divergence - TBD

Refusal metrics pending re-measurement.

A low KL divergence (< 1.0) indicates the model retains its original capabilities and reasoning quality.

🎯 Key Features

  • Reduced Censorship: Fewer refusals on typical “unsafe” prompts
  • Preserved Quality: Conservative abliteration maintains model coherence
  • Full StableLM 2 Capabilities: Instruction-tuned chat model with function-calling support
  • Efficient: Q4_K_M quantization at 7.4GB balances quality and size

🏷️ Available Versions

Tag Size BPW Notes
latest / Q4_K_M 7.4 GB 4.85 Recommended

BPW reference guide (bits per weight): IQ3_M 3.66 · IQ4_XS 4.25 · Q4_K_M 4.85 · Q5_K_M 5.68 · Q6_K 6.56 · Q8_0 8.5

💻 Quick Start

ollama run richardyoung/stablelm-2-12b-chat-abliterated

Pull a specific quantization:

ollama pull richardyoung/stablelm-2-12b-chat-abliterated:Q4_K_M

🛠️ Use Cases

  • General-purpose chat and instruction following
  • Creative writing and brainstorming
  • Coding assistance and explanation
  • Research into model alignment and refusal behavior

📋 System Requirements

VRAM Performance
8 GB Good
12 GB Excellent
16 GB+ Excellent

🔧 Technical Details

Base Model: stabilityai/stablelm-2-12b-chat | Parameters: 12B | Context: 4,096 tokens | Quantization: Q4_K_M (4.85 bits per weight)

Architecture: 40 transformer layers · hidden size 5,120 · 32 attention heads · 8 key-value heads (grouped-query attention) · vocab size 100,352 · ChatML prompt format · English.

⚠️ Disclaimer

This model has reduced safety guardrails. The removal of refusal behavior means the model will engage with a wider range of prompts. Use responsibly and in accordance with the base model’s license.

🙏 Acknowledgments

  • Base Model: Stability AI, released under the StabilityAI Non-Commercial Research Community License (non-commercial use only; commercial use requires a separate agreement with Stability AI). This abliterated derivative inherits the same license terms.
  • Abliteration: Heretic by p-e-w
  • Quantization: llama.cpp

Built & maintained by Richard Young · DeepNeuro