mannix/
smallthinker-abliterated

498 9 months ago

A new small reasoning model fine-tuned from the Qwen 2.5 3B Instruct model. I-Quants models, abliterated with uncensored prompt.

Models

View all →

15 models

smallthinker-abliterated:latest

1.8GB · 32K context window · Text · 9 months ago

smallthinker-abliterated:Q4_0

1.8GB · 32K context window · Text · 9 months ago

smallthinker-abliterated:q4_1

2.0GB · 32K context window · Text · 9 months ago

smallthinker-abliterated:Q4_K_S

1.8GB · 32K context window · Text · 9 months ago

smallthinker-abliterated:Q4_K_M

1.9GB · 32K context window · Text · 9 months ago

smallthinker-abliterated:Q5_0

2.2GB · 32K context window · Text · 9 months ago

smallthinker-abliterated:Q5_1

2.3GB · 32K context window · Text · 9 months ago

smallthinker-abliterated:Q5_K_S

2.2GB · 32K context window · Text · 9 months ago

smallthinker-abliterated:Q5_K_M

2.2GB · 32K context window · Text · 9 months ago

smallthinker-abliterated:q8_0

3.3GB · 32K context window · Text · 9 months ago

smallthinker-abliterated:IQ3_XXS

1.3GB · 32K context window · Text · 9 months ago

smallthinker-abliterated:iq3_xs

1.4GB · 32K context window · Text · 9 months ago

smallthinker-abliterated:IQ3_S

1.5GB · 32K context window · Text · 9 months ago

smallthinker-abliterated:IQ4_XS

1.7GB · 32K context window · Text · 9 months ago

smallthinker-abliterated:IQ4_NL

1.8GB · 32K context window · Text · 9 months ago

Readme

This is an uncensored version of PowerInfer/SmallThinker-3B-Preview created with abliteration (see remove-refusals-with-transformers to know more about it). This is a crude, proof-of-concept implementation to remove refusals from an LLM model without using TransformerLens.

  • Quantization from fp32
  • Using i-matrix calibration_datav3.txt
  • Abliterated with uncensored prompt

References

HuggingFace