LESSTHANSUPER/
THE_OMEGA_DIRECTIVE-Mistral_Small3.2-24b:Q4_K_S

100 2 weeks ago

Large dataset roleplaying model finetuned for natural response. Made by ReadyArt (Huggingface).

tools

2 weeks ago

2a3f87445fdb · 14GB ·

model
llama
·
23.6B
·
Q4_K_S
template
{{- if .Suffix }}<|fim_prefix|>{{ .Prompt }}<|fim_suffix|>{{ .Suffix }}<|fim_middle|> {{- else if .M
system
Write {{char}}'s next reply in this fictional roleplay with {{user}}.
params
{ "stop": [ "[INST]" ] }

Readme

THE OMEGA DIRECTIVE / I-MATRIX / 24B / I-QUANT

The creator has trained this model on multi-turn chat data, and has taken out all cliche LLM response mannerisms, earning it the “unslop” moniker. I can attest to this. Also, through testing this model myself, I’ve found this model to keep characters distinct in both single and group chats. To stuff as many parameters in as little VRAM as possible, weighted I-quants will be listed.

Note that I-quants forfeit some token generation speed relative to K-quants in exchange for storage efficiency. Any 4-bit, or smaller, model will work on 16GB GPUs, though the small K-quant is recommended, even over the I-quant, for speed. These models were taken from GGUF formats from Huggingface.

Original model (ReadyArt):

GGUF weighted quantizations (mradermacher):

OBLIGATORY_PICTURE_DIRECTIVE.png