secfa/
DeepSeek-R1-UD-IQ2_XXS:24g

1,394 7 months ago

Unsloth's DeepSeek-R1 , I just merged the thing and uploaded it here. This is the full 671b model. MoE Bits:2.22bit Type:UD-IQ2_XXS Disk Size:183GB Accuracy:Better Details:MoE all 2.06bit. down_proj in MoE mixture of 2.5/2.06bit

7 months ago

11123ececbec · 196GB

model
deepseek2
·
671B
·
IQ2_XXS
template
{{- if .System }}{{ .System }}{{ end }} {{- range $i, $_ := .Messages }} {{- $last := eq (len (slice
params
{ "num_gpu": 12, "stop": [ "<｜begin▁of▁sentence｜>", "<｜end▁of▁

Readme

tag latest ：The graphics card is not supported for both ollama and screen display at the same time

tag 24g for 24g vRAM: 20g model + 1.8g fp16 kv cache + windows system reserve