secfa/DeepSeek-R1-UD-IQ2

secfa/ DeepSeek-R1-UD-IQ2_XXS:24g

1,401 Downloads Updated 1 year ago

Unsloth's DeepSeek-R1 , I just merged the thing and uploaded it here. This is the full 671b model. MoE Bits:2.22bit Type:UD-IQ2_XXS Disk Size:183GB Accuracy:Better Details:MoE all 2.06bit. down_proj in MoE mixture of 2.5/2.06bit

ollama run secfa/DeepSeek-R1-UD-IQ2_XXS:24g

curl http://localhost:11434/api/chat \
  -d '{
    "model": "secfa/DeepSeek-R1-UD-IQ2_XXS:24g",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

from ollama import chat

response = chat(
    model='secfa/DeepSeek-R1-UD-IQ2_XXS:24g',
    messages=[{'role': 'user', 'content': 'Hello!'}],
)
print(response.message.content)

import ollama from 'ollama'

const response = await ollama.chat({
  model: 'secfa/DeepSeek-R1-UD-IQ2_XXS:24g',
  messages: [{role: 'user', content: 'Hello!'}],
})
console.log(response.message.content)

Details

Updated 1 year ago

1 year ago

11123ececbec · 196GB ·

model

archdeepseek2

parameters671B

quantizationIQ2_XXS

196GB

template

{{- if .System }}{{ .System }}{{ end }} {{- range $i, $_ := .Messages }} {{- $last := eq (len (slice

394B

params

{ "num_gpu": 12, "stop": [ "<｜begin▁of▁sentence｜>", "<｜end▁of▁

161B

Readme

tag latest ：The graphics card is not supported for both ollama and screen display at the same time

tag 24g for 24g vRAM: 20g model + 1.8g fp16 kv cache + windows system reserve