bluehawana/deepseek-v4-flash:iq2

bluehawana/ deepseek-v4-flash:iq2_m

24 Downloads Updated yesterday

DeepSeek-V4-Flash 284B MoE (13B active), single-file IQ2_M merged from AtomicChat's GGUF shards so Ollama can pull it. ~33 tok/s on a 128 GB Apple Silicon Mac. Model by DeepSeek (MIT), quantization by AtomicChat.

ollama run bluehawana/deepseek-v4-flash:iq2_m

curl http://localhost:11434/api/chat \
  -d '{
    "model": "bluehawana/deepseek-v4-flash:iq2_m",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

from ollama import chat

response = chat(
    model='bluehawana/deepseek-v4-flash:iq2_m',
    messages=[{'role': 'user', 'content': 'Hello!'}],
)
print(response.message.content)

import ollama from 'ollama'

const response = await ollama.chat({
  model: 'bluehawana/deepseek-v4-flash:iq2_m',
  messages: [{role: 'user', content: 'Hello!'}],
})
console.log(response.message.content)

Details

Updated yesterday

yesterday

53fa62a979e3 · 104GB ·

model

archdeepseek4

parameters284B

quantizationIQ3_XXS

104GB

Readme

Single-file IQ2_M quant of DeepSeek-V4-Flash-0731 for Ollama. 284B mixture-of-experts, 13B active per token, 1M context, MIT license.

Benchmarked on a MacBook Pro M5 Max (128 GB unified memory): ~33 tok/s generation, ~545 tok/s prompt processing, 3.9 s warm load.

Tip: 104 GB of weights in 128 GB RAM leaves little KV-cache headroom. Keep context modest: /set parameter num_ctx 8192

Credits: this is not our model. Base model by DeepSeek (MIT), quantization and imatrix by AtomicChat (huggingface.co/AtomicChat/DeepSeek-V4-Flash-0731-GGUF). We merged their four GGUF shards into one file (llama-gguf-split –merge, byte-preserving) and re-hosted it so Ollama can pull it.

Scripts, the do-it-yourself recipe for any sharded GGUF, and Mac tips: github.com/bluehawana/ollama-deepseek-v4-flash-iq2_m-mac-m5max

Also on Hugging Face: huggingface.co/bluehawana/DeepSeek-V4-Flash-0731-IQ2_M-GGUF