second_constantine/
deepseek-coder-v2:16b

158 3 weeks ago

This is a brand new Mixture of Export (MoE) model from DeepSeek, specializing in coding instructions. (quantized IQ4_XS)

tools 16b

3 weeks ago

5a596221f1f0 · 8.6GB ·

model
deepseek2
·
15.7B
·
IQ4_XS
params
{ "stop": [ "User:", "Assistant:" ] }
template
{{- if .Messages }} {{- if or .System .Tools }} {{- if .System }} {{ .System }} {{- end }} {{- if .T

Readme

Based on https://huggingface.co/lmstudio-community/DeepSeek-Coder-V2-Lite-Instruct-GGUF

Feature Value
vision false
thinking false
tools true
Device Speed, token/s Context VRAM, gb Versions
RTX 3090 24gb ~139 4096 10 IQ4_XS,0.12.2
RTX 3090 24gb ~139 15360 13 IQ4_XS,0.12.2
RTX 2080ti 11gb ~114 4096 10 IQ4_XS,0.12.2
RTX 2080ti 11gb ~37 15360 14 (23%/77% CPU/GPU) IQ4_XS,0.12.2
M1 Max 32gb ~80 4096 10 IQ4_XS,0.12.2
M1 Max 32gb ~81 15360 14 IQ4_XS,0.12.2