tobestyledintro/ minicpm5-2b:8k

283 1 week ago

Compact 2.5B model by OpenBMB — 2B-class open-source SOTA, built for on-device, local assistants, coding agents and tool-use workflows 

ollama run tobestyledintro/minicpm5-2b:8k

Details

1 week ago

bdc9b75c26a4 · 1.6GB

model
llama
·
2.52B
·
Q4_K_M
template
{{- if .Messages -}} {{- range .Messages -}} <|im_start|>{{ .Role }} {{ .Content }}<|im_end|> {{ end
system
You are MiniCPM5-2B, a small, efficient assistant created by OpenBMB. You answer concisely in the us
params
{ "num_ctx": 8192, "stop": [ "<|im_end|>", "</s>" ], "temperature":

Readme

MiniCPM5-2B

Compact 2.5B model by OpenBMB — 2B-class open-source SOTA, built for on-device, local assistants, coding agents and tool-use workflows.

Base: openbmb/MiniCPM5-2B-GGUF (Q4_K_M, 1.56 GB) — Apache-2.0.

Which tag should I choose?

Same weights, different context sizes. Bigger context = more VRAM for the KV cache.

Tag Context VRAM needed Best for
:8k 8,192 ~2.5 GB Laptops, iGPU, running cat
:32k 32,768 ~3.5 GB Default balanced, light RAG
:64k 65,536 ~5 GB Long documents, big RAG
:128k 131,072 ~8 GB Max native context, codebases
:latest 32,768 ~3.5 GB = :32k

VRAM = 1.56 GB weights + KV cache F16 + marge Ollama. Rule of thumb: ~43 KB per token of context in F16.

Pull examples:

ollama pull tobestyledintro/minicpm5-2b:8k
ollama pull tobestyledintro/minicpm5-2b:128k

Tip: halve the KV cache with quantized cache before ollama serve:

OLLAMA_FLASH_ATTENTION=1 OLLAMA_KV_CACHE_TYPE=q8_0 ollama serve

With q8_0 cache, :128k fits in ~5 GB and :64k in ~3.5 GB.

Highlights

  • 2B-class SOTA (avg 53.9), competitive with 4B-class models
  • Strong in code, math, reasoning and agentic tasks
  • 128K native long context
  • Tool calling (XML-style, native minicpm5 parser in SGLang)
  • Small footprint, ideal for laptops and edge
  • Architecture: standard LlamaForCausalLM, 42 layers, GQA (16Q / 2KV)

Parameters

All tags use OpenBMB recommended defaults (think mode):

  • temperature 1.0
  • top_p 0.95
  • stops: <|im_end|>,
  • num_ctx varies per tag (see table)

Think mode

MiniCPM5 is a deep-thinking model. With temperature=1.0 it thinks before answering. For more direct answers, lower temperature at request time, e.g. ollama run tobestyledintro/minicpm5-2b:8k –temperature 0.7.

Links

License

Apache-2.0