turintech/ minicpm5-2b-artemis:q4_k_s

136 Downloads Updated 6 days ago

MiniCPM5-2B compressed with Artemis by TurinTech: a 1.53 GB GGUF that got 188/200 held-out tool calls and 215/250 maths problems, against 182 and 180 for OpenBMB's Q4_K_M. Measured Q4_K_S: Mean KLD 0.050 vs BF16 on two texts (mean).

ollama run turintech/minicpm5-2b-artemis:q4_k_s

curl http://localhost:11434/api/chat \
  -d '{
    "model": "turintech/minicpm5-2b-artemis:q4_k_s",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

from ollama import chat

response = chat(
    model='turintech/minicpm5-2b-artemis:q4_k_s',
    messages=[{'role': 'user', 'content': 'Hello!'}],
)
print(response.message.content)

import ollama from 'ollama'

const response = await ollama.chat({
  model: 'turintech/minicpm5-2b-artemis:q4_k_s',
  messages: [{role: 'user', content: 'Hello!'}],
})
console.log(response.message.content)

Details

Updated 6 days ago

6 days ago

4a91a0dc45bd · 1.5GB

model

archllama

·

parameters2.52B

·

quantizationQ4_K_S

1.5GB

params

{ "num_ctx": 8192 }

17B

license

Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ TERMS AND CONDITIONS FOR US

12kB

Readme

MiniCPM5 2B Artemis

MiniCPM5-2B for local chat, tool calls and maths, measured with thinking off.

1.534 GB download for the default file, MiniCPM5-2B-Artemis-Q4_K_S.gguf.
Held out: 188 of 200 tool calls and 215 of 250 maths problems, against 182 and 180 for OpenBMB’s Q4_K_M (1.561 GB).

Run with Ollama

Tags: latest and q4_k_s are the same file, MiniCPM5-2B-Artemis-Q4_K_S.gguf (1.534 GB download).

ollama run turintech/minicpm5-2b-artemis

Results

Our file against OpenBMB’s official Q4_K_M, bartowski’s Q4_K_S of the same size, and the original. Tool calls and maths are held-out tasks set aside before the search started; the KL text helped choose the file.

File	Size	Tool calls (of 200)	GSM8K (of 250)	Closeness to original (KL, lower is better)
Ours, Q4_K_S	1.534 GB	188	215	0.050
OpenBMB Q4_K_M (official)	1.561 GB	182	180	0.070
bartowski Q4_K_S	1.534 GB	179	193	0.053
BF16 original	5.039 GB	183	194	reference

bartowski’s larger IQ4_NL and Q4_K_M keep the original’s next-token probabilities closer. Memory was not measured for this card. How it was tested.

Files

Every file we publish for this model. Sizes are download sizes, not memory.

File	Size	Use it for	Ollama tag
`MiniCPM5-2B-Artemis-Q4_K_S.gguf`	1.53 GB	Default	`q4_k_s` (latest)

The Ollama tag runs our package, which holds the same file: ollama run turintech/minicpm5-2b-artemis:<tag>.

Text only: no vision or MTP file.

Limitations

Held-out set of two tasks; maths uses a second, corrected scoring of the same items.
Tested with thinking off.
The numbers come from llama.cpp, not from Ollama or LM Studio.

Licence

Licence: Apache-2.0.

# MiniCPM5 2B Artemis

MiniCPM5-2B for local chat, tool calls and maths, measured with thinking off.

- 1.534 GB download for the default file, `MiniCPM5-2B-Artemis-Q4_K_S.gguf`.
- Held out: 188 of 200 tool calls and 215 of 250 maths problems, against 182 and 180 for OpenBMB's Q4_K_M (1.561 GB).

## Run with Ollama

Tags: `latest` and `q4_k_s` are the same file, `MiniCPM5-2B-Artemis-Q4_K_S.gguf` (1.534 GB download).

```
ollama run turintech/minicpm5-2b-artemis
```

## Results

Our file against OpenBMB's official Q4_K_M, bartowski's Q4_K_S of the same size, and the original. Tool calls and maths are held-out tasks set aside before the search started; the KL text helped choose the file.

| File | Size | Tool calls (of 200) | GSM8K (of 250) | Closeness to original (KL, lower is better) |
| --- | --- | --- | --- | --- |
| **Ours, Q4_K_S** | 1.534 GB | 188 | 215 | 0.050 |
| OpenBMB Q4_K_M (official) | 1.561 GB | 182 | 180 | 0.070 |
| bartowski Q4_K_S | 1.534 GB | 179 | 193 | 0.053 |
| BF16 original | 5.039 GB | 183 | 194 | reference |

bartowski's larger IQ4_NL and Q4_K_M keep the original's next-token probabilities closer. Memory was not measured for this card. [How it was tested](https://huggingface.co/turintech/MiniCPM5-2B-Artemis-GGUF#how-it-was-tested).

## Files

Every file we publish for this model. Sizes are download sizes, not memory.

| File | Size | Use it for | Ollama tag |
| --- | --- | --- | --- |
| [`MiniCPM5-2B-Artemis-Q4_K_S.gguf`](https://huggingface.co/turintech/MiniCPM5-2B-Artemis-GGUF/resolve/main/MiniCPM5-2B-Artemis-Q4_K_S.gguf) | 1.53 GB | Default | `q4_k_s` (latest) |

- The Ollama tag runs our package, which holds the same file: `ollama run turintech/minicpm5-2b-artemis:<tag>`.

Text only: no vision or MTP file.

## Limitations

- Held-out set of two tasks; maths uses a second, corrected scoring of the same items.
- Tested with thinking off.
- The numbers come from llama.cpp, not from Ollama or LM Studio.

## Licence

Licence: [Apache-2.0](https://github.com/OpenBMB/MiniCPM/blob/main/LICENSE).

Paste, drop or click to upload images (.png, .jpeg, .jpg, .svg, .gif)