This is the Q4_K converted version of the original Qiskit/mistral-small-3.2-24b-qiskit provided by IBM's Qiskit Huggingface. Please refer to the original mistral-small-3.2-24b-qiskit model card for more details.

ollama run Hydroxide538/mistral-small-3.2-24b-qiskit-GGUF

curl http://localhost:11434/api/chat \
  -d '{
    "model": "Hydroxide538/mistral-small-3.2-24b-qiskit-GGUF",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

from ollama import chat

response = chat(
    model='Hydroxide538/mistral-small-3.2-24b-qiskit-GGUF',
    messages=[{'role': 'user', 'content': 'Hello!'}],
)
print(response.message.content)

import ollama from 'ollama'

const response = await ollama.chat({
  model: 'Hydroxide538/mistral-small-3.2-24b-qiskit-GGUF',
  messages: [{role: 'user', content: 'Hello!'}],
})
console.log(response.message.content)

Models

View all →

Name

1 model

Size / Usage

Context

Input

mistral-small-3.2-24b-qiskit-GGUF:latest

14GB · 128K context window · Text · 2 weeks ago

mistral-small-3.2-24b-qiskit-GGUF:latest

14GB

128K

Text

Readme

base_model: Qiskit/mistral-small-3.2-24b-qiskit license: apache-2.0

quantized_by: Qiskit

Qiskit/mistral-small-3.2-24b-qiskit-GGUF

This is the Q4_K converted version of the original Qiskit/mistral-small-3.2-24b-qiskit. Please refer to the original mistral-small-3.2-24b-qiskit model card for more details.

Benchmark Results (Base vs GGUF)

Notes: For CrowsPairs (% stereotype), lower is better.

Metric	mistral-small-3.2-24b-qiskit (%)	mistral-small-3.2-24b-qiskit-GGUF (%)
QiskitHumanEval-Hard	32.45	30.46
QiskitHumanEval	47.02	37.75
HumanEval	77.49	75.00
ASDiv (acc)	3.77	4.55
MathQA (acc)	49.68	49.51
SciQ (acc)	97.50	97.40
IFEval (prompt strict)	48.44	46.04
CrowsPairs English (% stereotype)	67.08	65.89
TruthfulQA (MC1 acc)	39.41	38.80

---
base_model: Qiskit/mistral-small-3.2-24b-qiskit
license: apache-2.0
quantized_by: Qiskit
---
![mistral-small-3.2-24b-qiskit-GGUF](https://cdn-uploads.huggingface.co/production/uploads/66376b02dd21d441c54d57c6/k0s254UFu6vK7nzHqOsNz.png)

# Qiskit/mistral-small-3.2-24b-qiskit-GGUF

This is the Q4_K converted version of the original [Qiskit/mistral-small-3.2-24b-qiskit](https://huggingface.co/Qiskit/mistral-small-3.2-24b-qiskit).
Please refer to the [original mistral-small-3.2-24b-qiskit model card](https://huggingface.co/Qiskit/mistral-small-3.2-24b-qiskit) for more details.
## Benchmark Results (Base vs GGUF)
*Notes: For CrowsPairs (% stereotype), lower is better.*
| Metric                            | mistral-small-3.2-24b-qiskit (%) | mistral-small-3.2-24b-qiskit-GGUF (%) |
| --------------------------------- | --------------------------------: | -------------------------------------------: |
| QiskitHumanEval-Hard              |                             **32.45** |                                        30.46 |
| QiskitHumanEval                   |                             **47.02** |                                        37.75 |
| HumanEval                         |                             **77.49** |                                        75.00 |
| ASDiv (acc)                       |                             3.77 |                                        **4.55** |
| MathQA (acc)                      |                             **49.68** |                                        49.51 |
| SciQ (acc)                        |                             **97.50** |                                        97.40 |
| IFEval (prompt strict)            |                             **48.44** |                                        46.04 |
| CrowsPairs English (% stereotype) |                             67.08 |                                        **65.89** |
| TruthfulQA (MC1 acc)              |                             **39.41** |                                        38.80 |

Paste, drop or click to upload images (.png, .jpeg, .jpg, .svg, .gif)