Bielik-1.5B-v3.0-Instruct-GGUF

This repo contains GGUF format model files for SpeakLeash’s Bielik-1.5B-v.3.0-Instruct.

📚 Technical report: https://arxiv.org/abs/2505.02550

DISCLAIMER: Be aware that quantised models show reduced response quality and possible hallucinations!

Available quantization formats:

• q8_0: Almost indistinguishable from float16.
• fp16: Converted Bielik-1.5B-v3.0-Instruct to fp16 GGUF

Ollama Modfile

The GGUF file can be used with Ollama. To do this, you need to import the model using the configuration defined in the Modfile. For model eg. Bielik-1.5B-v3.0-Instruct.Q8_0.gguf (full path to model location) Modfile looks like:

FROM ./Bielik-1.5B-v3.0-Instruct.Q8_0.gguf

TEMPLATE """<s>{{ if .System }}<|start_header_id|>system<|end_header_id|>

{{ .System }}<|eot_id|>{{ end }}{{ if .Prompt }}<|start_header_id|>user<|end_header_id|>

{{ .Prompt }}<|eot_id|>{{ end }}<|start_header_id|>assistant<|end_header_id|>

{{ .Response }}<|eot_id|>"""

PARAMETER stop "<|start_header_id|>"
PARAMETER stop "<|end_header_id|>"
PARAMETER stop "<|eot_id|>"

# Remeber to set low temperature for experimental models (1-3bits)
PARAMETER temperature 0.1

Model description:

• Developed by: SpeakLeash & ACK Cyfronet AGH
• Language: Polish
• Model type: causal decoder-only
• Quant from: Bielik-1.5B-v3.0-Instruct
• Finetuned from: Bielik-1.5B-v3
• License: Apache 2.0 and Terms of Use