Bielik-11B-v3.0-Instruct-GGUF

This repo contains GGUF format model files for SpeakLeash’s Bielik-11B-v3.0-Instruct.

Link to GGUF files on HF: https://huggingface.co/speakleash/Bielik-11B-v3.0-Instruct-GGUF

DISCLAIMER: Be aware that quantised models show reduced response quality and possible hallucinations!

Available quantization formats:

• q4_k_m: Uses Q6_K for half of the attention.wv and feed_forward.w2 tensors, else Q4_K
• q5_k_m: Uses Q6_K for half of the attention.wv and feed_forward.w2 tensors, else Q5_K
• q6_k: Uses Q8_K for all tensors
• q8_0: Almost indistinguishable from float16. High resource use and slow. Not recommended for most users.
• 16bit: Converted to FP16 and BF16 GGUF format.

Ollama Modfile

The GGUF file can be used with Ollama. To do this, you need to import the model using the configuration defined in the Modfile. For model eg. Bielik-11B-v3.0-Instruct.Q8_0.gguf (full path to model location) Modfile looks like:

FROM ./Bielik-11B-v3.0-Instruct.Q8_0.gguf

TEMPLATE """{{- /* SYSTEM + TOOLS INJECTION */ -}}
{{- if or .System .Tools -}}
<|im_start|>system
{{- if .System }}
{{ .System }}
{{- end }}

{{- if .Tools }}
You are provided with tool signatures that you can use to assist with the user's query.
You do not have to use a tool if you can respond adequately without it.
Do not make assumptions about tool arguments. If required parameters are missing, ask a clarification question.

If you decide to invoke a tool, you MUST respond with ONLY valid JSON in the following format:
{"name":"<tool-name>","arguments":{...}}

Below is a list of tools you can invoke (JSON):
{{ .Tools }}
{{- end }}
<|im_end|>
{{- end }}

{{- /* MESSAGES */ -}}
{{- range $i, $_ := .Messages }}
<|im_start|>{{ .Role }}
{{ .Content }}<|im_end|>
{{- end }}

{{- /* GENERATION PROMPT */ -}}
<|im_start|>assistant"""

PARAMETER stop "<|start_header_id|>"
PARAMETER stop "<|end_header_id|>"
PARAMETER stop "<|eot_id|>"

PARAMETER temperature 0.1

Model description:

• Developed by: SpeakLeash & ACK Cyfronet AGH
• Language: Multilingual (32 European languages, optimized for Polish)
• Model type: causal decoder-only
• Quant from: Bielik-11B-v3.0-Instruct
• Finetuned from: speakleash/Bielik-11B-v3-Base-20250730
• License: Apache 2.0 and Terms of Use