herald / phi3-128k

Phi-3 128K version of Q5

3B

12.3K Pulls Updated 4 months ago

4 months ago

d8f0cff3daa1 · 2.7GB

model
llama
·
3.82B
·
Q5_K_M
template
{{ if .System }}<|system|> {{ .System }}<|end|> {{ end }}{{ if .Prompt }}<|user|> {{ .Prompt }}<|end|> {{ end }}<|assistant|> {{ .Response }}<|end|>
params
{"stop":["<|user|>","<|assistant|>","<|system|>","<|end|>","<|endoftext|>"]}

Readme

Convert from PrunaAI/Phi-3-mini-128k-instruct-GGUF-Imatrix-smashed, adopt Q5_K_8_4 quantization.

Its multilingual capabilities are clearly superior to version Q4 quantization.