sammcj/
qwen2.5-coder-32b-128k:q6_k

420 8 months ago

Qwen2.5 Coder 32B with the corrected 128k context

tools

8 months ago

bf89d7131e9e · 27GB ·

model
qwen2
·
32.8B
·
Q6_K
system
You are Qwen, created by Alibaba Cloud. You are a helpful assistant. You are an expert programmer. Y
template
{{- if .Suffix }}<|fim_prefix|>{{ .Prompt }}<|fim_suffix|>{{ .Suffix }}<|fim_middle|> {{- else if .M
params
{ "min_p": 0.9, "num_ctx": 32768, "num_keep": 256, "repeat_penalty": 1.05, "stop

Readme

This uses Unsloth’s GGUF which fixes the context length (the official Ollama model is wrong).

https://huggingface.co/unsloth/Qwen2.5-Coder-32B-Instruct-128K-GGUF/blob/main/Qwen2.5-Coder-32B-Instruct-Q6_K.gguf