vanilj/
qwq-32b-iq4_xs:short

247 9 months ago

IQ4_XS quant of Qwen/QwQ-32B

tools

9 months ago

18a74cf310e7 · 18GB ·

model
qwen2
·
32.8B
·
IQ1_M
system
You have extremely limited time to think and respond to the user’s query. Every additional second
license
Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ TERMS AND CONDITIONS FOR US
params
{ "min_p": 0.1, "num_ctx": 16384, "repeat_penalty": 1, "stop": [ "<|im_start
template
{{- if .Suffix }}<|fim_prefix|>{{ .Prompt }}<|fim_suffix|>{{ .Suffix }}<|fim_middle|> {{- else if .M

Readme

I added the :short, medium, and long tags, each one have a system prompt designed to keep the thinking short, medium or long. It also sets the default context length to 16k.

GGUF from bartowski/QwQ-32B-GGUF.

Original model from Qwen/QwQ-32B.

The parameters are from the recommended Usage Guidelines.