102 4 months ago

dynamic quants 2.0 from unsloth, merged

tools
e17efbed6b71 · 151B
{
"min_p": 0,
"num_predict": 131072,
"repeat_penalty": 1,
"stop": [
"<|im_start|>",
"<|im_end|>"
],
"temperature": 0.6,
"top_k": 20,
"top_p": 0.95
}