oamazonasgabriel/ qwen3.5-4b:q5_k_m-8gbGPU

148 2 weeks ago

A balanced configuration variant of Qwen3.5-4B, planned for FIM (Fill-In-the-Middle) and Tool Calling in restrict capacity environments. Using the Q3_K_S GGUF quantization from HuggingFace (unsloth).

... /
params
4b7792463abd · 319B
{
"frequency_penalty": 0,
"min_p": 0.05,
"num_batch": 256,
"num_ctx": 16384,
"num_gpu": 99,
"num_thread": 6,
"presence_penalty": 0,
"repeat_penalty": 1.05,
"stop": [
"<|im_start|>",
"<|im_end|>",
"<|fim_prefix|>",
"<|fim_middle|>",
"<|fim_suffix|>"
],
"temperature": 0.3,
"top_k": 40,
"top_p": 0.9
}