novaforgeai/ qwen2.5-3b:q2k

115 1 month ago

Qwen 2.5 3B – NovaForgeAI Edition Qwen 2.5 3B – NovaForgeAI Edition is a CPU-optimized, low-latency LLM designed for fast local inference on low-end and mid-range systems.

... /
params
96b4cb6dfab9 · 122B
{
"num_batch": 256,
"num_ctx": 768,
"num_gpu": 0,
"num_thread": 8,
"repeat_penalty": 1.1,
"temperature": 0.7,
"top_k": 40,
"top_p": 0.95
}