novaforgeai/ qwen2.5-3b:q4km

115 1 month ago

Qwen 2.5 3B – NovaForgeAI Edition Qwen 2.5 3B – NovaForgeAI Edition is a CPU-optimized, low-latency LLM designed for fast local inference on low-end and mid-range systems.

... /
params
87189f4c9e29 · 123B
{
"num_batch": 512,
"num_ctx": 2048,
"num_gpu": 0,
"num_thread": 8,
"repeat_penalty": 1.1,
"temperature": 0.7,
"top_k": 40,
"top_p": 0.95
}