oamazonasgabriel/ nemotron-nano-9b-v2:q4-km-16gbGPU

34 5 days ago

Q4_K_M and BF16 quantizations of NVIDIA Nemotron Nano 9B v2, NVIDIA’s open 9B reasoning model. Q4 quantized locally from the BF16 source and tuned for a single 16 GB GPU card with maximum KV cache.

tools thinking
... /
params
c8e8b882da7d · 168B
{
"num_ctx": 32768,
"num_gpu": 99,
"num_predict": 4096,
"repeat_penalty": 1,
"stop": [
"<|im_start|>",
"<|im_end|>"
],
"temperature": 0.7,
"top_k": 40,
"top_p": 0.95
}