oamazonasgabriel/ nemotron-nano-9b-v2:bf16

37 6 days ago

Q4_K_M and BF16 quantizations of NVIDIA Nemotron Nano 9B v2, NVIDIA’s open 9B reasoning model. Q4 quantized locally from the BF16 source and tuned for a single 16 GB GPU card with maximum KV cache.

tools thinking
... /
params
4198019c2cb1 · 112B
{
"num_ctx": 131072,
"num_gpu": 99,
"num_predict": 4096,
"repeat_penalty": 1,
"temperature": 0.7,
"top_k": 40,
"top_p": 0.95
}