turintech/ nemotron-3.5-bolt-e4b-artemis:iq2_xxs

559 1 week ago

Text-only Nemotron 3.5 Bolt E4B GGUF files, thinking off as measured. Default: q3_k_xl. TurinTech models, built with Artemis. Measured Q3_K_XL: Mean KLD 0.0414, Same top p 92.98% vs BF16 on held-out OASST1 chat; protocol rtx3060-heldout-2026-10-28.

tools thinking 
ollama run turintech/nemotron-3.5-bolt-e4b-artemis:iq2_xxs

Details

1 week ago

ba7646b7999d · 3.1GB

model
gemma4
·
7.52B
·
Q4_K_M
license
OpenMDW License Agreement, version 1.1 (OpenMDW-1.1) By exercising rights granted to you under this
license
Nemotron 3.5 Bolt E4B Artemis GGUF The GGUF files in this repository are a modified version of NVIDI
params
{ "num_ctx": 8192, "stop": [ "<bos>", "<|turn>", "<turn|>",
template
{{ if .System }}<bos><|turn>system {{ .System }}<turn|> {{ end }}{{ if .Prompt }}<|turn>user {{ .Pro

Readme

Nemotron 3.5 Bolt E4B Artemis

NVIDIA’s Nemotron 3.5 Bolt E4B for chat, tool calls and maths on a small GPU or a CPU, in files the size of Unsloth’s.

  • 4.22 GB download for the default file, Nemotron-3.5-Bolt-E4B-Artemis-Q3_K_XL.gguf.
  • Held-out KL from the original (wiki, chat, code; RTX 3060): .0574, .0414, .0376, against .0752, .0620, .0504 for Unsloth’s same-size UD-Q3_K_XL. CPU task suite, on two texts that also chose the files: .0661 / .0416 against .0816 / .0609.

Run with Ollama

ollama run turintech/nemotron-3.5-bolt-e4b-artemis:q3_k_xl --think=false

The card’s numbers have thinking off; Ollama turns it on. Keep --think=false, or send "think": false ("reasoning_effort": "none" on /v1). Prompts go through Ollama’s Gemma 4 renderer, not the file’s NVIDIA template. Checked with Ollama 0.40.13.

Results

Our default file against Unsloth’s file of the same size, NVIDIA’s own Q4_K_M and the original. KL: held-out chat text, RTX 3060. Tasks: CPU task suite on two Ryzen 9 5900X machines, where the first 200 tool calls and GSM8K are selection data.

File Size Closeness to original (KL, lower is better) Tool calls (of 400) GSM8K (of 250) IFEval (of 541)
Ours, Q3_K_XL 4.22 GB 0.0414 372 228 466
Unsloth UD-Q3_K_XL 4.22 GB 0.0620 372 228 468
NVIDIA Q4_K_M 6.18 GB 0.1047 369 231 471
BF16 original 15.05 GB reference 370 228 469

No significant difference on any task: no paired test separated our Q3_K_XL from these files. How it was tested.

Files

Every file we publish for this model. Sizes are download sizes, not memory.

File Size Use it for Ollama tag
Nemotron-3.5-Bolt-E4B-Artemis-Q3_K_XL.gguf 4.22 GB Default: the size of Unsloth’s UD-Q3_K_XL q3_k_xl (latest)
Nemotron-3.5-Bolt-E4B-Artemis-Q4_K_XL.gguf 5.06 GB Larger: the size of Unsloth’s UD-Q4_K_XL; the closest of ours to the original q4_k_xl
Nemotron-3.5-Bolt-E4B-Artemis-IQ3_XXS.gguf 3.70 GB Smallest; the size of Unsloth’s UD-IQ3_XXS iq3_xxs
  • The Ollama tag runs our package, which holds the same file: ollama run turintech/nemotron-3.5-bolt-e4b-artemis:<tag>.

Without a tag, -hf and hf.co fetch Nemotron-3.5-Bolt-E4B-Artemis-IQ3_XXS.gguf, not the default file, so always add the tag.

The Q3_K_XL and Q4_K_XL importance matrices are the same file. NVIDIA’s DFlash drafter is in NVIDIA’s own repository (see Run it).

Limitations

  • Further from the original by KL than our Q4_K_XL (.0661 / .0416 against .0375 / .0241 on the CPU).
  • Thinking on was not measured.
  • The numbers come from llama.cpp, not from Ollama or LM Studio.

Licence

OpenMDW-1.1 (licence), as released by NVIDIA. See the Hugging Face repository for LICENSE and NOTICE.