dlasher/ Nemotron-3.5-Lightning-30B-A3B:IQ4_NL

4 yesterday

pulled as BF16, requant to IQ4_NL, with combined_en_huge as dataset (English + Coding focus) ~25% faster than unsloth UD-IQ4_NL

tools thinking 
ollama run dlasher/Nemotron-3.5-Lightning-30B-A3B:IQ4_NL

Details

yesterday

18abf4604cb8 · 19GB

model
nemotron_h_moe
·
32.9B
·
IQ4_NL
template
{{- if .System }}<|im_start|>system {{ .System }}<|im_end|> {{- end }} {{- range .Messages }} {{- if
params
{ "min_p": 0.05, "num_batch": 2048, "num_ctx": 262144, "presence_penalty": 0, "r

Readme

==================================================
 PERPLEXITY QUALITY SCORECARD
 Base Model PPL      : 6.0896 (16-bit reference)
 Quantized Model PPL : 6.1728 (IQ4_NL target)
 Perplexity Drift    : +0.0832 (+1.37%)
 Quality Assessment  : [PASS] EXCELLENT (Near-Lossless / Gold Standard)
==================================================

image.png

What is Nemotron? NVIDIA Nemotron™ is a family of open models with open weights, training data, and recipes, delivering leading efficiency and accuracy for building specialized AI agents.

Description NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16 is a large language model (LLM) trained by NVIDIA. This is the full-precision (BF16) release of Nemotron 3.5 Lightning — the reference weights of the model, intended primarily as the starting point for customization: post-training (SFT, RL, distillation), domain adaptation, and producing your own quantized or GGUF variants. For latency- and throughput-optimized inference, use the NVFP4 release instead.

The model employs a hybrid Mixture-of-Experts architecture, utilizing interleaved Mamba-2 and MoE layers, along with select Attention layers. The Lightning 3.5 model is released alongside a number of speculative decoding methods for faster text generation. The model has 3B active parameters and 30B parameters in total.

This model is ready for commercial use.

SOURCES: