batiai/ nemotron3-nano:q5

57 Downloads Updated 2 months ago

ollama run batiai/nemotron3-nano:q5

curl http://localhost:11434/api/chat \
  -d '{
    "model": "batiai/nemotron3-nano:q5",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

from ollama import chat

response = chat(
    model='batiai/nemotron3-nano:q5',
    messages=[{'role': 'user', 'content': 'Hello!'}],
)
print(response.message.content)

import ollama from 'ollama'

const response = await ollama.chat({
  model: 'batiai/nemotron3-nano:q5',
  messages: [{role: 'user', content: 'Hello!'}],
})
console.log(response.message.content)

Details

Updated 2 months ago

2 months ago

72184ca3ce3d · 26GB ·

model

archnemotron_h_moe

·

parameters31.6B

·

quantizationQ5_K_M

26GB

system

You are a helpful AI assistant.

31B

params

{ "num_ctx": 131072, "stop": [ "<extra_id_1>", "<|endoftext|>" ], "t

97B

Readme

Nemotron 3 Nano 30B-A3B — Quantized by BatiAI

NVIDIA’s NemotronH MoE (Mamba+Attention hybrid). 30B params, only 3B active per token. Quantized directly from official NVIDIA weights.

Models

Tag	Size	RAM target	Use Case
iq3	17GB	24GB Mac	Compact MoE reasoning
iq4	17GB	32GB Mac	Recommended
q5	25GB	36GB+ Mac	Highest quality

Quick Start

ollama run batiai/nemotron3-nano:iq4

Why Nemotron 3 Nano?

30B-A3B MoE — 128 experts, 8 active per token, only 3B compute per token
Hybrid Mamba + Attention — efficient long-context state-space + accurate short-context attention
Reasoning + agentic — trained for tool use and structured outputs
NVIDIA Open Model License — commercial-friendly

RAM Requirements

Your Mac RAM	iq3 (17GB)	iq4 (17GB)	q5 (25GB)
16GB	⚠️ Heavy swap	⚠️ Heavy swap	❌
24GB	✅	✅	❌
32GB	✅ Fast	✅ Recommended	⚠️ Tight
36GB	✅	✅	✅
48GB+	✅	✅	✅ Headroom

For Other Macs

Your Mac	Recommended
16GB	`batiai/gemma4-e4b:q4`
24GB	`batiai/nemotron3-nano:iq3` (this)
32GB	`batiai/nemotron3-nano:iq4` (this, recommended)
36GB	`batiai/qwen3.5-35b:iq4`
48GB	`batiai/gemma4-31b:iq4` or `batiai/nemotron3-nano:q5`
128GB	`batiai/minimax-m2.7:iq3` (229B Dense)

Why BatiAI?

Quantized directly from official NVIDIA weights
imatrix calibrated (200 chunks wikitext-2) for IQ-quant quality
Verified on real Mac hardware
BatiAI metadata signed (general.author=BatiAI)

Built for BatiFlow

Free, on-device AI automation for Mac. 5MB app, 100% local, unlimited.

https://flow.bati.ai

Benchmarks

Machine	Quant	Cold start	Prompt eval	Token gen	Tested
MacBook Pro M4 Max 128GB	IQ3_XXS	1.599s	208.95 t/s	86.15 t/s	2026-05-03
MacBook Pro M4 Max 128GB	IQ4_XS	1.589s	206.43 t/s	88.77 t/s	2026-05-03
MacBook Pro M4 Max 128GB	Q5_K_M	5.036s	179.26 t/s	75.82 t/s	2026-05-03

# Nemotron 3 Nano 30B-A3B — Quantized by BatiAI

NVIDIA's NemotronH MoE (Mamba+Attention hybrid). 30B params, only 3B active per token.
Quantized directly from [official NVIDIA weights](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16).

## Models

| Tag | Size | RAM target | Use Case |
|-----|------|------------|----------|
| **iq3** | **17GB** | 24GB Mac | Compact MoE reasoning |
| **iq4** | **17GB** | 32GB Mac | **Recommended** |
| **q5** | **25GB** | 36GB+ Mac | Highest quality |

## Quick Start

```
ollama run batiai/nemotron3-nano:iq4
```

## Why Nemotron 3 Nano?

- **30B-A3B MoE** — 128 experts, 8 active per token, only 3B compute per token
- **Hybrid Mamba + Attention** — efficient long-context state-space + accurate short-context attention
- **Reasoning + agentic** — trained for tool use and structured outputs
- **NVIDIA Open Model License** — commercial-friendly

## RAM Requirements

| Your Mac RAM | iq3 (17GB) | iq4 (17GB) | q5 (25GB) |
|-------------|-----------|-----------|----------|
| 16GB | ⚠️ Heavy swap | ⚠️ Heavy swap | ❌ |
| **24GB** | **✅** | **✅** | ❌ |
| **32GB** | **✅ Fast** | **✅ Recommended** | ⚠️ Tight |
| **36GB** | ✅ | ✅ | **✅** |
| 48GB+ | ✅ | ✅ | ✅ Headroom |

## For Other Macs

| Your Mac | Recommended |
|----------|-------------|
| 16GB | `batiai/gemma4-e4b:q4` |
| 24GB | `batiai/nemotron3-nano:iq3` (this) |
| 32GB | `batiai/nemotron3-nano:iq4` (this, recommended) |
| 36GB | `batiai/qwen3.5-35b:iq4` |
| 48GB | `batiai/gemma4-31b:iq4` or `batiai/nemotron3-nano:q5` |
| 128GB | `batiai/minimax-m2.7:iq3` (229B Dense) |

## Why BatiAI?

- Quantized directly from official NVIDIA weights
- imatrix calibrated (200 chunks wikitext-2) for IQ-quant quality
- Verified on real Mac hardware
- BatiAI metadata signed (`general.author=BatiAI`)

## Built for BatiFlow

Free, on-device AI automation for Mac. 5MB app, 100% local, unlimited.

https://flow.bati.ai

## Benchmarks

| Machine | Quant | Cold start | Prompt eval | Token gen | Tested |
|---------|-------|------------|-------------|-----------|--------|
| MacBook Pro M4 Max 128GB | IQ3_XXS | 1.599s | 208.95 t/s | 86.15 t/s | 2026-05-03 |
| MacBook Pro M4 Max 128GB | IQ4_XS | 1.589s | 206.43 t/s | 88.77 t/s | 2026-05-03 |
| MacBook Pro M4 Max 128GB | Q5_K_M | 5.036s | 179.26 t/s | 75.82 t/s | 2026-05-03 |

Paste, drop or click to upload images (.png, .jpeg, .jpg, .svg, .gif)