rb-nano is a 48M-parameter chat model trained from scratch on a single consumer GPU — tiny, fast, and CPU-friendly.

rb-nano

A 48M-parameter, GPT-2-style decoder-only transformer trained from scratch as part of the Leopard AI Model Suite. Small enough to run on CPU or any GPU; built as a learning/research model, not a production assistant.

ollama run rafi-dev/rb-nano

What it is

rb-nano is a tiny chat model pretrained on web text and instruction-tuned for short, single- and multi-turn conversations. At 48M parameters it sits well below the knowledge capacity of mainstream models, so treat it as a fast, lightweight demonstrator rather than a factual reference.

Architecture


Type	Decoder-only transformer (GPT-2 family)
Parameters	~48M
Embedding dim (`d_model`)	512
Layers	10
Attention heads	8
Context length	1024 tokens
Position embeddings	Learned
Norm / activation	LayerNorm, GELU-tanh
Attention	Combined QKV, SDPA (flash)
Head	Weight-tied to token embeddings
Tokenizer	ByteLevel BPE, 32k vocab
Format	GGUF, f16 (`gpt2` architecture)

Training

Pretrain — FineWeb-Edu (sample-10BT), ~50M tokens. Final val loss ≈ 3.44.
Finetune — Alpaca, Alpaca-cleaned, CodeAlpaca-20k, Dolly-15k, and ShareGPT (full multi-turn threads, loss masked to assistant turns only). Final val loss ≈ 2.67.

Prompt format

The model is trained on a simple user: / ai: turn format (Ollama’s chat template handles this automatically):

user: hello
ai: Hi there! How can I help you today?
user: what is python?
ai:

Recommended parameters

temperature   0.7
top_k         40
top_p         0.9
repeat_penalty 1.3

Limitations

Knowledge. At 48M params the model has very limited factual knowledge and will confidently hallucinate (made-up libraries, wrong dates, etc.). It cannot be a reliable source of facts.
Coherence. Good for short exchanges; longer or more technical answers drift.
Scope. English-centric, 1024-token context. Best for demos, experimentation, and edge/CPU inference — not production use.

License / attribution

Trained on publicly available datasets (FineWeb-Edu, Alpaca, Dolly, CodeAlpaca, ShareGPT). Review each dataset’s license before redistributing derived outputs.

Made with care

rb-nano was built by Rafi and Buddi — pretrained and finetuned from scratch on a single RTX 4070 (8 GB VRAM). It’s a passion project: proof that a coherent little chat model can be trained end-to-end on consumer hardware.

If you enjoy it and want to support more experiments like this, you can buy us a coffee ☕. Thank you for trying rb-nano — we hope you like it.