Importer Note: Loads even when offloading, and I can say that works. I was able to set 256k, go way over VRAM, and the model loaded and responded. Yes, it was slower but not painful.

Card: 5060ti 16GB.

Quant: 4_0
Chat template/import bug? https://deltakitsune.org/report-a-bug/

Model: https://huggingface.co/GrootTeam/Groot-19B-A3B-261023
GGUF: https://huggingface.co/bartowski/GrootTeam_Groot-19B-A3B-261023-GGUF

Groot is a family of open-weight reasoning models for complex real-world tasks. The models are compact and easy to deploy, strong on reasoning and tool-using workloads, and efficient with token budgets.
Groot-19B-A3B-261023 has 19B total parameters and activates about 3B per token. It starts from Qwen3.6-35B-A3B and was post-trained on fewer than 0.1B tokens of open-source data. It is the smallest model in its comparison group, yet it scores higher than the larger Qwen3.6-35B-A3B and Gemma-4-26B-A4B, and it gives shorter answers than both.
Where it runs. We release BF16, FP8, NVFP4 and GGUF (Q4_K_M) weights. The device fits below are estimated from memory capacity. BF16 runs on a single 48GB or 80GB GPU (RTX 6000 Ada, A100, H100). FP8 and NVFP4 run the full 256K context on one RTX 5090. GGUF runs on everyday consumer cards: 128K on a 16GB RTX 4060 Ti, 256K on a 24GB RTX 4090 or 3090. It also runs 256K on 128GB unified-memory machines: NVIDIA DGX Spark and other GB10 systems, RTX Spark laptops, AMD Strix Halo mini PCs and Mac M5 Max. Compared with Qwen3.6-35B-A3B: on a 16GB card Groot runs 128K while Qwen3.6 does not load, and on an RTX 5090 Groot runs 256K while Qwen3.6 runs 64K.

Model size vs. capability

The chart compares total model parameters with the unweighted average across all 16 benchmarks. Smaller models appear farther left and higher average scores appear higher. Groot-19B-A3B uses 45.7% fewer total parameters than Qwen3.6-35B-A3B and 26.9% fewer than Gemma-4-26B-A4B, while averaging 41.44 compared with 38.08 and 30.05. This places it ahead of both larger models on the size and capability trade-off.

Benchmarks

Accuracy. Groot-19B-A3B-261023 beats both Qwen3.6-35B-A3B and Gemma-4-26B-A4B on 12 of the 16 benchmarks, for example Terminal Bench 2.1 (52.10 vs 43.82 / 26.52), NL2Repo-Bench (35.65 vs 28.77 / 16.55) and LongBench v2 (62.03 vs 48.12 / 47.99).
Token budget. Across these 16 evaluation runs, Groot-19B-A3B has the shortest average output among all 12 measured models: 23.1% fewer output tokens than Qwen3.6-35B-A3B and 65.6% fewer than Gemma-4-26B-A4B. Read the token totals together with the accuracy results above, since a smaller token budget is useful only when task quality is maintained.
Type: Natively multimodal Mixture-of-Experts causal language model
Total Parameters: ~19B, plus a 0.4B MTP layer
Activated Parameters: ~3B per token
Layers: 40 (10 full-attention, 30 linear-attention)
Hidden Size: 2048
Experts: 128 routed + 1 shared, 8 routed experts activated per token
Context Length: 262,144 natively, extensible to ~1M tokens with YaRN
Precision: BF16, with FP8, NVFP4 and GGUF releases available
Multi-Token Prediction: 1 layer

Citation

@misc{groot2026, title = {Scaling Down LLMs for Broader Deployment}, author = {The Groot Team}, year = {2026}, }