44 2 weeks ago

Gemma 4 26B A4B IT QAT Unsloth Dynamic 2.0 GGUF MoE

tools thinking
ollama run rbrisita/gemma-4-26b-a4b-it:qat-ud-q4_k_xl

Applications

Claude Code
Claude Code ollama launch claude --model rbrisita/gemma-4-26b-a4b-it:qat-ud-q4_k_xl
OpenCode
OpenCode ollama launch opencode --model rbrisita/gemma-4-26b-a4b-it:qat-ud-q4_k_xl
Hermes Agent
Hermes Agent ollama launch hermes --model rbrisita/gemma-4-26b-a4b-it:qat-ud-q4_k_xl
OpenClaw
OpenClaw ollama launch openclaw --model rbrisita/gemma-4-26b-a4b-it:qat-ud-q4_k_xl

Models

View all →

Readme

Instruction-Tuned Mixture-of-Experts with 4B Active Parameters. Quantization-Aware Training (QAT) reduces memory requirements while preserving model quality. This model contains a draft model that can be turned off by setting draft_num_predict=0.

Modelfile parameters taken from:

  1. Unsloth Gemma 4

Supporting Details:

  1. Gemma 4 QAT
  2. Unsloth Dynamic 2.0 GGUFs

Tested on:

 OS: Ubuntu 26.04 resolute
 Kernel: x86_64 Linux 7.0.0-30-generic
 Shell: bash 5.3.9
 Disk: 1TB
 CPU: 12th Gen Intel Core i7-12700H @ 20x 4.6GHz
 RAM: 64GB 
 GPU: NVIDIA GeForce RTX 3050 Laptop GPU
 GPU RAM: 4GB