KAT-Dev-72B

A 72B-parameter coding model built for agentic software engineering, issue resolution, multi-file refactors, and tool-driven development, quantized to GGUF for local use with Ollama.

🚀 Overview

KAT-Dev-72B is a GGUF quantization of Kwaipilot/KAT-Dev-72B-Exp, the experimental, large-scale reinforcement-learning version of Kwaipilot’s KAT-Coder model. It is purpose-built for software engineering and agentic coding workflows: reading and editing real repositories, resolving GitHub-style issues, and operating inside tool-use scaffolds.

The model is based on the Qwen2 architecture (Qwen2.5-72B-class: 80 layers, 8192 hidden size, 152064 vocab) and supports a 128K-token context window, making it well suited to working across large codebases and long agent trajectories. Multiple quantizations are provided so you can trade quality for memory on everything from a single 24 GB GPU to a multi-GPU or high-RAM workstation.

📊 Benchmarks

Note: The 74.6% figure is the authors’ result for the full-precision base model under the SWE-agent harness. Scores for quantized GGUF builds and other scaffolds will differ. Treat it as an upstream reference point, not a guarantee for every quant or setup.

🎯 Key Features

• Agentic coding focus, trained with large-scale agentic reinforcement learning for real-world issue resolution and multi-step tool use.
• Strong SWE-Bench performance, 74.6% on SWE-Bench Verified (SWE-agent scaffold) for the base model.
• 128K context, handle large repositories, long diffs, and extended agent histories.
• Multilingual code support, Python, JavaScript, TypeScript, Java, C++, Go, Rust, and more.
• Flexible quantization, six GGUF builds from IQ2_XXS (25 GB) to Q8_0 (77 GB).
• Apache-2.0 licensed, permissive use, including commercial.

🏷️ Available Versions

All versions use a 128K-token context window. Sizes are from the live Ollama tags page.

💻 Quick Start

Pull and run the recommended balanced build:

ollama run richardyoung/kat-dev-72b:iq4_xs

Pull a specific quantization:

# Highest quality
ollama run richardyoung/kat-dev-72b:q6_k

# Smallest footprint
ollama run richardyoung/kat-dev-72b:iq2_xxs

Use it from the API:

curl http://localhost:11434/api/generate -d '{
  "model": "richardyoung/kat-dev-72b:iq4_xs",
  "prompt": "Write a Python function that merges two sorted lists.",
  "options": { "temperature": 0.6 }
}'

Tip: The upstream model uses temperature 0.6 for agentic evaluation. Start there for coding and agent tasks.

🛠️ Use Cases

• Agentic software engineering, resolving issues, implementing features, and running multi-step edit/test loops inside scaffolds like SWE-agent.
• Code generation, functions, modules, and boilerplate across many languages.
• Refactoring & code review, restructuring and explaining large, multi-file changes.
• Debugging, diagnosing failures and proposing fixes with repository context.
• Repository Q&A, answering questions over large codebases within the 128K window.

📋 System Requirements

Approximate memory needed to load the weights (plus headroom for the KV cache, which grows with context length):

• A large KV cache at 128K context can add many GB on top of the weight sizes above, size memory accordingly.
• Ollama runs on Apple Silicon (unified memory), NVIDIA/AMD GPUs, and CPU; CPU-only inference at this scale is slow.

🔧 Technical Details

• Base model: Kwaipilot/KAT-Dev-72B-Exp
• Architecture: Qwen2 (Qwen2ForCausalLM), Qwen2.5-72B-class
• Parameters: ~73B (BF16 base)
• Context length: 128K tokens (max_position_embeddings = 131072)
• Layers / hidden size / vocab: 80 / 8192 / 152064
• Training: mid-training, SFT + RFT, then large-scale agentic reinforcement learning (experimental RL version of KAT-Coder)
• Format: GGUF (six quantizations)
• License: Apache-2.0

⚠️ Disclaimer

This is a community GGUF quantization of an experimental research model. Quantization reduces precision and can lower output quality relative to the full-precision base; smaller builds (IQ2/IQ3) trade more quality for size. Benchmark figures are upstream results for the base model under a specific scaffold and may not reproduce for quantized builds or different harnesses. Always review and test generated code before using it in production. The model and its outputs are provided “as is,” without warranty.

🙏 Acknowledgments

• Kwaipilot, for creating and open-sourcing KAT-Dev-72B-Exp and the KAT-Coder line.
• Qwen team, for the Qwen2 architecture underlying this model.
• llama.cpp, for GGUF quantization tooling.
• Ollama, for the local model runtime and distribution.

Built & maintained by Richard Young · DeepNeuro