Ornith-1.0-9B Q2_K — Optimized for 8GB VRAM

A 2-bit quantization of Ornith-1.0-9B, a state-of-the-art open-source agentic coding model from DeepReinforce. Optimized to run on GPUs with 8GB VRAM while preserving maximum intelligence.

Why This Exists

Ornith-1.0-9B is a reasoning model trained with RL to write and debug code autonomously. At Q4_K_M (5.6 GB), it barely fits 8GB VRAM — leaving only ~1.2 GB for KV cache, limiting context to ~16K tokens.

Q2_K compresses to 3.6 GB, freeing ~3.5 GB VRAM for KV cache, enabling 262K context on 8GB cards, multiple parallel subagents, or serving alongside a draft model for speculative decoding.

Benchmarks

Ornith-1.0-9B Q2_K retains strong agentic coding capabilities:

Q2_K preserves the model’s reasoning chain and tool-calling abilities while reducing model size by 37%.

Hardware Requirements

Runs comfortably on RTX ⁴⁰⁶⁰⁄₄₀₇₀ (8GB), RTX 3060 (12GB), any 8GB GPU.

Recommended Settings

ollama run venkataparswanadh/ornith:9b-q2

# Or with custom parameters:
ollama run venkataparswanadh/ornith:9b-q2
 --num-ctx 262144 --temperature 0.6 --top-k 20 --top-p 0.95

For optimal agentic coding performance, use with: - OpenCode: opencode (uses local Ollama provider) - Continue.dev: Point to Ollama’s OpenAI-compatible endpoint - OpenHands: Via LiteLLM with openai/ornith:9b-q2

Speculative Decoding

Pair with Qwen3.5-0.8B as a draft model via llama.cpp for ~35-55% faster generation:

llama-server \
  -m ornith-1.0-9b-Q2_K.gguf \
  -md qwen3.5-0.8b-Q4_K_M.gguf \
  --spec-draft-n-max 5

Technical Details

• Base model: Ornith-1.0-9B (Qwen3.5/Gemma4 architecture)
• Quantization: Q2_K (2.5 bits per weight average)
• Format: GGUF
• License: MIT
• Source quantization: Q6_K → Q2_K via llama-quantize

Credits

• DeepReinforce AI for Ornith-1.0
• llama.cpp for the GGUF format and quantization tools