Benchmarks

Overview

OpenGCM-v2 is a reasoning-focused 9B parameter model developed by NitrAI. The model is built on top of the next-generation Qwen3.5-9B base model, which features state-of-the-art architectures and a 262k context window.

The goal of OpenGCM-v2 is to distill complex coding-agent trajectories, multi-step math logic, and system-level reasoning from frontier LLMs (GPT-5.5, Claude-Fable-5, and GLM-5.2) into a highly efficient, lightweight consumer-hardware-friendly model.

Distillation Mixture

To prevent VRAM paging bottlenecks during training on consumer GPUs, the dataset was strictly audited, cleaned of outlier long sequences, and downsampled to fit an optimal token budget. The final fine-tuning dataset consists of 597 high-signal QA items containing 904,466 tokens in total.

Dataset Composition Breakdown

Training Methodology

The training was performed locally on a single consumer GPU setup using the Unsloth library (leveraging optimized Triton fused kernels for training acceleration) and DoRA (Weight-Decomposed Low-Rank Adaptation).

Hyperparameters & Settings

• Base Model: Qwen/Qwen3.5-9B

• PEFT Method: DoRA (Weight-Decomposed LoRA)

  • Rank ®: 64
  • Alpha (α): 128
  • Target Modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj

• Max Sequence Length: 2048 tokens

• Optimizer: adamw_8bit

• Learning Rate: \(1.5 \times 10^{-5}\)

• Warmup steps: 110 (10% of training steps)

• Training Steps: 1100

• Batch Size: 1 (Gradient Accumulation Steps = 4, effective batch size = 4)

• Precision: bfloat16

  Evaluation & Performance

We evaluated OpenGCM-v2 on a suite of hard benchmarks (AIME, SWE-bench Pro, GPQA, MMMU Pro, LiveCodeBench) and compared it to gemma4-coder-fable5:

Key Strengths & Weaknesses

• Strengths:
  • Exceptional math reasoning and step-by-step logical decomposition (solved AIME sequence problems perfectly).
  • Highly capable of localized code reasoning and bug patch verification (SWE-bench).
• Limitations:
  • Occasional instability / context drift during extremely long inference generation where it might switch focus or hallucinate the task constraints. A lower temperature (e.g. 0.2 or 0.4) and structured system prompts are recommended.

Usage

Ollama Configuration

You can easily run this model locally in Ollama by creating a Modelfile with the following configuration:

FROM ./opengcm_Q6_K.gguf
TEMPLATE """{{ if .System }}<|im_start|>system
{{ .System }}<|im_end|>
{{ end }}{{ if .Prompt }}<|im_start|>user
{{ .Prompt }}<|im_end|>
{{ end }}<|im_start|>assistant
{{ .Response }}<|im_end|>
"""
PARAMETER stop "<|im_start|>"
PARAMETER stop "<|im_end|>"

Transformers Inference Example

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "NitrAI/OpenGCM-v2"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)
messages = [
    {"role": "system", "content": "You are a helpful assistant. Use step-by-step reasoning enclosed in <think>...</think> tags before answering."},
    {"role": "user", "content": "Solve: a_1 = 1, a_2 = 3. For n >= 3, a_n is the smallest positive integer that hasn't appeared yet and is coprime to a_{n-1}. Find a_100."}
]
inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt"
).to(model.device)
outputs = model.generate(
    inputs,
    max_new_tokens=1024,
    temperature=0.4,
    do_sample=True
)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Citation & Acknowledgements

Special thanks to the open-source community, Hugging Face, Unsloth, and the creators of the original source datasets: * ansulev/GPT-5.5-Thinking-Max-Distill-25k * AletheiaResearch/GLM-5.2-Agent * Glint-Research/Fable-5-traces