🧠 Qwen-3.8-27B-AgentCoder-Q5-K-M

A Fine-Tuned Model for Enhanced Tool Calling, Code Generation, and Reasoning

• Author: Bruno Pistone
• Repository: br1-pist/Qwen3.8-27B-AgentCoder
• License: Apache 2.0
• Base model: Qwen/Qwen3.8-27B

Model Description

Qwen-3.8-27B-AgentCoder-Q5-K-M is a fine-tuned version of the Qwen/Qwen3.8-27B model, optimized for: - 🧮 Complex reasoning tasks - 🧰 Tool calling - 💻 Code generation

The model was developed through sequential fine-tuning, followed by a Direct Preference Optimization (DPO) post-training stage to improve alignment, coherence, and reasoning accuracy.

Highlights

• Post-trained with DPO using chosen/rejected pairs for better alignment
• Excellent balance between tool use, code generation, and reasoning

🚀 Direct Use

Qwen-3.8-27B-AgentCoder-Q5-K-M can be used directly for: - ✅ Tool calling in complex reasoning tasks - ✅ Code generation for Python, JS, and other languages - ✅ Multi-domain reasoning (math, logic, Q&A)

⚠️ Out-of-Scope Use

• ❌ Highly sensitive or confidential data
• ❌ Domains requiring expert-level specialization
• ❌ Tasks where full explainability is mandatory

🧠 Training Details

Training Procedure

Phase 1 — Post-Training: Reinforcement Learning from AI Feedback (RLAIF) with Group Relative Policy Optimization (GRPO)

After sequential fine-tuning and preference alignment, the model underwent a GRPO phase in which an LLM judge — not a static preference dataset — supplied the reward signal. For each prompt the policy samples a group of completions, the judge scores each one against a rubric, and the advantage is computed relative to the group mean, so no value network is required.

Reward model (RLAIF) - Judge: moonshotai/Kimi-K3 via the Amazon Bedrock Converse API - Rubric-scored 0–10, normalised to 0–1; rewards evidence-gathering, correct tool choice, root-cause reasoning and verification, and penalises speculation, wrong-tool calls and symptom patching - Unscorable completions (content filter, malformed verdict) are excluded from the group baseline rather than scored zero

GRPO parameters - Generations per prompt (group size): 8 - Beta (KL coefficient): 0.04 - Loss type: dapo - Reward scaling: group (unit variance within each group) - Sampling temperature: 1.0 - Max completion length: 640 tokens, truncated completions masked out of the loss

Training parameters - Learning rate: 1e-5, cosine schedule with a 10% floor - Warmup steps: 46 - Per-device batch size: 2 × 16 GPUs × gradient accumulation 4 = 128 completions (16 prompts) per step - Epochs: 1 → 230 optimiser steps - Max grad norm: 0.5 - LoRA: rank 16, alpha 32, dropout 0.1, applied to both the full-attention (q/k/v/o_proj) and linear-attention (in_proj_qkv, in_proj_z, out_proj) projections - Precision: bf16, FSDP full-shard - Hardware: 2 × ml.p5.48xlarge (16 × H100), 21h47m wall clock

GRPO Data

• 3,686 multi-turn, tool-call conversations (87% multi-turn; prompts left untruncated)
• Coding, general Q&A, data management, bash execution

Objective - Improve clarity, correctness, and helpfulness - Reduce hallucinations and verbosity

Outcome - Mean judge reward rose from 0.615 (first 25 steps) to 0.689 (final 25), +12.0% - Mean completion length and truncation rate both fell, i.e. the gain came from concision, not padding - Policy KL from the reference model rose from ~2.6e-4 to ~8.0e-3, confirming the policy actually moved

🖥️ Technical Specifications

Model Architecture

• Model type: Causal language model
  
• Parameters: 4.0B
  
• Context length: ~264K tokens
  
• Thinking mode: Enabled
  

Compute Infrastructure

Hardware - GPU: NVIDIA H100 (80 GB VRAM)
- System RAM: 2 TiB
- Memory per vCPU: 10.67 GiB

Software - Python: 3.12
- Transformers: 5.3.0
- Libraries: bitsandbytes, safetensors, torch, trl, scikit-learn, tokenizers, psutil, py7zr

🧭 Recommendations

• Tool use accuracy depends on task complexity
  
• Code generation may occasionally produce minor syntax issues
  
• Reasoning strongest in structured, logical, and mathematical contexts
  
• Avoid using this model for confidential or safety-critical applications