base_model: Qwen/Qwen3-4B-Instruct-2507 library_name: peft pipeline_tag: text-generation license: apache-2.0 language: - en tags: - code - compression - coding-agent - lora - sft - claude-code - cursor model-index: - name: paritok-4b-v1 results: - task: type: text-compression dataset: name: SWE-bench Verified type: swe-bench-verified metrics: - name: Quality retained type: quality-retained value: 86.5 - name: Compression rate type: compression-rate

value: 25.7

Paritok-4B-v1

The first open-source compression model trained specifically for coding agents.
Cut Claude / GPT input token bills by up to 95% while retaining 86.5% of full-context solve quality on SWE-bench Verified.

🔥 TL;DR

✨ Why Paritok?

• 🎨 Code-native. Trained end-to-end on 45K real coding-agent trajectories (file_read, bash_command, log_output…). Preserves function names, imports, paths, and error strings while compressing.
• 🚀 ~74% fewer tokens on typical workloads — up to 95% on heavy long-session traffic.
• 💰 Up to 95% off your input token bill on Claude Sonnet / GPT-4 (74% typical). Long-session teams save thousands per month.
• 🎯 86.5% of full-context solve quality retained on SWE-bench Verified — matching gpt-4.1-mini as compressor at less than half the token spend.
• 🪶 Small, fast, self-hostable. 4B LoRA adapter, bf16, single 24GB GPU. No SaaS, no lock-in, no per-token fee.
• 🔓 Fully open. Apache 2.0 weights, reproducible pipeline, real end-to-end benchmarks.

🚀 Quick Start

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
BASE_MODEL = "Qwen/Qwen3-4B-Instruct-2507"
ADAPTER    = "paritok/paritok-4b-v1"
tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL)
base = AutoModelForCausalLM.from_pretrained(
    BASE_MODEL, torch_dtype=torch.bfloat16, device_map="auto"
)
model = PeftModel.from_pretrained(base, ADAPTER)
model.eval()
# Wrap your code as [SEG id=1 kind=file_read] ... [/SEG]
user_msg = "[SEG id=1 kind=file_read]\n<your code here>\n[/SEG]"
prompt = tokenizer.apply_chat_template(
    [{"role": "system", "content": "<full system prompt — see GitHub>"},
     {"role": "user",   "content": user_msg}],
    tokenize=False, add_generation_prompt=True,
)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=1024, do_sample=False)
print(tokenizer.decode(out[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))

📎 Full runnable example + system prompt + reproduction pipeline: GitHub → Paritok-official/paritok-4b-v1

📊 SWE-bench Verified — Head-to-head

_{¹ Quality retained = compressor solve rate ÷ uncompressed baseline solve rate. Higher is better.}

Paritok compresses ~2× harder than gpt-4.1-mini on the same task while keeping the same solve rate — the only open-source entry trained end-to-end on real coding-agent trajectories.

💰 Cost Impact (Claude Sonnet, $3/M input tokens)

10-person team, 3-month project → save ~$10K on API bills.

📖 Citation

@misc{paritok2026,
  author       = {Paritok Team},
  title        = {Paritok: Fine-tuned Compression for AI Coding-Agent Context},
  year         = {2026},
  publisher    = {GitHub},
  howpublished = {\url{https://github.com/Paritok-official/paritok-4b-v1}},
}

📄 License

Apache 2.0 — see LICENSE. Base model (Qwen3-4B-Instruct-2507) is under its own license; please review before commercial deployment.

GitHub · paritok.com · Contact