pointbreaklab/ knot-delta:latest

6 days ago 

ollama run pointbreaklab/knot-delta

Details

6 days ago

b5d4fd464e51 · 5.6GB ·

model
qwen35
·
8.95B
·
Q4_K_M
system
You are Knot Delta, a repository-intelligence reviewer. You are given ONLY a git diff (no commit mes
params
{ "num_ctx": 16384, "temperature": 0.2, "top_p": 0.8 }

Readme

Knot Delta

Explains what changed, on your machine. Give it a git diff, get back a grounded report: what moved, why, and the risk — with every claim pinned to a real file and line in the diff. A 9B model, runs fully local.

ollama run pointbreaklab/knot-delta "$(git diff)"

Output is JSON: a change_type, behavioral changes, and evidence citing real paths from the diff.

Part of the Knot AI family with pointbreaklab/knot-scribe (which writes your commit messages). Knot Delta is the change-intelligence layer inside Knot.

Capability scorecard (measured, honest)

Every number is scored against objective ground truth (real git history) and reported beside the baseline a lazy model would score — a rate a null baseline matches is not a capability. Frozen held-out set: 325 commits across 9 unseen real repos (vite, vue, svelte, astro, ionic, gumroad, OpenDream, radicle, vllm). No single “intelligence score” is quoted.

Does it actually read intent from code?

The one capability that clears its baseline. change_type is inferred from the diff alone (no commit message given).

Change-type macro-F1 (235 commits)

lazy baseline (majority class)  █                         0.03
Knot Delta                      ████████████████          0.41    = 12.9x base rate

Is it grounded, or does it hallucinate?

325 held-out commits, 9 unseen repos            (bar = 0-100%)

cited files that exist in the diff  ████████████████████████████████████████  100%
reports fully grounded              ███████████████████████████████████████   98.5%
exact-line precision  (lenient)     █████████████████████████████████████▌    94%
exact-line precision  (strict)      █████████████████████████▌                64%
  • Never hallucinates a file — 100% of cited files are actually in the diff, on every repo tested (including Go, which is absent from its training data).
  • Always points at the right region (94% lenient).
  • Honest limit: it pins the exact line 52–74% of the time. It reliably knows which file and which part changed; the precise line is still improving.

Holds up on unseen repos and languages

repo language verified exact-line precision
flask Python 11⁄12 0.74
gin Go (unseen) 12⁄12 0.70
ripgrep Rust 12⁄12 0.52
frozen-325 JS/TS + 320⁄325 0.64

A note on comparisons

Knot Delta produces a structured, diff-grounded report — a different task from open-ended chat, so a fair number is its lift over a null baseline on the same ground truth (above: 12.9× chance on change-type; 100% vs any model that guesses file paths). It is not benchmarked against general chat models, because they don’t do this task in the same schema, and an unmeasured comparison would be exactly the base-rate laundering this scorecard avoids.

Model

  • Base: Qwen3.5-9B, fine-tuned for change reports
  • Format: text-only Q4_K_M GGUF (~5.2 GB), runs on CPU or GPU
  • Also on: Hugging Face
  • License: Apache-2.0