Abyssal/intent-classifier-general-v2

The first in the family that can decline. Define intents at prompt time, no retraining. It returns the single best-matching name, or none_of_the_above when nothing fits, refusing 84% of out-of-scope messages.

Recommended usage

Two modes, depending on whether you need out-of-scope rejection.

1. Reliable routing (always pick one of your intents). Pass your intent names as a JSON-schema enum in format. This grammar-constrains decoding so the answer is always one of your intents (never a hallucinated or out-of-list label):

curl http://localhost:11434/api/chat -d '{
  "model": "intent-classifier-general-v2:1.5b",
  "stream": false,
  "options": {"temperature": 0},
  "format": {"type": "string", "enum": ["refund", "tracking", "account"]},
  "messages": [{"role": "user", "content":
    "Candidate intents:\nrefund: wants money back\ntracking: where their order is\naccount: login or profile\n\nUser message: where is my package?\n\nAnswer with exactly one intent name from the list above."
  }]
}'
# -> "tracking"

2. Rejection (detect out-of-scope messages). Leave format off. In free generation the model returns none_of_the_above when no intent fits — the main thing a stock model cannot do (it always forces a pick). Enum mode removes this option, so use free generation when you need rejection:

curl http://localhost:11434/api/chat -d '{
  "model": "intent-classifier-general-v2:1.5b",
  "stream": false,
  "options": {"temperature": 0},
  "messages": [{"role": "user", "content":
    "Candidate intents:\nrefund: wants money back\ntracking: where their order is\n\nUser message: what time do you close on sundays?\n\nAnswer with exactly one intent name from the list above."
  }]
}'
# -> "none_of_the_above"

Getting the best results

• Write clear, specific descriptions. The model matches on the description, so this is the single biggest lever. Prefer tracking: track or locate a customer's order or shipment over a terse tracking: order status. Vague one- or two-word descriptions are the main cause of an occasional wrong none_of_the_above.
• Keep temperature 0 (the baked-in default) for deterministic, repeatable routing.
• Use the exact prompt shape shown above (Candidate intents: … User message: … Answer with exactly one intent name from the list above.) — it was trained on this format.
• Names are free-form — snake_case, CamelCase, hyphens or plain words all work, including names the model has never seen; it matches by description and echoes your name verbatim.

Under the hood

Base: LoRA fine-tune of Qwen2.5-1.5B-Instruct (Apache-2.0). Each training example presents a different set of intents with descriptions, so the model learns the skill “read whatever list you are given and match it” rather than a fixed taxonomy. It is also trained with a real none_of_the_above label (out-of-scope examples exclude the correct intent and its near-synonyms), and ~28% of examples are renamed to invented names so it matches by meaning instead of memorizing labels. Runs at temperature 0 and is fully deterministic.

Accuracy: — 6,076 calls, temperature 0, a 243-intent / 22-domain taxonomy with realistic 3-4 intent lists. In-scope accuracy is 94.0% with the enum constraint (87.8% free generation), and it rejects 92.6% of out-of-scope messages that a stock model would force into the list. It beats stock qwen2.5:1.5b on in-scope accuracy (95.7% vs 95.2%) while adding rejection the base model does not have (16.8%).

For the out-of-scope row, a low in-list number is the goal: the model escapes the list with none_of_the_above instead of guessing. Stock qwen2.5 stays in-list 83% of the time here (only 16.8% rejection).

Note on comparing versions. The numbers above are from the original 847-call / 87-category audit. Later models in the family are scored on the harder 6,076-call / 243-intent / 22-domain corpus, and the two are not comparable — the newer corpus uses tighter distractors. Re-scored on that corpus, v2 measures 87.8% free-gen and 94.0% enum in-scope accuracy with 83.8% rejection. Use those figures when ranking v2 against v3, overall, oav2, acv1 or accuracy.

Every number here is reproducible. All models in the family are scored on the same audit — 6,076 calls each, 243 intents across 22 domains, with zero message overlap with the training data. Raw per-call results and scoring code: abyssal-intent-classifier-audit

Good for: routing support tickets, chatbot intent detection, message tagging, triage — fast, local, fully customizable intents, with the option to flag messages that match nothing.

License: Apache-2.0 (base). ~3.1 GB, fits an 8 GB GPU.