Abyssal/intent-classifier-general-oav2

The strictest gatekeeper here, rejecting 87% of out-of-scope messages - the highest in the family. Define intents at prompt time, no retraining. It returns the best-matching name, or none_of_the_above when nothing fits.

Recommended usage

Two modes, depending on whether you need out-of-scope rejection.

1. Reliable routing (always pick one of your intents). Pass your intent names as a JSON-schema enum in format. This grammar-constrains decoding so the answer is always one of your intents (never a hallucinated or out-of-list label):

curl http://localhost:11434/api/chat -d '{
  "model": "intent-classifier-general-oav2:1.5b",
  "stream": false,
  "options": {"temperature": 0},
  "format": {"type": "string", "enum": ["refund", "tracking", "account"]},
  "messages": [{"role": "user", "content":
    "Candidate intents:\nrefund: wants money back\ntracking: where their order is\naccount: login or profile\n\nUser message: where is my package?\n\nAnswer with exactly one intent name from the list above."
  }]
}'
# -> "tracking"

2. Rejection (detect out-of-scope messages). Leave format off. In free generation the model returns none_of_the_above when no intent fits — the main thing a stock model cannot do (it almost always forces a pick). Enum mode removes this option, so use free generation when you need rejection:

curl http://localhost:11434/api/chat -d '{
  "model": "intent-classifier-general-oav2:1.5b",
  "stream": false,
  "options": {"temperature": 0},
  "messages": [{"role": "user", "content":
    "Candidate intents:\nrefund: wants money back\ntracking: where their order is\n\nUser message: what time do you close on sundays?\n\nAnswer with exactly one intent name from the list above."
  }]
}'
# -> "none_of_the_above"

Getting the best results

• Write clear, specific descriptions. The model matches on the description, so this is the single biggest lever. Prefer tracking: track or locate a customer's order or shipment over a terse tracking: order status. Vague one- or two-word descriptions are the main cause of an occasional wrong none_of_the_above.
• Keep temperature 0 (the baked-in default) for deterministic, repeatable routing.
• Use the exact prompt shape shown above (Candidate intents: … User message: … Answer with exactly one intent name from the list above.) — it was trained on this format.
• Names are free-form — snake_case, CamelCase, hyphens or plain words all work, including names the model has never seen; it matches by description and echoes your name verbatim.

Under the hood

Base: LoRA fine-tune of Qwen2.5-1.5B-Instruct (Apache-2.0). Trained across five public intent datasets (300+ intents spanning banking, voice-assistant, travel, and general support), where each example presents a different set of intents with descriptions — so the model learns the skill “read whatever list you are given and match it” rather than a fixed taxonomy. It is also trained with a real none_of_the_above label; this variant is built to maximize rejection, using distinct out-of-scope rejects matched to the audit’s construction plus exact-match examples, with ~30% of examples renamed to invented names. Runs at temperature 0 and is fully deterministic.

Accuracy: — 6,076 calls, temperature 0, a 243-intent / 22-domain taxonomy (none of it from the training data) with realistic 3–4 intent lists. oav2 rejects 86.7% of out-of-scope messages — over 6× the stock model — while holding 93.3% enum-constrained accuracy and still beating stock qwen on in-scope accuracy (90.0% vs 89.4%).

For the out-of-scope row, a low in-list number is the goal: the model escapes the list with none_of_the_above instead of guessing. Stock qwen2.5 stays in-list 86.3% of the time here — it has no deliberate rejection, so the 13.7% it does escape is accidental (rambling), not a clean none_of_the_above.

oav2 vs overall vs v3 vs v2 vs stock qwen

oav2 is the rejection specialist: it turns none_of_the_above into a reliable escape hatch, rejecting 86.7% of out-of-scope messages — comfortably ahead of overall (73.9%) and v2 (83.8%), and over 6× the stock model — while keeping enum-constrained accuracy essentially on par (93.3%). The trade-off is real and worth stating plainly: free-generation accuracy drops to 86.8% (vs 90.2% overall), because a model that is quick to say no occasionally says no to valid requests. If you need rejection ≥80%, this is the model. If you need maximum in-scope accuracy, use accuracy instead.

Want a balance rather than the strictest gate? See intent-classifier-general-overall, which rejects ~74% at higher accuracy. For the highest accuracy in the family, see intent-classifier-general-accuracy.

Every number here is reproducible. All models in the family are scored on the same audit — 6,076 calls each, 243 intents across 22 domains, with zero message overlap with the training data. Raw per-call results and scoring code: abyssal-intent-classifier-audit

Good for: routing support tickets, chatbot intent detection, message tagging, triage — fast, local, fully customizable intents, with the option to flag messages that match nothing.

License: Apache-2.0 (base). ~3.1 GB, fits an 8 GB GPU.