Nimble requires Ollama 0.35 or later.

Nimble is a 9B decision model from Bespoke Labs, fine-tuned from Qwen3.5-9B.

You send Nimble some text and a set of questions. It picks an answer for every question and gives you the probability of each allowed answer. Nimble reads the prompt once per question and scores the answer tokens directly.

There’s no reasoning step. This is what makes it fast.

ollama pull nimble

Highlights

• Three question types: Pick from a list, answer true or false, or place something on a rubric.
• Many questions per request: Ask up to 64 questions about the same text in one call.
• Fast: Nimble answers in under 100ms on a MacBook Pro with an M5 Max.
• Trained on contrastive pairs: The two examples in a pair differ by one fact, and that fact flips the answer.
• Apache 2.0 license

What you can build

API

Decision models use Ollama’s /v1/systemone endpoint, which follows TypeSafe’s Jev API. Put the text you want judged in state and your questions in questions. Ollama builds Nimble’s prompt for you.

Every question has a type, instructions, and usually criteria:

Choice and score questions take 2 to 26 options. confidence runs from 0 to 1 and shows how concentrated the probabilities are. It isn’t the chance that the answer is right.

Examples

cURL

curl http://localhost:11434/v1/systemone -d '{
  "model": "nimble",
  "state": {
    "ticket": "I was charged twice. Please refund the extra payment."
  },
  "questions": {
    "team": {
      "type": "choice",
      "instructions": "Which team should handle this ticket?",
      "criteria": {
        "billing": "Payments and refunds",
        "technical": "Bugs and integrations",
        "other": "None of the above"
      }
    },
    "refund": {
      "type": "noul",
      "instructions": "Does the customer explicitly ask for a refund?"
    },
    "urgency": {
      "type": "score",
      "instructions": "How urgent is this ticket?",
      "criteria": ["Routine", "Soon", "Urgent"]
    }
  }
}'

Output

{
  "model": "nimble",
  "answers": {
    "team": {
      "type": "choice",
      "choice": "billing",
      "probabilities": {"billing": 0.984, "technical": 0.011, "other": 0.005},
      "confidence": 0.918
    },
    "refund": {"type": "noul", "noul": 0.997},
    "urgency": {
      "type": "score",
      "score": 0.693,
      "legend": {"0": "Routine", "1": "Soon", "2": "Urgent"},
      "probabilities": {"0": 0.482, "1": 0.342, "2": 0.175},
      "confidence": 0.068
    }
  },
  "usage": {"input_tokens": 841, "output_tokens": 3}
}

Python

Use TypeSafe’s official Python SDK and point it at Ollama. The SDK requires an API key, but Ollama ignores it, so any value works.

pip install typesafe-sdk
export TYPESAFE_BASE_URL=http://localhost:11434
export TYPESAFE_API_KEY=ollama
export TYPESAFE_DEFAULT_MODEL=nimble

from typesafe_sdk import Choice, Noul, Score, TypeSafeClient

ticket = "I was charged twice. Please refund the extra payment."
questions = {
    "team": Choice(
        instructions="Which team should handle this ticket?",
        criteria={
            "billing": "Payments and refunds",
            "technical": "Bugs and integrations",
            "other": "None of the above",
        },
    ),
    "refund": Noul(
        instructions="Does the customer explicitly ask for a refund?",
    ),
    "urgency": Score(
        instructions="How urgent is this ticket?",
        criteria=["Routine", "Soon", "Urgent"],
    ),
}

with TypeSafeClient(timeout=120) as client:
    result = client.system_one(
        state={"ticket": ticket},
        questions=questions,
    )

print(result.choices["team"].choice)   # billing
print(result.nouls["refund"].noul)     # 0.997
print(result.scores["urgency"].score)  # 0.69

Decision models aren’t in the Ollama CLI or the Ollama Python and JavaScript libraries yet. Use the API or the TypeSafe SDK for now.

More examples

Ticket triage. With null descriptions, the option names describe themselves.

curl http://localhost:11434/v1/systemone -d '{
  "model": "nimble",
  "state": "Our checkout has returned 500 errors since 9am.",
  "questions": {
    "label": {
      "type": "choice",
      "instructions": "Which label fits this ticket?",
      "criteria": {"billing": null, "bug": null, "account": null}
    }
  }
}'

Model routing. Send each prompt to the model that fits it.

curl http://localhost:11434/v1/systemone -d '{
  "model": "nimble",
  "state": "Design a sharded database schema for a payments ledger.",
  "questions": {
    "model": {
      "type": "choice",
      "instructions": "Which model should answer this prompt?",
      "criteria": {"gemma4": "Small model", "gpt-6": "Large model"}
    }
  }
}'

Tool call moderation. Check an agent’s tool call before it runs.

curl http://localhost:11434/v1/systemone -d '{
  "model": "nimble",
  "state": "run_shell(command=\"rm -rf ~\")",
  "questions": {
    "harm": {
      "type": "noul",
      "instructions": "Could this tool call cause harm?"
    }
  }
}'

Evaluation

On Ollama

Mean accuracy across Bespoke Labs’ 13 public datasets with human labels, 3,880 decisions in all. Nimble and Tev1 ran on Ollama. The Jev result is from Bespoke Labs’ own run of the same decisions through TypeSafe’s API.

Bespoke Labs results

The numbers below come from Bespoke Labs and were measured on the original Nimble release.

Held-out set

Bespoke Labs held back 324 examples (162 contrastive pairs) from training to test on.

Public benchmarks

To test outside its own data, Bespoke Labs ran Nimble and Jev 1.13.0 on the same 3,880 records from 13 public datasets. People wrote the labels, and none of the tasks fall in Nimble’s training categories.

Accuracy means agreement with the human label. On the rubric subsets, it only counts when the top level matches the human level exactly, which is why some of those numbers run low for both models.

Training

Contrastive data curation

Bespoke Labs builds the training data in pairs. The two examples in a pair are the same except for one fact. Change that fact and the right answer flips. From these pairs the model learns which evidence should change its decision.

Before a pair is kept, separate model calls check it. The facts have to agree with the policy, and the text can’t give away the answer. Dropping either evidence sentence has to leave the deciding fact unknown. The labels come from running the checked rules in code.

Notes

• Text only.
• Nimble can only pick from the answers you give it. It won’t write explanations, nested JSON, or quotes pulled from the context.
• Questions are scored independently. If two answers need to agree, check that in your code.
• A probability of 0.9 doesn’t mean the answer is right 90% of the time on your data. Test any threshold on your own data before you rely on it.
• If none of your answers might fit, add a “no match” choice.
• Every question is scored with the full state and question set in its prompt, and that prompt has to fit in Nimble’s 8,192-token context. Shorter prompts are better tested.
• Request bodies can be up to 64 KiB.

Reference

• Ollama blog
• GitHub: bespokelabsai/nimble
• Hugging Face: Bespoke-Nimble-9B
• Public benchmarks
• Bespoke Labs