Cevra-ToolChoice-4B — experimental INT4

Paper: Evgenii Evstafev, Cevra-ToolChoice-4B: A System Card for Probabilistic Tool Selection, Zenodo preprint, 12 November 2026.

Cevra-ToolChoice-4B selects the next tool from a supplied set of candidates, including no_tool. It is a fine-tune of Qwen3.5-4B trained as a Tinker Decision Model. It does not generate tool-call arguments or execute tools.

This Ollama tag contains the merged model quantized to INT4 in Ollama’s safetensors/MLX format. It was tested with Ollama 0.40.16 on an Apple M1 Pro. Compatibility with other platforms and Ollama versions has not been tested.

Download

ollama pull chigwel/cevra-toolchoice-4b:4b-int4

Smoke test

This example sends the decision-model prompt directly to a local Ollama server. A corresponds to weather, and B to no_tool.

curl --silent --show-error http://127.0.0.1:11434/api/generate \
  -H 'Content-Type: application/json' \
  -d @- <<'JSON'
{
  "model": "chigwel/cevra-toolchoice-4b:4b-int4",
  "prompt": "<|im_start|>system\nAnswer the user's question by picking exactly one of the choices they supply. Reply with the id of your choice, written as `choice#?`, and nothing else.<|im_end|>\n<|im_start|>user\n{\"instructions\": \"Select the next tool to call, or no_tool if none is needed.\", \"state\": \"The user asks for the current weather in London. No tools have been called yet.\"}\n\nChoices (your answer must be exactly one of these):\nchoice#A = \"weather\": Get current weather for a city.\nchoice#B = \"no_tool\": Do not call a tool.<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\nchoice#",
  "raw": true,
  "stream": false,
  "truncate": false,
  "shift": false,
  "logprobs": true,
  "top_logprobs": 20,
  "options": {"num_predict": 1, "num_ctx": 8192, "temperature": 0}
}
JSON

The example only demonstrates a single decision. response is a generated token and may not be a valid candidate for every input. For candidate probabilities, extract the log probabilities of all candidate letters from logprobs[0].top_logprobs and normalize them with softmax. Ollama returns at most 20 alternatives and may omit a candidate even with fewer choices; if any are missing, the complete distribution cannot be recovered from this API. With Ollama 0.40.16, /v1/systemone reports that this model does not support the decision endpoint.

The text API retokenizes the rendered prompt differently from the original Tinker renderer at one newline boundary. Together with INT4 quantization, this changes results relative to the original BF16 model. Do not treat this smoke test as an exact Tinker implementation or as a production-ready decision service. The original model and exact scoring reference are described in the Hugging Face model card.

Evaluation and limitations

A limited local check compared the INT4 model with the previously evaluated BF16 model on 12 held-out examples: three each with 2, 5, 10 and 20 candidates. The selected candidate matched on all 12; maximum absolute probability difference was 0.059. This is a small smoke test, not a full INT4 benchmark or proof of calibration.

The full-weight model card describes evaluation on 406 ToolBench-derived test examples and its limitations. Those metrics apply to the BF16 model, not this quantized tag. The test is not an independent human-labeled benchmark. Validate performance and confidence on your own data; treat tool descriptions and tool outputs as untrusted input.

The public test split and system card provide further context.

License and provenance

The Qwen3.5-4B base is distributed under Apache-2.0; its license is included with this model. The training examples derive from ToolBench and contain descriptions of third-party APIs. The applicable rights and terms for those descriptions have not been independently determined; review them for your intended use.