Whistle — speech to text in 17 MB, on the CPU, for xOllama

Cactus Compute’s Whistle · 7 languages · word timestamps · 17 MB · runs on the CPU (Metal on a Mac) · optional Silero speech detector · OpenAI /v1/audio/transcriptions

Runs on xOllama, not on stock ollama. Stock ollama pulls this model but cannot show or run it: it carries only a media engine. It needs xOllama on the opencoti c12 engine, and no xOllama release carries that engine yet: the tags are published ahead of it and state REQUIRES 0.40.3-xollama.3. Releases up to v0.40.3-xollama.2 are on c11, which cannot load the model and says so (invalid ggml type 145). Neither can a stock whisper.cpp or llama.cpp.

Cactus Compute’s Whistle as one 17 MB file. It transcribes English, German, French, Spanish, Italian, Dutch and Polish, detects the language by itself, and gives a start and end time for every word. It runs on the CPU, so it takes no GPU memory away from a language model loaded beside it. The weights are Cactus’s own, byte for byte: nothing was re-quantized or re-trained.

Use

xollama pull mannix/whistle:vad
curl http://localhost:22434/v1/audio/transcriptions \
  -F model=mannix/whistle:vad -F file=@meeting.mp3

# word times
curl http://localhost:22434/v1/audio/transcriptions \
  -F model=mannix/whistle:vad -F file=@meeting.mp3 \
  -F response_format=verbose_json -F 'timestamp_granularities[]=word'

response_format is json, text, verbose_json, srt or vtt, and language is optional. wav and mp3 input were tested. Audio longer than 30 seconds is cut into windows at sentence ends, and the language is detected for each window. Translation to English is not supported; use mannix/whisper for that.

To give a chat model ears, add it to the model instead of running it apart:

xollama media create whistle-vad --to qwen3:8b

Which tag

Take vad for microphones, meetings and anything with silence, music or background noise in it: without a detector a transcriber writes words for sounds that are not speech. Take the plain tag for clean, already-cut clips, or when you run your own detector in front. The detector belongs to the model, not to the request: a tag with it uses it for every transcription.

Measured

The word error rates are the engine’s own measurement (opencoti; the Cactus reference engine scores 17.65 % on the same weights). The non-speech row was measured through xOllama on the c12 engine candidate (2610130822001), on the CPU, and matches the engine’s own figure. A 3-second clip transcribes in under a second on a desktop CPU, the load included.

Tags

Source

GGUF: ManniX-ITA/whistle-GGUF, from Cactus-Compute/whistle · License: Apache-2.0. Detector: Silero VAD in the ggml-org/whisper-vad conversion, unmodified · License: MIT (© 2020-present Silero Team). Both licence texts travel inside the tags (xollama show mannix/whistle:vad --license). Docs: xOllama media.