ibm/ granite4.1-speech:2b-plus-q8_0

21 4 days ago

vision tools 
ollama run ibm/granite4.1-speech:2b-plus-q8_0

Details

4 days ago

7f091e39c560 · 2.9GB

model
granite
·
1.63B
·
Q8_0
projector
clip
·
480M
·
F16
template
{{- /* ------ MESSAGE PARSING ------ */}} {{- /* Declare the system prompt chunks used for different
license
Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ TERMS AND CONDITIONS FOR US
params
{ "temperature": 0 }

Readme

Granite 4.1 Speech

Granite-Speech-4.1 is a compact and efficient speech-language model from IBM, purpose-built for multilingual automatic speech recognition (ASR). The 2b-plus model adds speaker-attributed ASR (speaker labels with word transcripts) and word-level timestamps, which are activated with a simple prompt change. It was created by modality-aligning an intermediate checkpoint of granite-4.0-1b-base to speech, and trained on 174,000 hours of audio from public corpora plus synthetic data, augmented with speaker turns and word-level timestamp tags.

Model update (October 2026): the weights and configuration were updated. The updated model is more accurate, especially for keyword biasing and for word-level timestamps on long samples. It is a drop-in replacement for the previous version (April 2026). In the tables below, v2610 is the updated model and v2604 is the original release.

Running

Pass a path to a .wav file and a task prompt directly on the command line:

ollama run ibm/granite4.1-speech:2b-plus /path/to/audio.wav "can you transcribe the speech into a written format?"

To run the default quantization:

ollama run ibm/granite4.1-speech:2b-plus

To run a specific quantization (e.g., Q4_K_M):

ollama run ibm/granite4.1-speech:2b-plus-q4_K_M /path/to/audio.wav "can you transcribe the speech into a written format?"

The model accepts mono, 16 kHz .wav audio. To trigger speech processing, include the <|audio|> token in your prompt. If the model receives an unfamiliar or malformed prompt, it falls back to transcription by default. Audio segments of up to 9 minutes work well for ASR and speaker-attributed ASR, and up to 3.5 minutes for timestamps.

Preferred Prompts by Task

Task Prompt
ASR can you transcribe the speech into a written format?
ASR (keyword biasing) can you transcribe the speech into a written format? Keywords: <kw1>, <kw2>, ... (e.g. can you transcribe the speech into a written format? Keywords: Kubernetes, gRPC, OpenShift.)
Speaker-attributed ASR Speaker attribution: Transcribe and denote who is speaking by adding [Speaker 1]: and [Speaker 2]: tags before speaker turns.
Word-level timestamps Timestamps: Transcribe the speech. After each word, add a timestamp tag showing the end time in centiseconds, e.g. hello [T:45] world [T:82]

Note: Non-English ASR still requires an English prompt.

Speaker-attributed output marks each turn with [Speaker N]:, numbered by order of appearance. In timestamp mode, each word is followed by a [T:N] tag giving the end of the word in centiseconds, silences are transcribed as _, and only the last three digits are given, so N rolls over every 10 seconds. To convert back to seconds, use t = N/100 + 10R, where R is the number of rollovers so far.

Supported Languages

English, French, German, Spanish, and Portuguese.

Unlike the base granite-speech-4.1-2b model, the Plus model does not provide punctuation and capitalization, does not support Japanese, and is not documented for speech translation (AST). Use the base model when you need those.

Intended Use

Granite-Speech-4.1 Plus is designed for enterprise applications that process speech inputs, especially when a rich transcript with speaker turns and timestamps is desired — converting speech to text in English, French, German, Spanish, and Portuguese.

Capabilities

  • Multilingual ASR — High-accuracy transcription across five languages, powered by a dual-head CTC conformer encoder (graphemic + BPE outputs) with frame importance sampling.
  • Speaker-Attributed ASR — Adds [Speaker N]: labels before each speaker turn, numbered by order of appearance.
  • Word-Level Timestamps — Adds an end-of-word timestamp tag after each word, including silences.
  • Keyword Biasing — Improved recognition of names, acronyms, and technical jargon when supplied with a keyword list. Keywords not present in the audio are allowed.

Performance

WER is Word Error Rate (%), lower is better. Best results in each column are in bold.

ASR — updated Open ASR leaderboard test sets (WER %, ↓):

Model Average WER AMI Earnings22 Gigaspeech LS Clean LS Other SPGISpeech URGENT2024 Monsoon (en_IN) Voxpopuli
v2604 6.01 7.88 8.03 9.02 1.21 2.75 3.48 12.27 5.22 4.23
v2610 6.10 9.04 7.43 8.70 1.09 2.69 3.65 12.74 4.88 4.71

Beam search with num_beams=4 and repetition_penalty=2.0, without speculative decoding.

Keyword list biasing — Keyword F1 score (%, ↑):

Mode Gigaspeech LS Clean LS Other SPGISpeech Voxpopuli TED_LIUM CV-en CV-de CV-es CV-fr CV-pt
v2604 without KWB 73.3 89.0 77.6 79.5 93.9 87.7 72.8 76.1 80.9 72.6 89.5
v2610 without KWB 74.1 89.6 77.2 79.9 94.1 88.7 72.7 76.9 81.6 73.1 90.8
v2604 with KWB 81.4 96.2 92.5 91.1 95.2 92.5 85.5 94.1 94.6 91.7 94.7
v2610 with KWB 84.4 98.4 97.4 92.2 95.0 94.3 82.4 95.5 96.1 95.3 97.0

Speaker-attributed ASR — WDER (%, ↓):

Model FISHER CALLHOME English AMI-SDM GALE
v2604 0.9 2.2 14.6 30.2
v2610 1.0 3.0 13.9 28.3

Averaged over 2-5 minute speech segments. WDER (Word Diarization Error Rate) is the percentage of words attributed to the wrong speaker.

Word-level timestamps — Accumulated Average Shift, AAS (ms, ↓):

Model English average Multilingual average
v2604 38.8 29.8
v2610 34.6 26.5

On long samples (2-5 minute segments built by concatenating short test segments), the update roughly halves the shift:

Test set v2604 v2610
CV-en 2339.4 925.3
CV-fr 119.0 71.3
CV-de 131.3 96.1
MLS-es 407.9 192.5
MLS-pt 1073.8 356.2
Average 814.3 328.3

For results on the original Open ASR leaderboard test sets and comparisons with other timestamp and diarization systems, see the model card.

Learn more