lfm3-omni:2b-mlx-bf16

102 3 hours ago

a single end-to-end model that understands text, images, video and audio, and answers in text and speech

vision tools audio 2b 
ollama run lfm3-omni:2b-mlx-bf16

Details

3 hours ago

a3eb4ae57b32 · 4.5GB

json
{ "architectures": [ "Lfm3OmniForConditionalGeneration" ], "audio_encoder_config": { "conv_channels"
json
{ "bos_token_id": 124894, "eos_token_id": [124900, 125014], "pad_token_id": 124893, "use_cache": tru
json
{ "audio_markers": true, "audio_out_token": "<|audio_out|>", "audio_token": "<|audio|>", "audio_toke
json
{ "version": "1.0", "truncation": null, "padding": null, "added_tokens": [ { "id": 124893, "content"
json
{ "backend": "tokenizers", "bos_token": "<|startoftext|>", "eos_token": "<|im_end|>", "extra_special
json
{ "cap_pixels_per_frame": false, "do_convert_rgb": true, "do_normalize": true, "do_rescale": true, "
json
{ "architectures": [ "LiquidCodecModel" ], "decoder_window": 30, "dtype": "bfloat16", "encoder_conv_
license
LFM Open License v1.0 TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 1. Definitions. "
params
{ "temperature": 0 }
868 tensors

Readme

LFM3-Omni-2B is Liquid AI’s first omni-modal model in the LFM3 family: a single end-to-end model that understands text, images, video and audio, and answers in text and speech. It is built for on-device use: one model replaces a separate ASR model, vision-language model, LLM and TTS model.

  • Any input, text or speech output: ask about a photo by voice, transcribe or translate speech, chat out loud, or ask questions about a video and its soundtrack together.
  • Watches and listens together: for videos with sound, the model sees and hears them at the same time, so it can answer questions that need both.
  • Small enough for devices: at 2.2B parameters, it is designed to run efficiently on phones, laptops and other edge hardware.
Property Specification
Checkpoint Post-trained omni-modal model
Parameters 2.17B total
Architecture LFM3 backbone, LFM3 vision encoder, LFM3 causal audio encoder, LiquidCodec audio tokenizer.
Inputs Text, images, video with or without its soundtrack, and 16 kHz audio. Images and video are processed at native resolution; each 2×2 block of patches becomes one language token. Video is sampled at 2 fps.
Outputs Text, 24 kHz speech, or interleaved text and speech
Context length 32K tokens (input + output)
Vocabulary 128,000 text tokens
Text languages English, Arabic, Chinese, French, German, Italian, Japanese, Korean, Portuguese, Spanish, Vietnamese, Thai, Indonesian, Hindi, Russian, Polish
Speech languages Arabic, Danish, German, Greek, British English, American English, Finnish, French, Hebrew, Italian, Japanese, Korean, Dutch, Polish, Portuguese, Spanish, Swedish, Turkish, Chinese
License LFM Open License v1.0

This is a non-reasoning model. Thinking is not supported.