richardyoung/ lfm2.5-24b-a2b:Q6_K

42 yesterday

LiquidAI LFM2.5-24B-A2B: 24B reasoning MoE with only 2.45B active. Fast agents and tool use, 16 languages. 

ollama run richardyoung/lfm2.5-24b-a2b:Q6_K

Details

yesterday

968a01b2752d · 20GB

model
lfm2moe
·
24B
·
Q6_K
params
{ "repeat_penalty": 1.05, "temperature": 0.1, "top_k": 50 }

Readme

LFM2.5-24B-A2B

LiquidAI LFM2.5-24B-A2B: 24B reasoning MoE with only 2.45B active. Fast agents and tool use, 16 languages.

🚀 Overview

GGUF build of LiquidAI/LFM2.5-24B-A2B, Liquid AI’s 24B-parameter reasoning model with only 2.45B active parameters (64 experts, top-4), built for fast, private agents on-device.

🎯 Key Features

  • Only 2.45B active parameters: fast even with most weights on the CPU
  • Always thinks before answering; tuned for agents and tool use
  • 131K-token context, 16 languages
  • Recommended sampling preset in the Ollama tags (temperature 0.1, top_k 50, repetition penalty 1.05)

🏷️ Available Versions

Tag Size BPW Notes
IQ3_M 10.5 GB 3.66 Smallest
IQ4_XS 12.8 GB 4.25 Great quality-size balance
latest / Q4_K_M 14.5 GB 4.85 Recommended
Q5_K_M 17.0 GB 5.68 Higher quality
Q6_K 19.7 GB 6.56 Very high quality
Q8_0 25.5 GB 8.5 Near-lossless

💻 Quick Start

ollama run richardyoung/lfm2.5-24b-a2b

🛠️ Use Cases

  • Local agents and tool calling
  • Reasoning on machines with little VRAM (MoE offload)
  • Multilingual assistants

✅ Verified

Tested through Ollama before upload (Q4_K_M and Q8_0): answers a basic prompt correctly and stops cleanly.

🔧 Technical Details

  • Base Model: LiquidAI/LFM2.5-24B-A2B
  • Parameters: 24.0B
  • Context Length: 128K tokens
  • Quantization: GGUF via llama.cpp with an importance matrix (calibration_datav3)
  • License: LFM Open License v1.0 (inherited from the base model)

🙏 Acknowledgments

  • Base Model: Liquid AI
  • Quantization: llama.cpp

Built & maintained by Richard Young · DeepNeuro