sam860/lfm2.5

sam860/ lfm2.5

1,607 Downloads Updated 1 month ago

An upgraded version of LFM2 trained on over twice as many tokens

ollama run sam860/lfm2.5:1.2b-Q8_0

curl http://localhost:11434/api/chat \
  -d '{
    "model": "sam860/lfm2.5:1.2b-Q8_0",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

from ollama import chat

response = chat(
    model='sam860/lfm2.5:1.2b-Q8_0',
    messages=[{'role': 'user', 'content': 'Hello!'}],
)
print(response.message.content)

import ollama from 'ollama'

const response = await ollama.chat({
  model: 'sam860/lfm2.5:1.2b-Q8_0',
  messages: [{role: 'user', content: 'Hello!'}],
})
console.log(response.message.content)

Models

Name

2 models

Size

Context

Input

lfm2.5:1.2b-Q8_0

1.2GB · 125K context window · Text · 1 month ago

lfm2.5:1.2b-Q8_0

1.2GB

125K

Text

lfm2.5:1.2b-F16

2.3GB · 125K context window · Text · 1 month ago

lfm2.5:1.2b-F16

2.3GB

125K

Text

Readme

Notes

Uploaded in fp16 and Q8_0

Q8_0 is the sweet‑spot for on‑device CPU inference
fp16 is useful if you have enough GPU memory and want the absolute best performance

Temperature: The model was tuned for very deterministic output, so start with 0.1 – 0.2. Raise to ≈0.6 only if you need more creative or exploratory answers.

Description

LFM2.5‑1.2B‑Instruct – a 1.2B parameter, hybrid Liquid‑architecture model built for edge deployment. I’ve only uploaded the Instruct variant.

Key points:

Hybrid design: 10 double‑gated LIV convolution blocks + 6 Grouped‑Query‑Attention (GQA) blocks give fast, low‑latency inference on CPUs, NPUs and mobile GPUs.
Extended training: 28 T tokens of pre‑training plus multi‑stage reinforcement learning, delivering quality comparable to much larger models.
Context length: 32 k tokens, 65 k‑vocab, multilingual (EN, AR, ZH, FR, DE, JA, KO, ES).
Agentic ready: Supports function‑calling/tool use via the built‑in <|tool_call_start|> / <|tool_call_end|> token wrappers.
Ideal use‑cases:
- On‑device assistants, chatbots, and RAG pipelines
- Structured JSON extraction / data‑wrangling
- Lightweight code completion (FIM)
- Edge AI in smartphones, laptops, vehicles, IoT devices

References

Model Card (HF): https://huggingface.co/LiquidAI/LFM2.5-1.2B-Instruct
Blog post (LFM2.5 launch): https://www.liquid.ai/blog/introducing-lfm2-5-the-next-generation-of-on-device-ai
Documentation (Chat template & tool use): https://docs.liquid.ai/lfm/key-concepts/
Inference guides:
- Transformers → https://docs.liquid.ai/lfm/inference/transformers
- llama.cpp → https://docs.liquid.ai/lfm/inference/llama-cpp

### Notes
Uploaded in **fp16** and **Q8_0**

- **Q8_0** is the sweet‑spot for on‑device CPU inference 
- **fp16** is useful if you have enough GPU memory and want the absolute best performance

**Temperature:** The model was tuned for very deterministic output, so start with **0.1 – 0.2**. Raise to **≈0.6** only if you need more creative or exploratory answers.

---

### Description
**LFM2.5‑1.2B‑Instruct** – a 1.2B parameter, hybrid Liquid‑architecture model built for *edge* deployment. I've only uploaded the Instruct variant.

Key points:

- **Hybrid design:** 10 double‑gated LIV convolution blocks + 6 Grouped‑Query‑Attention (GQA) blocks give fast, low‑latency inference on CPUs, NPUs and mobile GPUs.  
- **Extended training:** 28 T tokens of pre‑training plus multi‑stage reinforcement learning, delivering quality comparable to much larger models.  
- **Context length:** 32 k tokens, 65 k‑vocab, multilingual (EN, AR, ZH, FR, DE, JA, KO, ES).  
- **Agentic ready:** Supports function‑calling/tool use via the built‑in `<|tool_call_start|>` / `<|tool_call_end|>` token wrappers.  
- **Ideal use‑cases:**  
  - On‑device assistants, chatbots, and RAG pipelines  
  - Structured JSON extraction / data‑wrangling  
  - Lightweight code completion (FIM)  
  - Edge AI in smartphones, laptops, vehicles, IoT devices

---

### References
- **Model Card (HF):** https://huggingface.co/LiquidAI/LFM2.5-1.2B-Instruct  
- **Blog post (LFM2.5 launch):** https://www.liquid.ai/blog/introducing-lfm2-5-the-next-generation-of-on-device-ai  
- **Documentation (Chat template & tool use):** https://docs.liquid.ai/lfm/key-concepts/  
- **Inference guides:**  
  - Transformers → https://docs.liquid.ai/lfm/inference/transformers  
  - llama.cpp → https://docs.liquid.ai/lfm/inference/llama-cpp

Paste, drop or click to upload images (.png, .jpeg, .jpg, .svg, .gif)