richardyoung/ smolvlm2-2.2b-instruct:Q5_K_M

2,364 8 months ago

SmolVLM2-2.2B-Instruct is a lightweight yet powerful vision-language model that can understand images, read documents, and analyze video frames. At just 2.2B parameters, it runs efficiently on consumer hardware including laptops and smartphones, making 

ollama run richardyoung/smolvlm2-2.2b-instruct:Q5_K_M

Details

8 months ago

afa3e0f6b9f7 · 1.3GB ·

model
llama
·
1.81B
·
Q5_K_M
system
You are SmolVLM2, a helpful AI assistant with vision capabilities. You can understand and analyze im
license
# SmolVLM2-2.2B-Instruct Q5_K_M: Ultra-Compact Vision-Language Model ## Overview SmolVLM2-2.2B-Instr
params
{ "num_ctx": 8192, "stop": [ "<|im_end|>", "<|endoftext|>" ] }
template
{{ if .System }}<|im_start|>system {{ .System }}<|im_end|> {{ end }}{{ if .Prompt }}<|im_start|>user

Readme

SmolVLM2-2.2B-Instruct

A compact, efficient vision-language model that understands both images and text. Built on SmolVLM2-2.2B-Instruct from Hugging Face, this GGUF build runs comfortably on consumer hardware while delivering strong multimodal performance for its size.

🚀 Overview

SmolVLM2-2.2B-Instruct is a small but capable multimodal model designed for image-and-text understanding. It pairs a shape-optimized SigLIP vision encoder with the SmolLM2-1.7B language backbone (architecture based on Idefics3), producing a ~2.2B-parameter model that punches well above its weight on captioning, visual question answering, document understanding, and visual reasoning.

These quantized GGUF builds make the model practical to run locally with Ollama and llama.cpp, from ultra-light 1.0 GB quants up to full-precision f16, so you can trade size against fidelity to fit your hardware.

📊 Benchmarks

Reported by the base model authors (HuggingFaceTB/SmolVLM2-2.2B-Instruct):

Benchmark Score
MMMU 42.0
MathVista 51.5
OCRBench 72.9
MMStar 46.0
AI2D 70.0
ChartQA (Test) 68.84
ScienceQA 90.0
TextVQA (Val) 73.21
DocVQA (Val) 79.98

Scores reflect the original full-precision model; quantized builds may differ slightly.

🎯 Key Features

  • Multimodal understanding, reasons jointly over images and text in a single prompt
  • Small footprint, ~2.2B parameters; quants from 1.0 GB make it laptop- and edge-friendly
  • Strong document & chart skills, competitive DocVQA, OCRBench, and ChartQA results for its size
  • Flexible quantization, seven builds spanning iq4_xs through f16 to balance speed, memory, and quality
  • Open license, Apache 2.0, suitable for research and commercial use

🏷️ Available Versions

Tag Size BPW (approx.) Notes
iq4_xs 1.0 GB ~4.25 Smallest build; lowest memory use
latest 1.1 GB ~4.5 Default tag (alias of Q4_K_M)
Q4_K_M 1.1 GB ~4.5 Recommended balance of size and quality
Q5_K_M 1.3 GB ~5.5 Higher fidelity, modest size increase
Q6_K 1.5 GB ~6.5 Near-lossless quality
q8_0 1.9 GB ~8.5 Very high fidelity
f16 3.6 GB 16 Full precision; reference quality

💻 Quick Start

Pull and run the default build:

ollama run richardyoung/smolvlm2-2.2b-instruct

Run a specific quantization:

ollama run richardyoung/smolvlm2-2.2b-instruct:Q6_K

Ask a question about an image (Ollama supports image input directly):

ollama run richardyoung/smolvlm2-2.2b-instruct "Describe this image in detail." ./photo.jpg

🛠️ Use Cases

  • Image captioning, generate natural descriptions of photos and scenes
  • Visual question answering, answer questions grounded in an image
  • Document & chart understanding, read receipts, forms, charts, and diagrams
  • OCR-style extraction, pull text and structure from images
  • Visual storytelling, narrate or elaborate on visual content

📋 System Requirements

Build Approx. RAM/VRAM Best For
iq4_xs ~2 GB Edge devices, very constrained hardware
Q4_K_M ~2–3 GB Laptops and most consumer GPUs
Q5_K_M ~3 GB Higher quality on mainstream hardware
Q6_K ~3–4 GB Quality-focused local inference
q8_0 ~4 GB Near-reference output
f16 ~5–6 GB Maximum fidelity / evaluation

A modern CPU can run the smaller quants without a GPU; a GPU improves throughput, especially for image-heavy workloads.

🔧 Technical Details

Property Value
Base model HuggingFaceTB/SmolVLM2-2.2B-Instruct
Parameters ~2.2B
Architecture Idefics3-based (SigLIP vision encoder + SmolLM2-1.7B)
Modality Image + text in, text out
Context length 8K tokens
Quantization iq4_xs, Q4_K_M, Q5_K_M, Q6_K, q8_0, f16 (GGUF)
License Apache 2.0

⚠️ Disclaimer

This is a multimodal (image + text) model. It can misread images, hallucinate details that are not present, misinterpret text in pictures, and produce confident but incorrect answers. Outputs should be verified before use, and the model is not intended for high-stakes scenarios or critical decision-making (for example medical, legal, financial, or safety-critical use). Do not submit private, sensitive, or confidential images or text that you are not comfortable processing locally. You are responsible for complying with applicable laws and the Apache 2.0 license when using this model.

🙏 Acknowledgments

  • Hugging Face TB for creating and releasing the original SmolVLM2-2.2B-Instruct model
  • The SmolLM2 and SigLIP teams whose components power the architecture
  • The llama.cpp project for GGUF quantization and efficient local inference
  • The Ollama team for making local multimodal models easy to run

Built & maintained by Richard Young · DeepNeuro