Vision-language model fine-tuned for structured data extraction from photographed documents — IDs, invoices, receipts, and certificates in, clean JSON out. Built to replace manual data entry on real-world, imperfectly-photographed paperwork, not clean scans.

Usage

ollama run zuraai-vl-v3

Attach a document image and ask it to extract the data — no prompt engineering required, the model was trained to always return JSON.

Input: a photo of a PAN card Output: “`json { “document_type”: “pan_card”, “name”: “ARJUN R MENON”, “father_name”: “…”, “date_of_birth”: “14/03/1991”, “pan_number”: “HTQPM4821K” } Coverage Trained across 33 document categories, each with its own target schema:

Group Categories Identity passport, aadhaar, PAN card, driving license, visa, NIC, CID Financial invoice, receipt, purchase order, quotation, cheque, telegraphic transfer, POS payment slip Business & other business card, certificate, academic marksheet, medical certificate, menu card, product catalog, NOC, + 12 more Several categories (NIC, CID, purchase order, telegraphic transfer, POS slip, business card, certificate) include real decodable QR/barcode values as ground truth, not just visual layout.

Eval Scored on a held-out test split the model never trained on.

Version Base model Doc-type accuracy Mean field accuracy JSON validity v1 Qwen3-VL-8B 56% 80% 94% v3 (current) Qwen3.5-4B 83% 74% 86% v3 moved to a smaller, native-multimodal 4B base and roughly tripled document-type accuracy over v1, at a small cost to field accuracy — net positive since misclassifying the document type is the harder failure mode to recover from downstream. Eval set sizes differ slightly between milestones (34 vs 36 examples) as the schema was finalized, so treat this as a trend, not a strict apples-to-apples benchmark. Head-to-head numbers against other open VLMs (qwen3-vl:8b, minicpm-v4.6) are planned but not yet published.

Base & fine-tuning Base model: Qwen3.5-4B — native multimodal, hybrid linear/full-attention architecture Fine-tuning: LoRA (r=16, α=32), targeting both the full-attention projections (q/k/v/o_proj) and the Gated DeltaNet linear-attention projections (in_proj_qkv/a/b/z, out_proj); vision tower frozen Training data: 5,940 instruction examples (4,950 train / 990 test) across 33 categories — real photographed receipts (CORD) and invoices blended with synthetic identity documents (fake data only, no real people’s documents) Export: GGUF, f16 and Q4_K_M quantized, with a separate mmproj vision-projector file Serving: Ollama, using the qwen3-vl-thinking renderer and parser About Built by Krishna, Brilliant Systems Solutions Pvt Ltd — an IT solutions company headquartered in the Maldives, with engineering based in Madurai, Tamil Nadu, India. Services include custom software, ERP systems, web & mobile apps, IT consulting, and digital marketing.