Thox-ai/ thoxmini-3b:Q4_K_M

22 1 month ago

THOX.ai lightweight edge assistant. Llama-3.2-3B QLoRA fine-tune, Q4_K_M, sized for Milk-V Duo / Pi-class hardware. 

ollama run Thox-ai/thoxmini-3b:Q4_K_M

Details

1 month ago

5ea74417ae80 · 2.0GB ·

model
llama
·
3.21B
·
Q4_K_M
license
Base model: Llama 3.2 Community License (Meta) — https://ai.meta.com/llama/license/ THOX LoRA delt
license
Base model governed by its own license: Llama 3.2 Community License (https://ai.meta.com/llama/licen
template
{{ if .System }}<|begin_of_text|><|start_header_id|>system<|end_header_id|> {{ .System }}<|eot_id|>{
system
You are ThoxMini-3B, the THOX.ai lightweight edge assistant for Milk-V Duo, Pi Zero W, and MagStack
params
{ "num_ctx": 8192, "stop": [ "<|eot_id|>", "<|end_of_text|>" ], "tem

Readme

ThoxMini-3B

THOX.ai lightweight edge assistant. Llama-3.2-3B QLoRA fine-tune, Q4_K_M, sized for Milk-V Duo / Pi-class hardware.

Tags: edge-ai on-device llama 3b q4_k_m

At a glance

  • Base model: meta-llama/Llama-3.2-3B-Instruct
  • Fine-tune: QLoRA supervised fine-tune (THOX thoxllm-factory pipeline)
  • Architecture: llama
  • Parameters: 3.2B
  • Quantization: Q4_K_M (2.0 GB)
  • Native context: 131072 tokens (default runtime num_ctx 8192)
  • Prompt template: Llama 3 (<|start_header_id|> / <|eot_id|>)
  • License: Llama 3.2 Community License (Meta) — https://ai.meta.com/llama/license/. THOX fine-tune, Modelfile, and packaging: © 2026 THOX.ai LLC — see NOTICE.md.
  • Publisher: THOX.AI LLC

Applications

Launch this model directly into a supported tool:

ollama launch claude   --model thox-ai/thoxmini-3b:Q4_K_M   # Claude Code
ollama launch chatgpt  --model thox-ai/thoxmini-3b:Q4_K_M   # Codex App
ollama launch openclaw --model thox-ai/thoxmini-3b:Q4_K_M   # OpenClaw
ollama launch hermes   --model thox-ai/thoxmini-3b:Q4_K_M   # Hermes
ollama launch codex    --model thox-ai/thoxmini-3b:Q4_K_M   # Codex
ollama launch opencode --model thox-ai/thoxmini-3b:Q4_K_M   # OpenCode

Quick start

ollama pull thox-ai/thoxmini-3b:Q4_K_M
ollama run thox-ai/thoxmini-3b:Q4_K_M "your prompt"

Intended use

Concise on-device assistant for constrained edge targets (Milk-V Duo, Pi Zero-class, MagStack Air): short answers, small code snippets, quick lookups.

The model ships with the THOX factuality system prompt: it answers company questions only from verified facts and declines to invent non-public details (headquarters, headcount, funding, founding date).

Not intended for

Long-context reasoning, autonomous multi-step coding, legal or medical advice.

Notes

  • Context: the GGUF supports up to 131072 tokens; the published default num_ctx is 8192 for predictable memory on edge devices. Override at runtime with --num-ctx or a client parameter.
  • On-device: built for local execution on THOX hardware and desktops. Your AI. Your Data. Your Rules.

Links

Published by THOX.AI LLC from the thoxllm-factory pipeline.