From-scratch THOX base model: 123M Llama-arch decoder pretrained by THOX (2,000 steps, AMD MI300X); 49.3% HumanEval / 44.6% MBPP zero-shot. Q4_K_M GGUF, edge-sized. A base/completion model, not a chat assistant. Apache-2.0.

thox-micro-125m

**From-Scratch THOX Base Model — a THOX-owned base/completion model trained from scratch: a 123M-parameter decoder for edge and on-device use** **Safetensors weights · GGUF: Q4_0, Q4_K_M, thox-micro-125m.f16.gguf**

Your AI. Your Data. Your Rules.™

Status: PUBLISHED — RESEARCH MODEL. A THOX-owned base/completion model trained from scratch (123M-parameter decoder) for edge and on-device use. See Files for exactly what this repo contains.

Summary

thox-micro-125m is a THOX-owned base/completion model trained from scratch: a 123M-parameter decoder for edge and on-device use. It is not an instruction-tuned assistant; for the chat build, see Thox-ai/thox-micro-125m-instruct. GGUF builds are bundled in this repo (Q4_0, Q4_K_M, F16) and in Thox-ai/thox-micro-125m-GGUF.

Highlights

thox-micro-125m is a THOX-owned base/completion model trained from scratch: a 123M-parameter decoder for edge and on-device use.

from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("Thox-ai/thox-micro-125m")
model = AutoModelForCausalLM.from_pretrained("Thox-ai/thox-micro-125m")

ollama run hf.co/Thox-ai/thox-micro-125m:Q4_K_M

Specifications


Base model	See card body
Parameters	123M
Formats	Safetensors weights · GGUF: Q4_0, Q4_K_M, thox-micro-125m.f16.gguf · Ollama Modelfile
License	Apache-2.0. Trained from scratch by THOX.ai LLC; covered under Apache-2.0.
This repository contains	Full weights and tokenizer in this repo

Files

File	Size	SHA-256
`thox-micro-125m.f16.gguf`	248,987,840 B	`1fa8b56e5e338e4bce48deabc8cbd63a7462620764e9d52976671cd2ebddda2f`
`model.safetensors`	247,186,856 B	`d0a7cbd9bd2ca32e7150bb6b61ed78d935b64b0de046baca2ed036c86461cc49`
`thox-micro-125m.Q4_K_M.gguf`	84,664,256 B	`51e3442728ae2ce339416f8afc745c00da6b65f278d20f85bcea98d7fc8c6bbb`
`thox-micro-125m.Q4_0.gguf`	81,318,848 B	`cbd9d0ff263140a7c11c85b7a5689d7f9e83b9ddc3f82130f5c8dc4706a3a503`
`tokenizer.json / NOTICE / .gitattributes / Modelfile / LICENSE / config.json / tokenizer_config.json`	—	Supporting files (see repo)

Architecture

Field	Value
Architecture	LlamaForCausalLM
Layers	12
Hidden size	768
Attention heads	12
KV heads	12 (full MHA, no GQA)
FFN / intermediate	2048
Vocab	50,304
Max context	1024
Tied embeddings	yes
dtype	float16

Run it

from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("Thox-ai/thox-micro-125m")
model = AutoModelForCausalLM.from_pretrained("Thox-ai/thox-micro-125m")

ollama run hf.co/Thox-ai/thox-micro-125m:Q4_K_M

Platforms

Platform / device	Run path
Hugging Face	Download the builds listed under Files from this repo (see Run it)

Intended use

On-device and edge text completion within the THOX stack.
Not a safety-aligned public assistant unless deployed behind THOX guardrails.
Out of scope: medical, legal, financial, safety-critical, or other high-stakes decisions without human review; unattended control of physical devices without a confirmation step.

Training data and procedure

Trained by THOX from scratch as part of the THOX small-model research line. The corpus and procedure are as described in this card at release; no external base model was used.

Evaluation

Best validation loss 2.6099. No THOX GATE49 result is published for this model.

Limitations

A small base model: completion-style behaviour, weak factual reliability; verify important answers.
Behaviour and bias reflect the training data, which is mostly English; quality in other languages is lower.
Quantization trades accuracy for size; lower-bit files degrade more.

License

Apache-2.0. Trained from scratch by THOX.ai LLC; covered under Apache-2.0.

THOX.ai — Your AI. Your Data. Your Rules.™

Provenance

Published by THOX.ai as Thox-ai/thox-micro-125m. THOX from-scratch research line; no upstream base model. This card was conformed to THOX Model Card Standard v1.0 on 2026-10-31 (frontmatter, tags, header, files, and section structure); the technical content is as published at release.

Research

Weight-Space Programming for THOX Edge Models: Evidence Review, Small-Scale Recreation, and a Signed Patch Channel

THOX.ai Research technical report v1.1.0, 4 November 2026. Craig Ross and Tommy Xaypanya, THOX.ai LLC. DOI: https://doi.org/10.5281/zenodo.23955394 — Paper page: https://thox.ai/research/papers/weight-space-programming — PDF: https://huggingface.co/datasets/Thox-ai/thox-whitepapers/resolve/main/weight-space-programming/Weight-Space-Programming-for-THOX-Edge-Models_v1.1.0.pdf

The paper names thox-micro-125m as its in-house target: trained by THOX from scratch, a weight-space patch here would be the first fully THOX-owned artifact of the program if the paper’s gates pass. Its loop baseline is part of the paper’s pre-registered Section 5 protocol; the paper claims no weight-space result for this model.

THOX MeshStack Community Mesh: Opt-In User-Contributed Hardware for a Privacy-Centric Shared Computing and Distributed Inference Mesh

THOX.ai Research technical report v1.0.0, 10 November 2026. Craig Ross and Tommy Xaypanya, THOX.ai LLC. DOI: https://doi.org/10.5281/zenodo.24115596 — GitHub: https://github.com/ttracx/thox-community-meshstack-paper — PDF: https://huggingface.co/datasets/Thox-ai/thox-whitepapers/resolve/main/community-meshstack/THOX_Community_MeshStack_Research_Paper_v1.0.0.pdf

Citation

@misc{thoxai2026thoxmicro125m,
  title={thox-micro-125m},
  author={THOX.ai},
  year={2026},
  url={https://huggingface.co/Thox-ai/thox-micro-125m}
}