8 10 months ago

A THOX configuration and distribution of Meta's Llama 3.1 405B, published for local research-scale inference and MeshStack deployments. Llama 3.1 Community License.

ollama run Thox-ai/thox-cluster-deep

Details

10 months ago

75b8f671a4c5 · 243GB

llama
·
406B
·
Q4_K_M
LLAMA 3.1 COMMUNITY LICENSE AGREEMENT Llama 3.1 Version Release Date: July 23, 2024 “Agreement”
You are Thox.ai Cluster Deep, a privacy-first frontier AI assistant running on a distributed MagStac
{ "num_ctx": 131072, "repeat_penalty": 1.1, "stop": [ "<|start_header_id|>",
{{ if .System }}{{ .System }}{{ end }}{{ if .Prompt }}{{ .Prompt }}{{ end }}

Readme

Thox-Cluster-Deep — Frontier Reasoning Configuration (Llama 3.1 405B, THOX-Configured)

License Params Base Access

Your AI. Your Data. Your Rules.™

Thox-Cluster-Deep is a THOX configuration and distribution of Meta’s Llama 3.1 405B, published for local research-scale inference and MeshStack deployments. The weight layer in this model’s Ollama manifest is byte-identical to the upstream provider’s Ollama weight layer (llama3.1:405b). THOX’s work in this release is the MeshStack clustering layer these configurations were made for, configuration (runtime parameters and, where present, a system prompt), packaging, and distribution. THOX did not train these weights.

Correction notice. This page previously quoted grant-success, publication-rate, and cost-saving figures; those were inherited catalog copy with no THOX deployment evidence behind them and have been removed. No benchmark, throughput, latency, concurrency, uptime, cost-saving, or compliance figure is claimed for this release; figures of that kind that appeared on earlier versions of this page were inherited catalog copy and have been removed pending THOX-run evaluation on THOX deployments. Earlier versions of this page also stated the license as MIT; that was incorrect and has been corrected (see License).

The published Ollama model Thox-ai/thox-cluster-deep is 243 GB and reuses the upstream weight layer plus the THOX configuration layers from this model’s Ollama manifest.

Run it

Ollama (published model)

ollama pull Thox-ai/thox-cluster-deep
ollama run Thox-ai/thox-cluster-deep

Summary

A THOX configuration and distribution of Llama 3.1 405B for local research-scale inference and MeshStack deployments: the largest open-weight model in the family, for research-grade analysis and synthesis. Configuration-derived: THOX did not train these weights. Per Meta’s model record, Llama 3.1 405B was trained on 15T+ tokens.

Specification Details
Base model Llama 3.1 405B by Meta
Parameters 405.9B (config model_type), llama family
Quantization Q4_K_M (GGUF file_type in the manifest config)
Context window (served) 131,072 tokens, from the params layer (num_ctx) in this model’s Ollama manifest.
Deployment Local inference via Ollama; distributed orchestration via MeshStack (separate from this download)

Intended use

Local inference through Ollama for research-scale work, on hardware that fits a 405B Q4-class model:

  • University and laboratory literature review, meta-analysis, and hypothesis work
  • Consulting, finance, legal, and policy research on private data
  • Long-horizon strategic analysis under complete local control

Local inference with complete control over research data. Downloading this model gives you local inference in Ollama. Distributed orchestration across devices is provided by MeshStack; a model download alone does not create a cluster or pool memory across devices.

Training data and procedure

No training by THOX. Base: meta-llama/Llama-3.1-405B-Instruct by Meta, via Ollama tag llama3.1:405b. The original THOX release materials for this model claim no separate THOX fine-tuning run, training dataset, or training procedure, and none is claimed here. Base-model training data and procedure are those of the base model’s provider; see the base model card for that record. Datasets added by THOX for this release: none.

Evaluation

No THOX-run evaluation has been published for this model. THOX claims no benchmark, throughput, latency, or concurrency figure for this release. The only figures stated are upstream characteristics from Meta’s base model record (405B parameters, 15T+ training tokens). This section will be updated with method and results when a THOX-run evaluation lands.

Hardware Reference (from the original card)

Reference configuration from the original card, per device: minimum 96 GB RAM, 100 GB storage, NVIDIA Jetson Orin AGX class GPU (64GB VRAM), 10 Gbps network. Recommended 128 GB RAM, 150 GB NVMe SSD.

Limitations

  • Outputs can be wrong. Professional, medical, legal, financial, and security-relevant outputs require qualified human review.
  • Local deployment only in this distribution; behavior and speed depend entirely on the operator’s hardware and configuration.
  • The operator is responsible for meeting any regulatory obligations that apply to their deployment.
  • Research outputs require expert review; the model supports, and does not replace, researcher judgment.
  • All processing is local by design; the model has no access to online information or external services unless the operator wires them in.

License

The model weights are distributed under the Llama 3.1 Community License Agreement (Meta, release date July 23, 2024), per the license layer in this model’s Ollama manifest (from the upstream llama3.1:405b distribution). Earlier versions of this page stated MIT; that was incorrect and has been corrected. Use and redistribution are governed by the Llama 3.1 terms, including their acceptable-use conditions. THOX claims no license over the weights beyond the upstream terms.

MagStack and MeshStack technology is proprietary to Thox.ai LLC. Publication of this model does not license THOX mesh technology, patents, or trademarks beyond identifying the model.

Provenance

THOX MeshStack Cluster family. First listed on Ollama under Thox-ai in 2025 and held private; publicly released 2026-10-31 by order of Tommy Xaypanya, CTO and Co-Founder of THOX.ai, as part of the MeshStack model family release. Upstream: meta-llama/Llama-3.1-405B-Instruct, Ollama tag llama3.1:405b; the weight layer is byte-identical to the upstream provider’s Ollama weight layer (SHA-256 verified in the 2026-10-31 release review). Model card: https://huggingface.co/Thox-ai/thox-cluster-deep. Correction history: inherited catalog claims (grant-success, publication-rate, cost-saving and concurrency figures) removed and the license statement corrected from MIT to the Llama 3.1 Community License, as recorded in the correction notice above. Enterprise: enterprise@thox.ai.