Thox-ai/ thox-cluster-deep:latest

8 10 months ago

A THOX configuration and distribution of Meta's Llama 3.1 405B, published for local research-scale inference and MeshStack deployments. Llama 3.1 Community License.

... /
params
7d9faec36541 ยท 175B
{
"num_ctx": 131072,
"repeat_penalty": 1.1,
"stop": [
"<|start_header_id|>",
"<|end_header_id|>",
"<|eot_id|>"
],
"temperature": 0.7,
"top_k": 40,
"top_p": 0.9
}