fauxpaslife/ Moonlight-16B-A3B-Instruct:latest

1 11 hours ago

Moonlight, a 3B/16B-parameter Mixture-of-Expert (MoE) model trained with 5.7T tokens using Muon. Their "model improves the current Pareto frontier, achieving better performance with much fewer training FLOPs compared to prior models."

... /
params
91f5973a7c6d ยท 119B
{
"stop": [
"<|im_system|>",
"<|im_middle|>",
"<|im_end|>",
"<|im_user|>",
"<|im_assistant|>",
"<|im_user|>user<|im_middle|>"
]
}