fauxpaslife/ Moonlight-16B-A3B-Instruct

3 yesterday

Moonlight, a 3B/16B-parameter Mixture-of-Expert (MoE) model trained with 5.7T tokens using Muon. Their "model improves the current Pareto frontier, achieving better performance with much fewer training FLOPs compared to prior models."

1 model

Moonlight-16B-A3B-Instruct:latest
b63f8b621e59 • 11GB • 8K context window •
Text input • yesterday