OLMoE-1B-7B-0924 Model Summary

OLMoE-1B-7B is a Mixture-of-Experts Large Language Model (LLM) released in September 2024. It has 1 billion active parameters and a total of 7 billion parameters. The model yields state-of-the-art performance among models with a similar cost (1B active parameters) and is competitive with much larger models like Llama2-13B.

The OLMoE model is 100% open-source, allowing for transparency and community contributions. It was developed by the AllenAI research group and the details about the model can be found on the OLMoE GitHub repository and the HuggingFace platform.