💨 Notus 7B is an open source LLM released by Argilla, fine-tuned using Direct Preference Optimization (DPO) and AIF (AI Feedback) techniques. This model is fine-tuned with a better curated version of the Ultrafeedback dataset.
385 Pulls Updated 9 months ago
Updated 9 months ago
9 months ago
67f4c4f6e741 · 3.8GB
Readme
💨 Notus
Summary
- Developed by: Argilla (based on HuggingFace H4 and MistralAI)
- Shared by: Argilla
- Hugging Face Hub: https://huggingface.co/argilla/notus-7b-v1
- Model type: GPT-like 7B model DPO fine-tuned
- Language(s) (NLP): Mainly English
- License: MIT (same as Zephyr 7B-beta)
- Finetuned from model: alignment-handbook/zephyr-7b-sft-full
- Repository: https://github.com/argilla-io/notus
- Paper: N/A
- Play with Notus on HuggingChat: https://argilla-notus-chat-ui.hf.space/
Performance
Table adapted from Zephyr-7b-β and Starling’s original tables for MT-Bench and AlpacaEval benchmarks. Results are shown sorted by AlpacaEval win rates and ommit some >7B for brevity.
Notus stays on par with Zephyr on MT-Bench, while surpassing Zephyr and Claude 2 on AlpacaEval. Making Notus the most-competitive 7B commercial model on AlpacaEval.
Model | Size | Alignment | MT-Bench (score) | AlpacaEval (win rate %) | License |
---|---|---|---|---|---|
GPT-4-turbo | - | ? | 9.32 | 97.70 | Proprietary |
XwinLM 70b V0.1 | 70B | dPPO | - | 95.57 | LLaMA 2 License |
GPT-4 | - | RLHF | 8.99 | 95.03 | Proprietary |
Tulu 2+DPO 70B V0.1 | 70B | dDPO | 6.29 | 95.28 | Proprietary |
LLaMA2 Chat 70B | 70B | RLHF | 6.86 | 92.66 | LLaMA 2 License |
Starling-7B | 7B | C-RLFT + APA | 8.09 | 91.99 | CC-BY-NC-4.0 |
Notus-7b-v1 | 7B | dDPO | 7.30 | 91.42 | MIT |
Claude 2 | - | RLHF | 8.06 | 91.36 | Proprietary |
Zephyr-7b-β | 7B | dDPO | 7.34 | 90.60 | MIT |
Cohere Command | - | RLHF | - | 90.62 | Proprietary |
GPT-3.5-turbo | - | RLHF | 7.94 | 89.37 | Proprietary |
Usage
CLI
ollama run argilla/notus
API
Example:
curl -X POST http://localhost:11434/api/generate -d '{
"model": "notus",
"prompt":"Here is a story about llamas eating grass"
}'
Context
You can find the entire process of the creation of Notus in our blogpost.
Notus is a collection of fine-tuned models using SFT, DPO, SFT+DPO, and/or any other RLAIF/RLHF techniques; following a data-first, human-centric approach, since that’s what we do best at Argilla.
Notus models are intended to be used as assistants via chat-like applications, and are evaluated with Chat (MT-Bench, AlpacaEval) and Academic (Open LLM Leaderboard) benchmarks for a direct comparison with other similar LLMs.
Notus name comes from the ancient Greek god Notus, as a wink to Zephyr, which comes from the ancient Greek god Zephyrus; with the difference that Notus is the god of the south wind, and Zephyr the god of the west wind. More information at https://en.wikipedia.org/wiki/Anemoi.
Being able to fine-tune LLMs while still keeping a data-first approach wouldn’t have been possible without the inestimable help of the open source community and all the amazing resources out there intended for the general public. We are very grateful for that, and we hope that our work can be useful for others as well.
🎩 h/t HuggingFace H4 team for their amazing work with alignment-handbook
, and also for the fruitful discussions we had with them and their support.
News
- December 1st, 2023: Notus 7B v1 is released! 🎉 Using the same DPO fine-tuning approach as Zephyr 7B Beta, but changing the data source from UltraFeedback to binarize it using the average of the different criterias, instead of the critique score. Notus 7B improved in both AlpacaEval and LM Eval Harness compared to Zephyr 7B Beta, while for MT-Bench the results were on par. More information at
v1/
.
Resources
🤗 HuggingFace Hub Collection
Available at: Hugging Face