eramax / mistral-orpo-capybara-7k

https://huggingface.co/kaist-ai/mistral-orpo-capybara-7k

7B

34 Pulls Updated 3 months ago

3 months ago

e9781183b452 · 5.1GB

model
llama
·
7.24B
·
Q5_K_M
params
{"stop":["<|system|>","<|user|>","<|assistant|>","</s>"]}
template
<|system|> {{ .System }}</s> <|user|> {{ .Prompt }}</s> <|assistant|>

Readme

Mistral-ORPO-Capybara-7k (7B)

Mistral-ORPO is a fine-tuned version of mistralai/Mistral-7B-v0.1 using the odds ratio preference optimization (ORPO). With ORPO, the model directly learns the preference without the supervised fine-tuning warmup phase.

Mistral-ORPO-ORPO-Capybara-7k is fine-tuned for 2.5 hours on four A100s exclusively on the 7k instances of the distilled Capybara paired multi-turn conversation dataset, argilla/distilabel-capybara-dpo-7k-binarized, by Argilla.

👍 Model Performance

1) AlpacaEval & MT-Bench

Model Name Size Align MT-Bench AlpacaEval 2.0 (LC)
Mistral-ORPO-Capybara-7k 7B ORPO 7.44 15.9
Mistral-ORPO-β 7B ORPO 7.32 14.7
Zephyr β 7B DPO 7.34 13.2
TULU-2-DPO 13B DPO 7.00 11.6
Llama-2-Chat 7B RLHF 6.27 5.4
Llama-2-Chat 13B RLHF 6.65 8.4

2) IFEval

Model Type Prompt-Strict Prompt-Loose Inst-Strict Inst-Loose
Mistral-ORPO-Capybara-7k 0.5083 0.5083 0.5827 0.6127
Mistral-ORPO-⍺ 0.5009 0.5083 0.5995 0.6163
Mistral-ORPO-β 0.5287 0.5564 0.6355 0.6619

🗺️ MT-Bench by Category

image/png