jpacifico/
chocolatine-14b

279 8 months ago

Quantized 4-bit version of the original Chocolatine LLM, best performing 13B model on the OpenLLM Leaderboard.

Models

View all →

1 model

chocolatine-14b:latest

8.6GB · 4K context window · Text · 8 months ago

Readme

Chocolatine-14B-Instruct-DPO-v1.2 (Q4_K_M)

DPO fine-tuned of microsoft/Phi-3-medium-4k-instruct (14B params)
using the jpacifico/french-orca-dpo-pairs-revised rlhf dataset.
Training in French also improves the model in English, surpassing the performances of its base model.
Window context = 4k tokens

Limitations

The Chocolatine model series is a quick demonstration that a base model can be easily fine-tuned to achieve compelling performance.
It does not have any moderation mechanism.

  • Developed by: Jonathan Pacifico, 2024
  • Model type: LLM
  • Language(s) (NLP): French, English
  • License: MIT