150 Downloads Updated 1 year ago
Name
5 models
llama-varco:latest
16GB · 8K context window · Text · 1 year ago
llama-varco:q4_K_M
4.9GB · 8K context window · Text · 1 year ago
llama-varco:q5_K_S
5.6GB · 8K context window · Text · 1 year ago
llama-varco:q5_K_M
5.7GB · 8K context window · Text · 1 year ago
llama-varco:q6_K
6.6GB · 8K context window · Text · 1 year ago
This model was not trained by the uploader! simply uploaded to Ollama with the weights as is. Original source and weights: here
Llama-VARCO-8B-Instruct is a generative model built with Llama, specifically designed to excel in Korean through additional training. The model uses continual pre-training with both Korean and English datasets to enhance its understanding and generation capabilites in Korean, while also maintaining its proficiency in English. It performs supervised fine-tuning (SFT) and direct preference optimization (DPO) in Korean to align with human preferences.
We used the LogicKor code to measure performance. For the judge model, we used the officially recommended gpt-4-1106-preview. The score includes only the 0-shot evaluation provided in the default.
| Model | Math | Reasoning | Writing | Coding | Understanding | Grammer | Single turn | Multi turn | Overall |
|---|---|---|---|---|---|---|---|---|---|
| Llama-VARCO-8B-Instruct | 6.71 / 8.57 | 8.86 / 8.29 | 9.86 / 9.71 | 8.86 / 9.29 | 9.29 / 10.0 | 8.57 / 7.86 | 8.69 | 8.95 | 8.82 |
| EXAONE-3.0-7.8B-Instruct | 6.86 / 7.71 | 8.57 / 6.71 | 10.0 / 9.29 | 9.43 / 10.0 | 10.0 / 10.0 | 9.57 / 5.14 | 9.07 | 8.14 | 8.61 |
| Meta-Llama-3.1-8B-Instruct | 4.29 / 4.86 | 6.43 / 6.57 | 6.71 / 5.14 | 6.57 / 6.00 | 4.29 / 4.14 | 6.00 / 4.00 | 5.71 | 5.12 | 5.42 |
| Gemma-2-9B-Instruct | 6.14 / 5.86 | 9.29 / 9.0 | 9.29 / 8.57 | 9.29 / 9.14 | 8.43 / 8.43 | 7.86 / 4.43 | 8.38 | 7.57 | 7.98 |
| Qwen2-7B-Instruct | 5.57 / 4.86 | 7.71 / 6.43 | 7.43 / 7.00 | 7.43 / 8.00 | 7.86 / 8.71 | 6.29 / 3.29 | 7.05 | 6.38 | 6.71 |