wangshenzhi出品，模型来源：https://hf-mirror.com/shenzhi-wang/Llama3-8B-Chinese-Chat-GGUF-4bit，基于Meta-Llama-3-8B-Instruct
262 Pulls Updated 5 months ago
Updated 5 months ago
5 months ago
b4a46dbb319f · 4.7GB
Readme
Llama3-8B-Chinese-Chat-v2.1-Q4_0（from wangshenzhi）
2024-5-19
Llama3-Chinese必要性： Llama3对中文支持并不好，经常会出现中文问题给出英文答案，本人收集了市场上Llama3的中文微调版本，量化并适配了modelfile，现分享给大家。
以下来自wangshenzhi
[May 6, 2024] We now introduce Llama3-8B-Chinese-Chat-v2.1! Compared to v1, the training dataset of v2.1 is 5x larger (~100K preference pairs), and it exhibits significant enhancements, especially in roleplay, function calling, and math capabilities! Compared to v2, v2.1 surpasses v2 in math and is less prone to including English words in Chinese responses. The training dataset of Llama3-8B-Chinese-Chat-v2.1 will be released soon. If you love our Llama3-8B-Chinese-Chat-v1 or v2, you won’t want to miss out on Llama3-8B-Chinese-Chat-v2.1!
Model Summary Llama3-8B-Chinese-Chat is an instruction-tuned language model for Chinese & English users with various abilities such as roleplaying & tool-using built upon the Meta-Llama-3-8B-Instruct model.
Developed by: Shenzhi Wang (王慎执) and Yaowei Zheng (郑耀威)
License: Llama-3 License Base Model: Meta-Llama-3-8B-Instruct Model Size: 8.03B Context length: 8K
This is the first model specifically fine-tuned for Chinese & English user through ORPO [1] based on the Meta-Llama-3-8B-Instruct model.
Compared to the original Meta-Llama-3-8B-Instruct model, our Llama3-8B-Chinese-Chat-v1 model significantly reduces the issues of “Chinese questions with English answers” and the mixing of Chinese and English in responses.
Compared to Llama3-8B-Chinese-Chat-v1, our Llama3-8B-Chinese-Chat-v2 model significantly increases the training data size (from 20K to 100K), which introduces great performance enhancement, especially in roleplay, tool using, and math.
[1] Hong, Jiwoo, Noah Lee, and James Thorne. “Reference-free Monolithic Preference Optimization with Odds Ratio.” arXiv preprint arXiv:2403.07691 (2024).
Training framework: LLaMA-Factory.
Training details:
epochs: 2 learning rate: 5e-6 learning rate scheduler type: cosine Warmup ratio: 0.1 cutoff len (i.e. context length): 8192 orpo beta (i.e. \(\lambda\) in the ORPO paper): 0.05 global batch size: 128 fine-tuning type: full parameters optimizer: paged_adamw_32bit
模型来源：https://hf-mirror.com/shenzhi-wang/Llama3-8B-Chinese-Chat-GGUF-4bit
QQ: 83649263