catsarethebest/
qwen2.5-N2

51 1 month ago

Qwen2.5 0.5B and 1.5B instruct fine-tuned with a synthetic dataset

tools 1.5b

Models

View all →

2 models

qwen2.5-N2:latest

995MB · 32K context window · Text · 1 month ago

qwen2.5-N2:1.5b

3.1GB · 32K context window · Text · 1 month ago

Readme

Qwen2.5-N2

Base Model

Qwen2.5-N2 is a fine-tuned version of Qwen2.5-0.5B-Instruct and Qwen2.5-1.5B-Instruct..

Performance

Qwen2.5-N2 is the newer version of Qwen2.5-N. Even though it was trained on less data, it performs noticeably better.

Datasets

This model was trained using one dataset available on Hugging Face:

Training

Qwen2.5-N2 was trained using a reverse curriculum learning approach. The data was divided into easy, medium, and hard categories, and the model was trained in stages that gradually shifted focus between difficulty levels:

Training Stage Epoch Easy Data Medium Data Hard Data
Stage 1: Hard 1–3 0% 20% 80%
Stage 2: Medium Transition 4–6 10% 50% 40%
Stage 3: Medium Dominance 7–9 25% 60% 15%
Stage 4: Easy Reinforcement 10 40% 50% 10%

All work is released under the original Qwen2.5 Apache 2.0 license.