A new small reasoning model fine-tuned from the Qwen 2.5 3B Instruct model. I-Quants models.
75 Pulls Updated 3 months ago
Updated 3 months ago
3 months ago
4eed365cc9d6 · 2.0GB
Readme
A new model fine-tuned from the Qwen2.5-3b-Instruct model.
- Quantization from
fp32 - Using i-matrix
calibration_datav3.txt
SmallThinker is designed for the following use cases:
- Edge Deployment: Its small size makes it ideal for deployment on resource-constrained devices.
- Draft Model for QwQ-32B-Preview: SmallThinker can serve as a fast and efficient draft model for the larger QwQ-32B-Preview model, yielding a 70% speedup.
For achieving reasoning capabilities, it’s crucial to generate long chains of COT reasoning. Therefore, based on QWQ-32B-Preview, the authors used various synthetic techniques(such as personahub) to create the QWQ-LONGCOT-500K dataset. Compared to other similar datasets, over 75% of the author’s samples have output tokens exceeding 8K. To encourage research in the open-source community, the dataset was also made publicly available.