10.7B model, depth upscaled version of two mistral based finetunes

24 Pulls Updated 8 months ago

Readme

Chikuma

Chikuma is a 10.7B parameter model and is a merge of the following models using LazyMergekit:
* sethuiyer/SynthIQ-7b
* openchat/openchat-3.5-0106

The name “Chikuma” is inspired by the Chikuma River, the longest in Japan, known for its continuous flow and meandering path.
This metaphorically represents the model’s depth, fluidity, and adaptability in processing and understanding language.

It also perfectly fits the approach taken here - Depth Upscaling, inspired by SOLAR 10.7B.