eramax / senku

https://huggingface.co/dranger003/Senku-70B-iMat.GGUF

70B

79 Pulls Updated 4 months ago

4 months ago

d05bab22a86c · 20GB

model
llama
·
69.0B
·
IQ2_XS
template
<|im_start|>system {{ .System }}<|im_end|> <|im_start|>user {{ .Prompt }}<|im_end|> <|im_start|>assistant
params
{"stop":["<|im_start|>","<|im_end|>"]}

Readme

ShinojiResearch/Senku-70B-Full

UPDATE: 85.09 EQ-Bench with ChatML template

  • EQ-Bench: (Mistral) 84.89 -> 85.09 (ChatML)
  • GSM8k: (Mistral) 77.18 -> 71.04 (ChatML)
  • Hellaswag: (Mistral) 87.67 -> ??

Finetune of miqu-70b-sf dequant of miqudev’s leak of Mistral-70B (allegedly an early mistral medium). My diffs are available under CC-0 (That is the Senku-70B repo, full includes the merge), this is a merge with the leaked model, you can use the other repository to save bandwidth.

Update: Upon further testing a score of 85.09 was achieved using ChatML instead of Mistral’s prompt.

Prompt Template

I recommend using the ChatML format instead, I will run more benchmarks. This also fixes the bug with Miqu dequant failing to provide a stop.

<|im_start|>system 
Provide some context and/or instructions to the model.
<|im_end|> 
<|im_start|>user 
The user’s message goes here
<|im_end|> 
<|im_start|>assistant <|im_end|>