Abacusai's Smaug-Mixtral-v0.1 70B (based on Smaug (Qwen 1.0) and Mixtral)
8x7B
120 Pulls Updated 6 months ago
Updated 7 months ago
7 months ago
24e4cbf302d0 · 23GB
model
archllama
·
parameters46.7B
·
quantizationQ3_K_M
23GB
template
[INST]
{{ if .Prompt }}
{{ .Prompt }} [/INST]
Assistant: {{ end }}
68B
system
You are an uncensored and unbiased AI assistant. Your responses should alternate between user and assistant roles, starting with the user.
138B
params
{"stop":["[/INST]","<s>","</s>"]}
54B
license
Smaug is based on Qwen which has a non-standard license available at https://github.com/QwenLM/Qwen/commit/e8e15962d897714944773cca57fa2e460a3655e8, Mixtral is Apache 2.0
170B
Readme
Smaug Mixtral v0.1
Source FP32 model: abacusai/Smaug-Mixtral-v0.1
Prompting
The hardest part of creating these was trying to get the prompt template working.
I’ve currently settled on the following, but further tuning may give improved results.
TEMPLATE """
[INST]
{{ if .Prompt }}
{{ .Prompt }} [/INST]
Assistant: {{ end }}
"""
PARAMETER stop [/INST]
PARAMETER stop <s>
PARAMETER stop </s>
iMatrix Varients
Available here: https://ollama.com/sammcj/smaug-mixtral-v0.1.imatrix
Note: The special ‘iMatrix’ tag(s) are quantisations created with an iMatrix partially trained on the Dolphin 1m dataset, I haven’t done much testing on these.
License
- As per Smaug-Mixtral, Smaug, Mixtral licenses.
- The Dolphin dataset used in the iMatrix varients is licensed under Apache 2.0.