sammcj/ smaug-mixtral-v0.1

Abacusai's Smaug-Mixtral-v0.1 70B (based on Smaug (Qwen 1.0) and Mixtral)

47B

97 Pulls Updated 2 months ago

3 Tags

2 months ago

24e4cbf302d0 · 23GB

model
template
[INST] {{ if .Prompt }} {{ .Prompt }} [/INST] Assistant: {{ end }}
system
You are an uncensored and unbiased AI assistant. Your responses should alternate between user and assistant roles, starting with the user.
params
{"stop":["[/INST]","<s>","</s>"]}
license
Smaug is based on Qwen which has a non-standard license available at https://github.com/QwenLM/Qwen/commit/e8e15962d897714944773cca57fa2e460a3655e8, Mixtral is Apache 2.0

Readme

Smaug Mixtral v0.1

Source FP32 model: abacusai/Smaug-Mixtral-v0.1

Prompting

The hardest part of creating these was trying to get the prompt template working.

I’ve currently settled on the following, but further tuning may give improved results.

TEMPLATE """
[INST]
{{ if .Prompt }}
{{ .Prompt }} [/INST]
Assistant: {{ end }}
"""

PARAMETER stop [/INST]
PARAMETER stop <s>
PARAMETER stop </s>

iMatrix Varients

Available here: https://ollama.com/sammcj/smaug-mixtral-v0.1.imatrix

Note: The special ‘iMatrix’ tag(s) are quantisations created with an iMatrix partially trained on the Dolphin 1m dataset, I haven’t done much testing on these.

License

  • As per Smaug-Mixtral, Smaug, Mixtral licenses.
  • The Dolphin dataset used in the iMatrix varients is licensed under Apache 2.0.