koesn / mistral-7b-instruct

Fixed num_ctx to 32768. This Mistral 7B v0.2 Instruct model is ready to use for full model's 32k contexts window.

7B

184 Pulls Updated 4 months ago

4 months ago

39d8715d6a19 · 4.1GB

model
llama
·
7.24B
·
Q4_0
params
{"num_ctx":32768,"stop":["<|im_start|>","<|im_end|>"]}
template
<|im_start|>system {{ .System }} <|im_start|>user {{ .Prompt }} <|im_start|>assistant
system
Perform the task to the best of your ability.

Readme

Mistral-7B-Instruct-v0.2

Model Quants Size Bit Perplexity
mistral-7b-instruct:Q4_0 4.1GB 4 +0.2166 ppl
mistral-7b-instruct:Q4_K_M 4.4GB 4 +0.0532 ppl
mistral-7b-instruct:Q5_K_M 5.1GB 5 +0.0122 ppl
mistral-7b-instruct:Q6_K 5.9GB 6 +0.0008 ppl
Config

“max_position_embeddings” : 32768
“rope_theta” : 1000000.0
“vocab_size” : 32000

Remarks
  • ‘latest’ model points to Q4_0
  • modelfile has 32768 num_ctx activated (Ollama default only 2048)