koesn/mistral-7b-instruct:Q4_K

koesn/

mistral-7b-instruct:Q4_K_M

1,335 Downloads Updated 1 year ago

Fixed num_ctx to 32768. This Mistral 7B v0.2 Instruct model is ready to use for full model's 32k contexts window.

Updated 1 year ago

1 year ago

c8a5a62846a3 · 4.4GB

archllama

parameters7.24B

quantizationQ4_K_M

4.4GB

<|im_start|>system {{ .System }} <|im_start|>user {{ .Prompt }} <|im_start|>assistant

87B

Perform the task to the best of your ability.

47B

{ "num_ctx": 32768, "stop": [ "<|im_start|>", "<|im_end|>" ] }

75B

Mistral-7B-Instruct-v0.2

Model Quants	Size	Bit	Perplexity
mistral-7b-instruct:Q4_0	4.1GB	4	+0.2166 ppl
mistral-7b-instruct:Q4_K_M	4.4GB	4	+0.0532 ppl
mistral-7b-instruct:Q5_K_M	5.1GB	5	+0.0122 ppl
mistral-7b-instruct:Q6_K	5.9GB	6	+0.0008 ppl

“max_position_embeddings” : 32768
“rope_theta” : 1000000.0
“vocab_size” : 32000