koesn/mistral-7b-instruct

koesn/

mistral-7b-instruct

2,168 Downloads Updated 1 year ago

Fixed num_ctx to 32768. This Mistral 7B v0.2 Instruct model is ready to use for full model's 32k contexts window.

Name

5 models

Size

Context

Input

mistral-7b-instruct:latest

4.1GB · 32K context window · Text · 1 year ago

mistral-7b-instruct:latest

4.1GB

32K

Text

mistral-7b-instruct:Q4_0

4.1GB · 32K context window · Text · 1 year ago

mistral-7b-instruct:Q4_0

4.1GB

32K

Text

mistral-7b-instruct:Q4_K_M

4.4GB · 32K context window · Text · 1 year ago

mistral-7b-instruct:Q4_K_M

4.4GB

32K

Text

mistral-7b-instruct:Q5_K_M

5.1GB · 32K context window · Text · 1 year ago

mistral-7b-instruct:Q5_K_M

5.1GB

32K

Text

mistral-7b-instruct:Q6_K

5.9GB · 32K context window · Text · 1 year ago

mistral-7b-instruct:Q6_K

5.9GB

32K

Text

Mistral-7B-Instruct-v0.2

Model Quants	Size	Bit	Perplexity
mistral-7b-instruct:Q4_0	4.1GB	4	+0.2166 ppl
mistral-7b-instruct:Q4_K_M	4.4GB	4	+0.0532 ppl
mistral-7b-instruct:Q5_K_M	5.1GB	5	+0.0122 ppl
mistral-7b-instruct:Q6_K	5.9GB	6	+0.0008 ppl

“max_position_embeddings” : 32768
“rope_theta” : 1000000.0
“vocab_size” : 32000