koesn/wizardlm2-7b:Q4_K

koesn/

wizardlm2-7b:Q4_K_M

254 Downloads Updated 1 year ago

Fixed num_ctx to 32768. This WizardLM-2 7B model is ready to use for full model's 32k contexts window.

Updated 1 year ago

1 year ago

cf9ad469dee1 · 4.4GB ·

archllama

parameters7.24B

quantizationQ4_K_M

4.4GB

{ "num_ctx": 32768, "stop": [ "<|im_start|>", "<|im_end|>" ] }

75B

{{ if .System }}{{ .System }} {{ end }}{{ if .Prompt }}USER: {{ .Prompt }} {{ end }}ASSISTANT: {{ .R

112B

A chat between a curious user and an artificial intelligence assistant. The assistant gives helpful,

156B

WizardLM-2-7B

Model Quants	Size	Bit	Perplexity
wizardlm2-7b:Q4_0	4.1GB	4	+0.2166 ppl
wizardlm2-7b:Q4_K_M	4.4GB	4	+0.0532 ppl
wizardlm2-7b:Q5_K_M	5.1GB	5	+0.0122 ppl
wizardlm2-7b:Q6_K	5.9GB	6	+0.0008 ppl

max_position_embeddings : 32768
rope_theta : 500000.0
vocab_size : 32000