koesn / wizardlm2-7b

Fixed num_ctx to 32768. This WizardLM-2 7B model is ready to use for full model's 32k contexts window.

7B

100 Pulls Updated 4 months ago

4 months ago

0fa4bbf46bc7 · 4.1GB

model
llama
·
7.24B
·
Q4_0
params
{"num_ctx":32768,"stop":["<|im_start|>","<|im_end|>"]}
template
{{ if .System }}{{ .System }} {{ end }}{{ if .Prompt }}USER: {{ .Prompt }} {{ end }}ASSISTANT: {{ .Response }}
system
A chat between a curious user and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the user's questions.

Readme

WizardLM-2-7B

Model Quants Size Bit Perplexity
wizardlm2-7b:Q4_0 4.1GB 4 +0.2166 ppl
wizardlm2-7b:Q4_K_M 4.4GB 4 +0.0532 ppl
wizardlm2-7b:Q5_K_M 5.1GB 5 +0.0122 ppl
wizardlm2-7b:Q6_K 5.9GB 6 +0.0008 ppl
Config

max_position_embeddings : 32768
rope_theta : 500000.0
vocab_size : 32000

Remarks
  • ‘latest’ model points to Q4_0
  • modelfile has 32768 num_ctx activated (Ollama default only 2048)