Fixed num_ctx to 32768. This WizardLM-2 7B model is ready to use for full model's 32k contexts window.
109 Pulls Updated 6 months ago
Updated 6 months ago
6 months ago
0fa4bbf46bc7 · 4.1GB
model
archllama
·
parameters7.24B
·
quantizationQ4_0
4.1GB
params
{"num_ctx":32768,"stop":["\u003c|im_start|\u003e","\u003c|im_end|\u003e"]}
75B
template
{{ if .System }}{{ .System }} {{ end }}{{ if .Prompt }}USER: {{ .Prompt }} {{ end }}ASSISTANT: {{ .
112B
system
A chat between a curious user and an artificial intelligence assistant. The assistant gives helpful
156B
Readme
WizardLM-2-7B
| Model Quants | Size | Bit | Perplexity |
|---|---|---|---|
| wizardlm2-7b:Q4_0 | 4.1GB | 4 | +0.2166 ppl |
| wizardlm2-7b:Q4_K_M | 4.4GB | 4 | +0.0532 ppl |
| wizardlm2-7b:Q5_K_M | 5.1GB | 5 | +0.0122 ppl |
| wizardlm2-7b:Q6_K | 5.9GB | 6 | +0.0008 ppl |
Config
max_position_embeddings : 32768
rope_theta : 500000.0
vocab_size : 32000
Remarks
- ‘latest’ model points to Q4_0
- modelfile has 32768 num_ctx activated (Ollama default only 2048)