koesn/
wizardlm2-7b

254 1 year ago

Fixed num_ctx to 32768. This WizardLM-2 7B model is ready to use for full model's 32k contexts window.

Models

View all →

5 models

wizardlm2-7b:latest

4.1GB · 32K context window · Text · 1 year ago

wizardlm2-7b:Q4_0

4.1GB · 32K context window · Text · 1 year ago

wizardlm2-7b:Q4_K_M

4.4GB · 32K context window · Text · 1 year ago

wizardlm2-7b:Q5_K_M

5.1GB · 32K context window · Text · 1 year ago

wizardlm2-7b:Q6_K

5.9GB · 32K context window · Text · 1 year ago

Readme

WizardLM-2-7B

Model Quants Size Bit Perplexity
wizardlm2-7b:Q4_0 4.1GB 4 +0.2166 ppl
wizardlm2-7b:Q4_K_M 4.4GB 4 +0.0532 ppl
wizardlm2-7b:Q5_K_M 5.1GB 5 +0.0122 ppl
wizardlm2-7b:Q6_K 5.9GB 6 +0.0008 ppl
Config

max_position_embeddings : 32768
rope_theta : 500000.0
vocab_size : 32000

Remarks
  • ‘latest’ model points to Q4_0
  • modelfile has 32768 num_ctx activated (Ollama default only 2048)