mannix/
wizardlm2:7b-iq2_xs

83 1 year ago

State of the art large language model from Microsoft AI with improved performance on complex chat, multilingual, reasoning and agent use cases. More quantizations.

1 year ago

59c096c63c6e · 2.2GB

model
llama
·
7.24B
·
IQ2_XS
system
A chat between a curious user and an artificial intelligence assistant. The assistant gives helpful,
params
{ "num_ctx": 4096, "stop": [ "USER:", "ASSISTANT:" ] }
template
{{ if .System }}{{ .System }} {{ end }}{{ if .Prompt }}USER: {{ .Prompt }} {{ end }}ASSISTANT: {{ .R

Readme

WizardLM-2 is a next generation state-of-the-art large language model with improved performance on complex chat, multilingual, reasoning and agent use cases. This family includes three cutting-edge models:

  • wizardlm2:7b: fastest model, comparable performance with 10x larger open-source models. All quantizations are made with the i-matrix.
  • wizardlm2:8x22b: the most advanced model, and the best opensource LLM in Microsoft’s internal evaluation on highly complex tasks. Not using the i-matrix for now.

These are additionals quantizations from the official fp16 model: (wizardlm2)[https://ollama.com/library/wizardlm2]