vanilj/qwq-32b-iq4

vanilj/ qwq-32b-iq4_xs

248 Downloads Updated 1 year ago

IQ4_XS quant of Qwen/QwQ-32B

tools

ollama run vanilj/qwq-32b-iq4_xs

curl http://localhost:11434/api/chat \
  -d '{
    "model": "vanilj/qwq-32b-iq4_xs",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

from ollama import chat

response = chat(
    model='vanilj/qwq-32b-iq4_xs',
    messages=[{'role': 'user', 'content': 'Hello!'}],
)
print(response.message.content)

import ollama from 'ollama'

const response = await ollama.chat({
  model: 'vanilj/qwq-32b-iq4_xs',
  messages: [{role: 'user', content: 'Hello!'}],
})
console.log(response.message.content)

Applications

Claude Code ollama launch claude --model vanilj/qwq-32b-iq4_xs

Codex ollama launch codex --model vanilj/qwq-32b-iq4_xs

OpenCode ollama launch opencode --model vanilj/qwq-32b-iq4_xs

OpenClaw ollama launch openclaw --model vanilj/qwq-32b-iq4_xs

Models

View all →

Name

4 models

Size

Context

Input

qwq-32b-iq4_xs:latest

18GB · 128K context window · Text · 1 year ago

qwq-32b-iq4_xs:latest

18GB

128K

Text

qwq-32b-iq4_xs:long

18GB · 128K context window · Text · 1 year ago

qwq-32b-iq4_xs:long

18GB

128K

Text

qwq-32b-iq4_xs:medium

18GB · 128K context window · Text · 1 year ago

qwq-32b-iq4_xs:medium

18GB

128K

Text

qwq-32b-iq4_xs:short

18GB · 128K context window · Text · 1 year ago

qwq-32b-iq4_xs:short

18GB

128K

Text

Readme

I added the `:short`, `medium`, and `long` tags, each one have a system prompt designed to keep the thinking short, medium or long. It also sets the default context length to 16k.

GGUF from bartowski/QwQ-32B-GGUF.

Original model from Qwen/QwQ-32B.

The parameters are from the recommended Usage Guidelines.

IQ4_XS quant of Qwen/QwQ-32B

Applications

Models

Readme

I added the :short, medium, and long tags, each one have a system prompt designed to keep the thinking short, medium or long. It also sets the default context length to 16k.

I added the `:short`, `medium`, and `long` tags, each one have a system prompt designed to keep the thinking short, medium or long. It also sets the default context length to 16k.