fireboi25/ Qwythos-V2:q4

218 Downloads Updated 6 days ago

Qwythos-V2-Abliterated-GGUF + Patched to natively support VSC extension by patching the jinja template issue

ollama run fireboi25/Qwythos-V2:q4

curl http://localhost:11434/api/chat \
  -d '{
    "model": "fireboi25/Qwythos-V2:q4",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

from ollama import chat

response = chat(
    model='fireboi25/Qwythos-V2:q4',
    messages=[{'role': 'user', 'content': 'Hello!'}],
)
print(response.message.content)

import ollama from 'ollama'

const response = await ollama.chat({
  model: 'fireboi25/Qwythos-V2:q4',
  messages: [{role: 'user', content: 'Hello!'}],
})
console.log(response.message.content)

Details

Updated 6 days ago

6 days ago

f1131beb2f2b · 5.6GB ·

model

archqwen35

·

parameters8.95B

·

quantizationQ4_K_M

5.6GB

Readme

Qwythos V2 GGUF

Qwythos-V2 is a 9B Qwen 3.5 architecture model converted to GGUF and published with Q8, Q5, and Q4 quantizations.

These builds include an Ollama-compatible chat template patch for Anthropic-style clients such as Claude Code. The upstream template required every system message to appear at the beginning of the message list. Some clients send system messages later in the conversation, which caused Ollama to return a parser error. These GGUFs fold all system messages into the initial system block so they work without a compatibility proxy.

Ollama

ollama pull fireboi25/qwythos-v2:q4
ollama pull fireboi25/qwythos-v2:q5
ollama pull fireboi25/qwythos-v2:q8

Run one of the quantized tags:

ollama run fireboi25/qwythos-v2:q4

Quantizations

Tag	Quantization	Approx. Size	Notes
`fireboi25/qwythos-v2:q4`	`Q4_K_M`	5.6 GB	Smallest and fastest to load
`fireboi25/qwythos-v2:q5`	`Q5_K_M`	6.5 GB	Balanced size and quality
`fireboi25/qwythos-v2:q8`	`Q8_0`	9.5 GB	Highest quality of these uploads

Local Files

Patched GGUF files:

gguf/Qwythos-V2-Q4_K_M-ollama.gguf
gguf/Qwythos-V2-Q5_K_M-ollama.gguf
gguf/Qwythos-V2-Q8_0-ollama.gguf

Supporting Ollama files:

ollama/Modelfile.Qwythos-V2-Q4
ollama/Modelfile.Qwythos-V2-Q5
ollama/Modelfile.Qwythos-V2-Q8
ollama/chat_template_qwythos_v2_ollama.jinja

Compatibility Fix

The patched template addresses this Ollama error:

Unable to generate parser for this template.
Jinja Exception: System message must be at the beginning.

Instead of requiring system messages to be first, the patched template collects all system messages and renders them inside the initial system prompt. This allows direct use with Claude Code and other Anthropic-compatible clients without running a proxy.

Model Details

Architecture: qwen35
Parameters: 9.0B
Context length: 1,048,576
Embedding length: 4096
Capabilities: tools, thinking, completion

Source

Converted from:

https://huggingface.co/usermma/Qwythos-9B-v2-Heretic-Abliterated

Notes

This is a thinking model, so some clients may expose reasoning or <think> output depending on their settings and how they call Ollama.

# Qwythos V2 GGUF

Qwythos-V2 is a 9B Qwen 3.5 architecture model converted to GGUF and published with Q8, Q5, and Q4 quantizations.

These builds include an Ollama-compatible chat template patch for Anthropic-style clients such as Claude Code. The upstream template required every `system` message to appear at the beginning of the message list. Some clients send system messages later in the conversation, which caused Ollama to return a parser error. These GGUFs fold all system messages into the initial system block so they work without a compatibility proxy.

## Ollama

```bash
ollama pull fireboi25/qwythos-v2:q4
ollama pull fireboi25/qwythos-v2:q5
ollama pull fireboi25/qwythos-v2:q8
```

Run one of the quantized tags:

```bash
ollama run fireboi25/qwythos-v2:q4
```

## Quantizations

| Tag | Quantization | Approx. Size | Notes |
| --- | --- | ---: | --- |
| `fireboi25/qwythos-v2:q4` | `Q4_K_M` | 5.6 GB | Smallest and fastest to load |
| `fireboi25/qwythos-v2:q5` | `Q5_K_M` | 6.5 GB | Balanced size and quality |
| `fireboi25/qwythos-v2:q8` | `Q8_0` | 9.5 GB | Highest quality of these uploads |

## Local Files

Patched GGUF files:

```text
gguf/Qwythos-V2-Q4_K_M-ollama.gguf
gguf/Qwythos-V2-Q5_K_M-ollama.gguf
gguf/Qwythos-V2-Q8_0-ollama.gguf
```

Supporting Ollama files:

```text
ollama/Modelfile.Qwythos-V2-Q4
ollama/Modelfile.Qwythos-V2-Q5
ollama/Modelfile.Qwythos-V2-Q8
ollama/chat_template_qwythos_v2_ollama.jinja
```

## Compatibility Fix

The patched template addresses this Ollama error:

```text
Unable to generate parser for this template.
Jinja Exception: System message must be at the beginning.
```

Instead of requiring `system` messages to be first, the patched template collects all system messages and renders them inside the initial system prompt. This allows direct use with Claude Code and other Anthropic-compatible clients without running a proxy.

## Model Details

```text
Architecture: qwen35
Parameters: 9.0B
Context length: 1,048,576
Embedding length: 4096
Capabilities: tools, thinking, completion
```

## Source

Converted from:

```text
https://huggingface.co/usermma/Qwythos-9B-v2-Heretic-Abliterated
```

## Notes

This is a thinking model, so some clients may expose reasoning or `<think>` output depending on their settings and how they call Ollama.

Paste, drop or click to upload images (.png, .jpeg, .jpg, .svg, .gif)