LoPld/ qwen3-coder-30b-a3b-q4_K_S:latest

118 Downloads Updated 2 weeks ago

Quantized Q4_K_S version of Qwen3-coder 30B FP16 model

tools

ollama run LoPld/qwen3-coder-30b-a3b-q4_K_S

curl http://localhost:11434/api/chat \
  -d '{
    "model": "LoPld/qwen3-coder-30b-a3b-q4_K_S",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

from ollama import chat

response = chat(
    model='LoPld/qwen3-coder-30b-a3b-q4_K_S',
    messages=[{'role': 'user', 'content': 'Hello!'}],
)
print(response.message.content)

import ollama from 'ollama'

const response = await ollama.chat({
  model: 'LoPld/qwen3-coder-30b-a3b-q4_K_S',
  messages: [{role: 'user', content: 'Hello!'}],
})
console.log(response.message.content)

Details

Updated 2 weeks ago

2 weeks ago

c77acbd168af · 17GB ·

model

archqwen3moe

·

parameters30.5B

·

quantizationQ4_K_S

17GB

license

Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ TERMS AND CONDITIONS FOR US

11kB

params

{ "repeat_penalty": 1.05, "stop": [ "<|im_start|>", "<|im_end|>", "<

148B

Readme

Qwen3-coder – 30B (Q4_K_S)

This is a quantized version (Q4_K_S) of the original:

Qwen3-coder – 30B (FP16)

Overview

Base model: Qwen3-coder – 30B (FP16)
Quantization: Q4_K_S
License: Apache 2.0
Format: Ollama model
Type: Inference-optimized snapshot

Purpose

This model was created to:

Reduce memory usage compared to FP16
Improve inference speed on consumer GPUs
Enable running a 24B model on more modest hardware
Preserve a stable snapshot of the original model

Performance / Size

Model footprint: ~17 GB (Q4_K_S)
VRAM usage increases with context length due to KV cache
Suitable for GPU inference on mid-to-high range consumer hardware

Hardware Notes (estimated)

20 GB VRAM GPU:
- Stable at ~4K context length in typical setups
~24 GB VRAM GPU :
- Likely supports ~6K–8K context depending on backend and KV cache settings (Not tested yet)

Trade-offs

Compared to FP16:

✔ Lower VRAM usage
✔ Faster inference
❌ Slight reduction in output quality and precision

Usage

Run locally:

ollama run LoPld/qwen3-coder-30b-a3b-q4_K_S

Reproducibility

Created from:

FROM qwen3-coder:30b-a3b-fp16

Quantized using:

ollama create <model> --quantize q4_K_S -f Modelfile

License

Original model: Apache 2.0 This quantized version inherits the same license terms.

Notes

This is a frozen snapshot of the original model
No upstream updates are expected for this version
Intended for long-term local use and reproducibility

# Qwen3-coder – 30B (Q4_K_S)

This is a **quantized version (Q4_K_S)** of the original:

> Qwen3-coder – 30B (FP16)

---

## Overview

* Base model: Qwen3-coder – 30B (FP16)
* Quantization: Q4_K_S
* License: Apache 2.0
* Format: Ollama model
* Type: Inference-optimized snapshot

---

## Purpose

This model was created to:

* Reduce memory usage compared to FP16
* Improve inference speed on consumer GPUs
* Enable running a 24B model on more modest hardware
* Preserve a stable snapshot of the original model

---

## Performance / Size

* Model footprint: ~17 GB (Q4_K_S)
* VRAM usage increases with context length due to KV cache
* Suitable for GPU inference on mid-to-high range consumer hardware

---

## Hardware Notes (estimated)

* 20 GB VRAM GPU:

* Stable at ~4K context length in typical setups

* ~24 GB VRAM GPU :

* Likely supports ~6K–8K context depending on backend and KV cache settings (Not tested yet)

---

## Trade-offs

Compared to FP16:

* ✔ Lower VRAM usage
* ✔ Faster inference
* ❌ Slight reduction in output quality and precision

---

## Usage

Run locally:

```bash
ollama run LoPld/qwen3-coder-30b-a3b-q4_K_S 
```

---

## Reproducibility

Created from:

```text
FROM qwen3-coder:30b-a3b-fp16
```

Quantized using:

```bash
ollama create <model> --quantize q4_K_S -f Modelfile
```

---

## License

Original model: Apache 2.0
This quantized version inherits the same license terms.

---

## Notes

* This is a frozen snapshot of the original model
* No upstream updates are expected for this version
* Intended for long-term local use and reproducibility

---

Paste, drop or click to upload images (.png, .jpeg, .jpg, .svg, .gif)