Qwen3-coder – 30B (Q4_K_S)

This is a quantized version (Q4_K_S) of the original:

Qwen3-coder – 30B (FP16)

Overview

• Base model: Qwen3-coder – 30B (FP16)
• Quantization: Q4_K_S
• License: Apache 2.0
• Format: Ollama model
• Type: Inference-optimized snapshot

Purpose

This model was created to:

• Reduce memory usage compared to FP16
• Improve inference speed on consumer GPUs
• Enable running a 24B model on more modest hardware
• Preserve a stable snapshot of the original model

Performance / Size

• Model footprint: ~17 GB (Q4_K_S)
• VRAM usage increases with context length due to KV cache
• Suitable for GPU inference on mid-to-high range consumer hardware

Hardware Notes (estimated)

• 20 GB VRAM GPU:

  • Stable at ~4K context length in typical setups

• ~24 GB VRAM GPU :

  • Likely supports ~6K–8K context depending on backend and KV cache settings (Not tested yet)

Trade-offs

Compared to FP16:

• ✔ Lower VRAM usage
• ✔ Faster inference
• ❌ Slight reduction in output quality and precision

Usage

Run locally:

ollama run LoPld/qwen3-coder-30b-a3b-q4_K_S 

Reproducibility

Created from:

FROM qwen3-coder:30b-a3b-fp16

Quantized using:

ollama create <model> --quantize q4_K_S -f Modelfile

License

Original model: Apache 2.0 This quantized version inherits the same license terms.

Notes

• This is a frozen snapshot of the original model
• No upstream updates are expected for this version
• Intended for long-term local use and reproducibility