smtek/ Qwen3.8-27B-AD:IQ4_XS

31 3 days ago

AtomicChat/Qwen3.8-27B-GGUF IQ4_XS optimized for 24 GB

vision 
ollama run smtek/Qwen3.8-27B-AD:IQ4_XS

Details

3 days ago

6942e2b163a8 · 17GB ·

model
qwen35
·
27.3B
·
Q8_0
projector
clip
·
461M
·
F16
template
{{ .Prompt }}
params
{ "draft_num_predict": 3, "min_p": 0, "num_batch": 512, "num_ctx": 163840, "num_

Readme

The Qwen3.8-27B model optimized for Ollama, utilizing the Atomic Dynamic (AD) layout structure to reduce quantization loss.

Model Overview

  • Model Name: smtek/Qwen3.8-27B-AD
  • File Size: ~17 GB
  • Context Window: 160K tokens
  • Capabilities: Text, Image (Vision)
  • Default Tag: IQ4_XS

Recommended Optimization

By default, Ollama allocates the KV-Cache in f16, which will exceed VRAM over long contexts. To run stably up to the full 256K window, set these environment variables in your system (/etc/systemd/system/ollama.service.d/override.conf on Linux):

[Service]
Environment="OLLAMA_FLASH_ATTENTION=1"
Environment="OLLAMA_KV_CACHE_TYPE=q8_0"
  • Flash Attention: Dramatically reduces context memory footprint.
  • q8_0 Cache: Compresses context tensors without the severe logic degradation caused by 4-bit (q4_0) cache quantization.

After editing, reload and restart:

sudo systemctl daemon-reload
sudo systemctl restart ollama

For more details, please refer to the source repository on https://huggingface.co/AtomicChat/Qwen3.8-27B-GGUF