smtek/ Qwen3.8-27B-AD

31 3 days ago

AtomicChat/Qwen3.8-27B-GGUF IQ4_XS optimized for 24 GB

vision 
ollama run smtek/Qwen3.8-27B-AD:IQ4_XS

Models

View all →

1 model

Qwen3.8-27B-AD:IQ4_XS

17GB · 256K context window · Text, Image · 3 days ago

Readme

The Qwen3.8-27B model optimized for Ollama, utilizing the Atomic Dynamic (AD) layout structure to reduce quantization loss.

Model Overview

  • Model Name: smtek/Qwen3.8-27B-AD
  • File Size: ~17 GB
  • Context Window: 160K tokens
  • Capabilities: Text, Image (Vision)
  • Default Tag: IQ4_XS

Recommended Optimization

By default, Ollama allocates the KV-Cache in f16, which will exceed VRAM over long contexts. To run stably up to the full 256K window, set these environment variables in your system (/etc/systemd/system/ollama.service.d/override.conf on Linux):

[Service]
Environment="OLLAMA_FLASH_ATTENTION=1"
Environment="OLLAMA_KV_CACHE_TYPE=q8_0"
  • Flash Attention: Dramatically reduces context memory footprint.
  • q8_0 Cache: Compresses context tensors without the severe logic degradation caused by 4-bit (q4_0) cache quantization.

After editing, reload and restart:

sudo systemctl daemon-reload
sudo systemctl restart ollama

For more details, please refer to the source repository on https://huggingface.co/AtomicChat/Qwen3.8-27B-GGUF