Qwen3.8-27B Q4_K_M, default context 131072, tuned for a 32 GB NVIDIA RTX 5090.

• Registry: https://ollama.com/dna5rm/qwen3.8:27b-128k
• Upstream: qwen3.8:27b-q4_K_M (17 GB, native 262144, vision + tools + thinking)
• Requires Ollama ≥ 0.32.12 (built/pushed with 0.32.14)

ollama pull dna5rm/qwen3.8:27b-128k

Why 128K, not 256K

Weights ~18 GB. Hybrid attention: only 16 of 64 layers keep a full KV cache (4 KV heads × dim 256). FP16 KV at 262144 tokens is 16 GiB on top of weights (~34 GB+) — does not fit 32 GB.

Host env (32 GB card)

[Service]
Environment=OLLAMA_FLASH_ATTENTION=1
Environment=OLLAMA_KV_CACHE_TYPE=q8_0
Environment=OLLAMA_NUM_PARALLEL=1
Environment=OLLAMA_MAX_LOADED_MODELS=1
Environment=OLLAMA_GPU_OVERHEAD=1073741824

q8_0 KV requires flash attention. Without it Ollama stays on f16 KV and 128K will be tight / spill.

Do not use 27b-q8_0 (~30 GB weights) on 32 GB.

Modelfile (this tag)

FROM qwen3.8:27b-q4_K_M
PARAMETER num_ctx 131072
PARAMETER temperature 1
PARAMETER top_k 20
PARAMETER top_p 0.95
PARAMETER min_p 0
PARAMETER repeat_penalty 1
PARAMETER presence_penalty 0

Upstream sampling left as-is. Architecture still reports 262144 max; default allocation is 128K.

Verify

ollama show dna5rm/qwen3.8:27b-128k
ollama run dna5rm/qwen3.8:27b-128k
# another terminal:
ollama ps
# expect 100% GPU, CONTEXT 131072

Stretch / shorter

• More context on 32 GB: raise num_ctx toward 196608 only with OLLAMA_KV_CACHE_TYPE=q8_0 and nothing else on the GPU.
• Full 256K: 48 GB class, or q8 KV with almost zero headroom on 32 GB.