Qwen3.8-27B — Ollama Model (32 GB)

Model: oamazonasgabriel/qwen3.8-27b:q4-km-32gbGPU

DESCRIPTION

A full-quality Qwen3.8-27B in Q4_K_M quantization, packaged for hardware with 32 GB+ VRAM. Unlike MoE models, this is a dense 27.8B-parameter model — every parameter is active on every token — paired with a hybrid attention architecture (linear + full attention) that delivers both long context and fast inference.

The ideal heavy-lift model for: dual RTX 5090 · RTX 4090 ×2 · RTX 5080 ×2 · Apple Silicon M2/M3/M4 Ultra · any setup with 32 GB+ unified memory.

🔗 llama.cpp configuration & GGUF files: Need the full 256K context or want to run on less VRAM (via --no-kv-offload)? Grab the ready-to-run llama.cpp config from the Hugging Face repo → impacte/Qwen3.8-27B-GGUF

Key Features

• Dense 27.8B model — all parameters active, no MoE routing, maximum quality per token
• Hybrid attention: 48 linear-attention layers + 16 full-attention layers (3:1 ratio) — efficient long-context without quadratic cost
• 256K native context (262,144 tokens) — 64K configured for VRAM budget
• Multimodal: Text + Image + Video (via mmproj projector)
• Thinking mode: Enabled by default — step-by-step reasoning in thinking blocks
• Tool calling: Native support via <tool_call><function=...> XML format
• Q4_K_M quantization: imatrix-calibrated from bartowski
• Apache 2.0 license: free for commercial and personal use

Architecture

REQUIREMENTS

Platform support: - NVIDIA GPU (dual): 2× 16 GB (RTX 5060 Ti ×2, RTX 5080 ×2, etc.) - NVIDIA GPU (single): 32 GB+ VRAM (RTX 5090, A6000, L40, etc.) - Apple Silicon: Mac with 32 GB+ unified memory (M2/M3/M4 Ultra) - AMD GPU: ROCm-compatible with 32 GB+ VRAM

💡 Why 32 GB? The model weights are ~17.8 GB at Q4_K_M. With a 64K context window and f16 KV cache, total VRAM usage is ~24-27 GB. 32 GB gives comfortable headroom; 48 GB unlocks the full 256K context.

QUICK START

1. Install Ollama

# macOS (Homebrew)
brew install ollama

# Linux
curl -fsSL https://ollama.com/install.sh | sh

# Windows
# Download from https://ollama.com/download

2. Pull & Run

# Pull the model (downloads ~18 GB)
ollama pull oamazonasgabriel/qwen3.8-27b:q4-km-32gbGPU

# Run interactively
ollama run oamazonasgabriel/qwen3.8-27b:q4-km-32gbGPU

# Single prompt
ollama run oamazonasgabriel/qwen3.8-27b:q4-km-32gbGPU \
  "Explain quantum computing in simple terms"

3. (Optional) Max Context

For the full 128K+ context on 48 GB VRAM, set this before starting Ollama:

export OLLAMA_KV_CACHE_TYPE=q4_0

# Linux permanent:
sudo systemctl edit ollama
# Add: Environment="OLLAMA_KV_CACHE_TYPE=q4_0"
# Then: sudo systemctl daemon-reload && sudo systemctl restart ollama

# macOS permanent:
launchctl setenv OLLAMA_KV_CACHE_TYPE q4_0
# Then restart Ollama from Applications

USAGE

CLI

# Interactive chat
ollama run oamazonasgabriel/qwen3.8-27b:q4-km-32gbGPU

# With system prompt
ollama run oamazonasgabriel/qwen3.8-27b:q4-km-32gbGPU \
  --system "You are a senior software architect. Be precise and thorough."

REST API

# Chat completion
curl -s http://127.0.0.1:11434/api/chat \
  -d '{
    "model": "oamazonasgabriel/qwen3.8-27b:q4-km-32gbGPU",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

# OpenAI-compatible endpoint
curl -s http://127.0.0.1:11434/v1/chat/completions \
  -d '{
    "model": "oamazonasgabriel/qwen3.8-27b:q4-km-32gbGPU",
    "messages": [{"role": "user", "content": "Hello!"}],
    "stream": false
  }'

Python (ollama library)

pip install ollama

import ollama

# Chat
response = ollama.chat(
    model='oamazonasgabriel/qwen3.8-27b:q4-km-32gbGPU',
    messages=[{'role': 'user', 'content': 'Write a Python decorator for caching'}],
)
print(response.message.content)

# With thinking mode (reasoning_content is returned separately)
response = ollama.chat(
    model='oamazonasgabriel/qwen3.8-27b:q4-km-32gbGPU',
    messages=[{'role': 'user', 'content': 'Debug this complex concurrency bug'}],
    options={'enable_thinking': True},
)

JavaScript (ollama.js)

npm install ollama

import ollama from 'ollama'

const response = await ollama.chat({
  model: 'oamazonasgabriel/qwen3.8-27b:q4-km-32gbGPU',
  messages: [{ role: 'user', content: 'Design a REST API schema' }],
})
console.log(response.message.content)

SAMPLING PARAMETERS

These are baked into the model via its Modelfile:

MEMORY & PERFORMANCE

VRAM Budget

Context Window Scaling

On 32 GB VRAM:

Performance

• Prompt processing: ~60-120 tok/s (hybrid linear attention is fast on long prompts)
• Text generation: ~20-40 tok/s on 32 GB dual-GPU
• Model load time: ~60-90 seconds (~18 GB load)

The hybrid architecture (3:1 linear:full attention) means prompt processing stays near-linear even for very long contexts, unlike pure-attention models that degrade quadratically.

Opencode Integration

Add to ~/.config/opencode/opencode.jsonc:

"oamazonasgabriel/qwen3.8-27b:q4-km-32gbGPU": {
  "name": "oamazonasgabriel/qwen3.8-27b:q4-km-32gbGPU",
  "options": {
    "supportsThinking": true,
    "contextWindow": 65536
  }
}

Use as agent:

"model": "ollama/oamazonasgabriel/qwen3.8-27b:q4-km-32gbGPU"

Or launch directly:

ollama launch opencode --model oamazonasgabriel/qwen3.8-27b:q4-km-32gbGPU

TROUBLESHOOTING

Model fails to load — “unable to allocate buffer”

Cause: Not enough free VRAM. This is a 17.8 GB model and needs 32 GB+ VRAM. Fix: Free at least 24 GB before loading, or run via llama.cpp with --no-kv-offload (KV cache in system RAM).

CPU offloading detected

Run ollama ps — if CPU% appears, reduce num_ctx or set OLLAMA_KV_CACHE_TYPE=q4_0.

Slow generation

On 32 GB you should get 20-40 tok/s. If slower: - Check for CPU offloading (ollama ps) - Verify GPU is on CUDA (not Vulkan) - Close competing GPU processes

Vision not working

This Ollama tag is text-focused. For multimodal (image/video), use the llama.cpp setup with the mmproj projector — see the Hugging Face repo.

ALTERNATIVE MODELS

# Full 256K context via llama.cpp (KV cache in system RAM)
# See: https://huggingface.co/impacte/Qwen3.8-27B-GGUF

# Smaller dense Qwen (needs 16 GB)
ollama pull oamazonasgabriel/qwen3.5-9b:q4-16gbGPU

# MoE Qwen for 24 GB setups
ollama pull oamazonasgabriel/qwen3.6-35b-a3b:q4-24gbGPU

# Higher precision (needs ~48 GB VRAM)
ollama pull hf.co/bartowski/Qwen3.8-27B-GGUF:Q8_0

CREDITS

LINKS