Qwen3.8-27B — Ollama Model (32 GB)

Model: oamazonasgabriel/qwen3.8-27b:q4-km-32gbGPU

---

DESCRIPTION

A full-quality Qwen3.8-27B in Q4_K_M quantization, packaged for hardware with 32 GB+ VRAM. Unlike MoE models, this is a dense 27.8B-parameter model — every parameter is active on every token — paired with a hybrid attention architecture (linear + full attention) that delivers both long context and fast inference.

The ideal heavy-lift model for: dual RTX 5090 · RTX 4090 ×2 · RTX 5080 ×2 · Apple Silicon M2/M3/M4 Ultra · any setup with 32 GB+ unified memory.

🔗 llama.cpp configuration & GGUF files: Need the full 256K context or want to run on less VRAM (via --no-kv-offload)? Grab the ready-to-run llama.cpp config from the Hugging Face repo → impacte/Qwen3.8-27B-GGUF

Key Features

• Dense 27.8B model — all parameters active, no MoE routing, maximum quality per token
• Hybrid attention: 48 linear-attention layers + 16 full-attention layers (3:1 ratio) — efficient long-context without quadratic cost
• 256K native context (262,144 tokens) — 64K configured for VRAM budget
• Multimodal: Text + Image + Video (via mmproj projector)
• Thinking mode: Enabled by default — step-by-step reasoning in thinking blocks
• Tool calling: Native support via <tool_call><function=...> XML format
• Q4_K_M quantization: imatrix-calibrated from bartowski
• Apache 2.0 license: free for commercial and personal use

Architecture

Property	Value
Architecture	Qwen3.5 hybrid (linear attention + full attention, dense)
Total Parameters	27.78B (all active — not MoE)
Layers	64 (48 linear + 16 full attention)
Hidden Size	5,120
Attention Heads	24 (4 KV heads)
Native Context	262,144 tokens (256K)
Modalities	Text + Image + Video
Quantization	Q4_K_M imatrix-calibrated
Model Size	~17.8 GB (weights)
License	Apache 2.0
Upstream	Qwen/Qwen3.8-27B

---

REQUIREMENTS

Resource	Minimum	Recommended
GPU Memory	32 GB VRAM	48 GB+ VRAM
System RAM	32 GB	64 GB
Disk Space	20 GB free	50 GB+ free
NVIDIA Driver	525+	550+
Ollama Version	0.30.6+	Latest

Platform support: - NVIDIA GPU (dual): 2× 16 GB (RTX 5060 Ti ×2, RTX 5080 ×2, etc.) - NVIDIA GPU (single): 32 GB+ VRAM (RTX 5090, A6000, L40, etc.) - Apple Silicon: Mac with 32 GB+ unified memory (M2/M3/M4 Ultra) - AMD GPU: ROCm-compatible with 32 GB+ VRAM

💡 Why 32 GB? The model weights are ~17.8 GB at Q4_K_M. With a 64K context window and f16 KV cache, total VRAM usage is ~24-27 GB. 32 GB gives comfortable headroom; 48 GB unlocks the full 256K context.

---

QUICK START

1. Install Ollama

# macOS (Homebrew)
brew install ollama

# Linux
curl -fsSL https://ollama.com/install.sh | sh

# Windows
# Download from https://ollama.com/download

2. Pull & Run

# Pull the model (downloads ~18 GB)
ollama pull oamazonasgabriel/qwen3.8-27b:q4-km-32gbGPU

# Run interactively
ollama run oamazonasgabriel/qwen3.8-27b:q4-km-32gbGPU

# Single prompt
ollama run oamazonasgabriel/qwen3.8-27b:q4-km-32gbGPU \
  "Explain quantum computing in simple terms"

3. (Optional) Max Context

For the full 128K+ context on 48 GB VRAM, set this before starting Ollama:

export OLLAMA_KV_CACHE_TYPE=q4_0

# Linux permanent:
sudo systemctl edit ollama
# Add: Environment="OLLAMA_KV_CACHE_TYPE=q4_0"
# Then: sudo systemctl daemon-reload && sudo systemctl restart ollama

# macOS permanent:
launchctl setenv OLLAMA_KV_CACHE_TYPE q4_0
# Then restart Ollama from Applications

---

USAGE

CLI

# Interactive chat
ollama run oamazonasgabriel/qwen3.8-27b:q4-km-32gbGPU

# With system prompt
ollama run oamazonasgabriel/qwen3.8-27b:q4-km-32gbGPU \
  --system "You are a senior software architect. Be precise and thorough."

REST API

# Chat completion
curl -s http://127.0.0.1:11434/api/chat \
  -d '{
    "model": "oamazonasgabriel/qwen3.8-27b:q4-km-32gbGPU",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

# OpenAI-compatible endpoint
curl -s http://127.0.0.1:11434/v1/chat/completions \
  -d '{
    "model": "oamazonasgabriel/qwen3.8-27b:q4-km-32gbGPU",
    "messages": [{"role": "user", "content": "Hello!"}],
    "stream": false
  }'

Python (ollama library)

pip install ollama

import ollama

# Chat
response = ollama.chat(
    model='oamazonasgabriel/qwen3.8-27b:q4-km-32gbGPU',
    messages=[{'role': 'user', 'content': 'Write a Python decorator for caching'}],
)
print(response.message.content)

# With thinking mode (reasoning_content is returned separately)
response = ollama.chat(
    model='oamazonasgabriel/qwen3.8-27b:q4-km-32gbGPU',
    messages=[{'role': 'user', 'content': 'Debug this complex concurrency bug'}],
    options={'enable_thinking': True},
)

JavaScript (ollama.js)

npm install ollama

import ollama from 'ollama'

const response = await ollama.chat({
  model: 'oamazonasgabriel/qwen3.8-27b:q4-km-32gbGPU',
  messages: [{ role: 'user', content: 'Design a REST API schema' }],
})
console.log(response.message.content)

---

SAMPLING PARAMETERS

These are baked into the model via its Modelfile:

Parameter	Value	Rationale
num_ctx	65536	64K default — balances context vs VRAM
num_gpu	99	Offload all layers to GPU
temperature	0.7	Balanced creativity vs coherence
top_p	0.95	Standard nucleus sampling
top_k	40	Focused token selection
repeat_penalty	1.0	No repetition penalty
num_predict	8192	Max output tokens

---

MEMORY & PERFORMANCE

VRAM Budget

Component	Size
Model weights (Q4_K_M)	~17.8 GB
KV cache (f16, 64K context)	~5-7 GB
Ollama process overhead	~0.5-1 GB
Total (f16 + 64K)	~24-27 GB ✅
Total (q4_0 + 64K)	~21-23 GB ✅

Context Window Scaling

On 32 GB VRAM:

Context	KV Cache	Fits 32 GB?	Notes
8,192	~0.7-1 GB	✅ Lots of headroom	~19 GB total
32,768	~2.5-3.5 GB	✅ Plenty	~21-22 GB total
65,536	~5-7 GB	✅ Recommended	~24-27 GB total
131,072	~10-14 GB	⚠️ Tight / q4_0 only	~28-32 GB total
262,144 (native)	~20-28 GB	❌ Needs 48 GB+

Performance

• Prompt processing: ~60-120 tok/s (hybrid linear attention is fast on long prompts)
• Text generation: ~20-40 tok/s on 32 GB dual-GPU
• Model load time: ~60-90 seconds (~18 GB load)

The hybrid architecture (3:1 linear:full attention) means prompt processing stays near-linear even for very long contexts, unlike pure-attention models that degrade quadratically.

---

Opencode Integration

Add to ~/.config/opencode/opencode.jsonc:

"oamazonasgabriel/qwen3.8-27b:q4-km-32gbGPU": {
  "name": "oamazonasgabriel/qwen3.8-27b:q4-km-32gbGPU",
  "options": {
    "supportsThinking": true,
    "contextWindow": 65536
  }
}

Use as agent:

"model": "ollama/oamazonasgabriel/qwen3.8-27b:q4-km-32gbGPU"

Or launch directly:

ollama launch opencode --model oamazonasgabriel/qwen3.8-27b:q4-km-32gbGPU

---

TROUBLESHOOTING

Model fails to load — “unable to allocate buffer”

Cause: Not enough free VRAM. This is a 17.8 GB model and needs 32 GB+ VRAM. Fix: Free at least 24 GB before loading, or run via llama.cpp with --no-kv-offload (KV cache in system RAM).

CPU offloading detected

Run ollama ps — if CPU% appears, reduce num_ctx or set OLLAMA_KV_CACHE_TYPE=q4_0.

Slow generation

On 32 GB you should get 20-40 tok/s. If slower: - Check for CPU offloading (ollama ps) - Verify GPU is on CUDA (not Vulkan) - Close competing GPU processes

Vision not working

This Ollama tag is text-focused. For multimodal (image/video), use the llama.cpp setup with the mmproj projector — see the Hugging Face repo.

---

ALTERNATIVE MODELS

# Full 256K context via llama.cpp (KV cache in system RAM)
# See: https://huggingface.co/impacte/Qwen3.8-27B-GGUF

# Smaller dense Qwen (needs 16 GB)
ollama pull oamazonasgabriel/qwen3.5-9b:q4-16gbGPU

# MoE Qwen for 24 GB setups
ollama pull oamazonasgabriel/qwen3.6-35b-a3b:q4-24gbGPU

# Higher precision (needs ~48 GB VRAM)
ollama pull hf.co/bartowski/Qwen3.8-27B-GGUF:Q8_0

---

CREDITS

Role	Entity
Base Model	Qwen Team, Alibaba Group
Original Model	Qwen3.8-27B
GGUF Quantization	bartowski — Q4_K_M imatrix-calibrated
Ollama Packaging	impacte.tech
License	Apache 2.0

---

LINKS

Resource	URL
This model on Ollama	https://ollama.com/oamazonasgabriel/qwen3.8-27b
Upstream model (HuggingFace)	https://huggingface.co/Qwen/Qwen3.8-27B
GGUF + llama.cpp config	https://huggingface.co/impacte/Qwen3.8-27B-GGUF
bartowski quantized version	https://huggingface.co/bartowski/Qwen3.8-27B-GGUF
Ollama documentation	https://docs.ollama.com
Project repository	https://github.com/oamazonasgabriel/ollama-training
Built by	impacte.tech