Qwen 3.6 35B Text — 16 GB VRAM, IQ3_S

A compact text-only build of Qwen 3.6 35B, prepared for local use with Ollama.

This model is designed to fit within 16 GB of VRAM, including a very large context window of up to 262,144 tokens, when used with:

• IQ3_S model quantization
• Q4_0 KV-cache quantization
• Flash Attention

Model

byczech/qwen3.6-16G-UD-IQ3_S:35b

Highlights

• Text-only Qwen 3.6 35B model
• IQ3_S quantization
• Up to 262,144 tokens of context
• Optimized for GPUs with 16 GB VRAM
• Suitable for long documents, coding, analysis, summarization and general chat
• Lower memory overhead than the multimodal variant

Run

ollama run byczech/qwen3.6-16G-UD-IQ3_S:35b

Recommended Ollama configuration

To use the full 262,144-token context while minimizing VRAM usage, start the Ollama server with:

OLLAMA_FLASH_ATTENTION=1 \
OLLAMA_KV_CACHE_TYPE=q4_0 \
OLLAMA_CONTEXT_LENGTH=262144 \
ollama serve

Then run the model in another terminal:

ollama run byczech/qwen3.6-16G-UD-IQ3_S:35b

You can verify whether the model is fully loaded on the GPU with:

ollama ps

The PROCESSOR column should report 100% GPU.

API example

curl http://localhost:11434/api/chat -d '{
  "model": "byczech/qwen3.6-16G-UD-IQ3_S:35b",
  "messages": [
    {
      "role": "user",
      "content": "Explain the main trade-offs of KV-cache quantization."
    }
  ],
  "options": {
    "num_ctx": 262144
  },
  "stream": false
}'

Context length

Ollama may select a much smaller default context on systems with less than 24 GB of VRAM. Set OLLAMA_CONTEXT_LENGTH=262144 before starting the server, or configure num_ctx explicitly in your client (or you can create your own Modelfile with num_ctx parameter).

The maximum context consumes substantially more memory than short prompts. The 16 GB target assumes Q4_0 KV cache, Flash Attention, one loaded model and no parallel requests.

Quality and memory trade-off

Q4_0 KV-cache quantization uses roughly one quarter of the memory required by the default F16 KV cache. This makes the very large context practical on a 16 GB GPU, but it can reduce precision, especially near the maximum context length.

For higher KV-cache precision, use q8_0, but expect higher VRAM usage and a smaller practical context window.

Notes

• Actual VRAM use depends on the Ollama version, GPU backend, prompt length and runtime overhead.
• Close other GPU-intensive applications when using the maximum context.
• Keep OLLAMA_NUM_PARALLEL=1 for the lowest memory usage.