Qwen3.5-4B — Ollama Model (8 GB) FIM-Optimized

Model: oamazonasgabriel/qwen3.5-4b:q3_k_s-8gbGPU

DESCRIPTION

A balanced configuration variant of Qwen3.5-4B, planned for FIM (Fill-In-the-Middle) and Tool Calling in restrict capacity environments. Using the Q3_K_S GGUF quantization from HuggingFace (unsloth). At only ~2 GB, it fits com fortably on any 8 GB single GPU with headroom for 8K context — perfect for real-time code completions, inline suggestions, and lightweight agentic coding tasks.

The ideal lightweight coding companion for: RTX 4060 · RTX 5060 · RTX 3060 · GTX 1660 · Intel Arc · Apple Silicon M-series · any GPU with 4 GB+ VRAM

Key Features

• FIM-optimized: temperature 0.5, top_p 0.8, repeat_penalty 1.15 — tuned for precise fill-in-the-middle code completions
• Efficient size: Q3_K_S quantization (~2 GB) — excellent quality-to-size ratio for the 4B parameter model
• Full capability: 4B dense model (all parameters active per token) — no MoE routing, maximum quality per response
• Blazing fast: 100-300+ tok/s on modern GPUs
• 8K context: ideal for function bodies, class definitions, and single-file completions
• FIM tokens: native <|fim_prefix|>, <|fim_middle|>, <|fim_suffix|> support
• ChatML format: compatible with standard instruct-tuning pipelines
• Thinking mode: toggleable via enable_thinking API parameter
• Tool calling: native support via qwen3_coder parser
• Apache 2.0 license: free for commercial and personal use

Architecture

REQUIREMENTS

Platform support: - Any GPU (NVIDIA, AMD, Intel Arc, Apple Silicon) with 2 GB+ VRAM - CPU-only — runs acceptably on modern CPUs - Integrated GPU — Intel UHD, AMD Radeon Graphics (iGPU) all work

💡 Why 8 GB GPU? The model is only ~2 GB at Q3_K_S. An 8 GB card has 6 GB of headroom — you can run this alongside your browser, IDE, and other GPU workloads w ithout breaking a sweat.

QUICK START

1. Install Ollama

# macOS (Homebrew)
brew install ollama

# Linux
curl -fsSL https://ollama.com/install.sh | sh

# Windows
# Download from https://ollama.com/download

2. Pull & Run

# Pull the model (downloads ~2 GB)
ollama pull oamazonasgabriel/qwen3.5-4b:q3_k_s-8gbGPU

# Run interactively
ollama run oamazonasgabriel/qwen3.5-4b:q3_k_s-8gbGPU

# Single code prompt
ollama run oamazonasgabriel/qwen3.5-4b:q3_k_s-8gbGPU \
  "Write a Python function that merges two sorted lists"

USAGE

CLI

# Interactive chat
ollama run oamazonasgabriel/qwen3.5-4b:q3_k_s-8gbGPU

# With system prompt
ollama run oamazonasgabriel/qwen3.5-4b:q3_k_s-8gbGPU \
  --system "You are a helpful coding assistant. Provide concise code examples."

# FIM-style prompt (inline)
ollama run oamazonasgabriel/qwen3.5-4b:q3_k_s-8gbGPU \
  "Complete this function:\ndef fibonacci(n):\n    "

REST API

# Chat completion
curl -s http://127.0.0.1:11434/api/chat \
  -d '{
    "model": "oamazonasgabriel/qwen3.5-4b:q3_k_s-8gbGPU",
    "messages": [{"role": "user", "content": "Write a binary search in Python"}]
  }'

# Generate (completion)
curl -s http://127.0.0.1:11434/api/generate \
  -d '{
    "model": "oamazonasgabriel/qwen3.5-4b:q3_k_s-8gbGPU",
    "prompt": "Explain what FIM means in code completion",
    "stream": false
  }'

# FIM Completion (Fill-in-the-Middle)
curl -s http://127.0.0.1:11434/api/generate \
  -d '{
    "model": "oamazonasgabriel/qwen3.5-4b:q3_k_s-8gbGPU",
    "prompt": "<|fim_prefix|>def hello():\n    <|fim_suffix|>\n    return False<|fim_middle|>",
    "stream": false,
    "options": {
      "temperature": 0.5,
      "top_p": 0.8,
      "repeat_penalty": 1.15
    }
  }'

# With thinking mode
curl -s http://127.0.0.1:11434/api/chat \
  -d '{
    "model": "oamazonasgabriel/qwen3.5-4b:q3_k_s-8gbGPU",
    "messages": [{"role": "user", "content": "Solve this leetcode problem: ..."}],
    "options": {"enable_thinking": true}
  }'

# OpenAI-compatible endpoint
curl -s http://127.0.0.1:11434/v1/chat/completions \
  -d '{
    "model": "oamazonasgabriel/qwen3.5-4b:q3_k_s-8gbGPU",
    "messages": [{"role": "user", "content": "Write a React component"}],
    "stream": false
  }'

FIM Token Reference

Python (ollama library)

pip install ollama

import ollama

# Chat
response = ollama.chat(
    model='oamazonasgabriel/qwen3.5-4b:q3_k_s-8gbGPU',
    messages=[{'role': 'user', 'content': 'Write a JavaScript debounce function'}],
)
print(response.message.content)

# FIM Completion
response = ollama.generate(
    model='oamazonasgabriel/qwen3.5-4b:q3_k_s-8gbGPU',
    prompt='<|fim_prefix|>def hello():\n    <|fim_suffix|>\n    return False<|fim_middle|>',
    options={
        'temperature': 0.5,
        'top_p': 0.8,
        'repeat_penalty': 1.15,
    },
)
print(response.response)

# With thinking mode
response = ollama.chat(
    model='oamazonasgabriel/qwen3.5-4b:q3_k_s-8gbGPU',
    messages=[{'role': 'user', 'content': 'Debug this SQL query'}],
    options={'enable_thinking': True},
)

JavaScript (ollama.js)

npm install ollama

import ollama from 'ollama'

// Chat
const response = await ollama.chat({
  model: 'oamazonasgabriel/qwen3.5-4b:q3_k_s-8gbGPU',
  messages: [{ role: 'user', content: 'Write a CSS animation' }],
})
console.log(response.message.content)

SAMPLING PARAMETERS

These are baked into the model via its Modelfile:

Equivalent llama.cpp Command

./llama-cli \
  -m Qwen3.5-4B-Q3_K_S.gguf \
  -ngl 99 \
  -c 8192 \
  --temp 0.5 \
  --top-p 0.8 \
  --repeat-penalty 1.15 \
  -cnv

MEMORY & PERFORMANCE

VRAM Budget

Context Window Scaling

On any GPU with 4 GB+ VRAM:

Performance

Opencode Integration

Add to ~/.config/opencode/opencode.jsonc:

"oamazonasgabriel/qwen3.5-4b:q3_k_s-8gbGPU": {
  "name": "oamazonasgabriel/qwen3.5-4b:q3_k_s-8gbGPU",
  "options": {
    "supportsThinking": false,
    "contextWindow": 8192
  }
}

Use as agent:

"model": "ollama/oamazonasgabriel/qwen3.5-4b:q3_k_s-8gbGPU"

Or launch directly:

ollama launch opencode --model oamazonasgabriel/qwen3.5-4b:q3_k_s-8gbGPU

TROUBLESHOOTING

ALTERNATIVE MODELS

# Larger coding model (needs 16 GB)
ollama pull oamazonasgabriel/qwen3.5-9b:q4-16gbGPU

# Larger MoE model (needs 24 GB)
ollama pull oamazonasgabriel/qwen3.6-35b-a3b:q4-24gbGPU

# Even smaller FIM model (needs 2 GB)
ollama pull oamazonasgabriel/qwen2.5-coder-0.5b:fp16-8gbGPU

CREDITS

LINKS