Qwen3.5-2B — Ollama Model (BF16, 8 GB GPU)

Model: oamazonasgabriel/qwen3.5-2b-mlx:fp16-8gbGPU

---

DESCRIPTION

A multimodal vision-language model variant of Qwen3.5-2B optimized for MLX conversion workflows, using BF16 quantization for maximum quality. Despite its compact 2B parameter size, it offers tool calling, thinking mode, and an enormous 262K token context window.

Designed to run on hardware with 8 GB GPU memory (e.g., RTX ³⁰⁷⁰⁄₄₀₆₀, Apple Silicon with 8 GB unified memory). This is one of the smallest models with native thinking/reasoning capabilities and tool calling support.

⚠️ Note on Vision: This GGUF variant is text-only. The upstream model supports vision (image input), but Ollama 0.30.6 does not yet support the MMPROJ Modelfile directive needed for vision encoder integration. Use the official Qwen3.5-2B model for full multimodal support.

Key Features

• Thinking Mode: Native support for chain-of-thought reasoning
• Tool Calling: Structured function calling via Qwen35 parser
• Massive Context: 262K native context window (supports up to ~1M with YaRN)
• Fast Generation: ~30-70 tok/s on compatible GPU hardware
• Extremely Compact: Only 1.9B active parameters — punches far above its weight

Architecture

Property	Value
Architecture	Qwen35 (Decoder-only Transformer)
Parameters	1.9B
Layers	28
Embedding Size	2,048
Native Context	262,144 tokens
Extended Context	up to ~1,010,000 via YaRN
Modalities	Text (this variant); upstream supports Image
Quantization	BF16 (Brain Float 16)
Model Size	3.9 GB
License	Apache 2.0
Upstream	Qwen/Qwen3.5-2B
GGUF Source	bartowski/Qwen_Qwen3.5-2B-GGUF

---

REQUIREMENTS

Resource	Minimum	Recommended
GPU Memory	6 GB VRAM	8 GB+ VRAM
System RAM	8 GB	16 GB
Disk Space	5 GB free	10 GB+ free
Ollama Version	0.30.6+	Latest

Platform support: - NVIDIA GPU: GPU with 6 GB+ VRAM (RTX 3060+, RTX 4060+, etc.) - Apple Silicon: Mac with 8 GB+ unified memory (M1, M2, M3, M4) - AMD GPU: ROCm-compatible with 6 GB+ VRAM - CPU-only: Works, but slower (~8-12 tok/s)

---

QUICK START

1. Install Ollama

# macOS (Homebrew)
brew install ollama

# Linux
curl -fsSL https://ollama.com/install.sh | sh

# Windows
# Download from https://ollama.com/download

2. Pull & Run

# Pull the model (downloads ~3.9 GB)
ollama pull oamazonasgabriel/qwen3.5-2b-mlx:fp16-8gbGPU

# Run interactively
ollama run oamazonasgabriel/qwen3.5-2b-mlx:fp16-8gbGPU

# Single prompt
ollama run oamazonasgabriel/qwen3.5-2b-mlx:fp16-8gbGPU \
  "What is 27 * 45? Think step by step."

---

USAGE

CLI

# Interactive chat
ollama run oamazonasgabriel/qwen3.5-2b-mlx:fp16-8gbGPU

# With thinking mode (default on for this model)
ollama run oamazonasgabriel/qwen3.5-2b-mlx:fp16-8gbGPU \
  "Explain the theory of relativity"

REST API

# Chat completion
curl -s http://127.0.0.1:11434/api/chat \
  -d '{
    "model": "oamazonasgabriel/qwen3.5-2b-mlx:fp16-8gbGPU",
    "messages": [{"role": "user", "content": "What is the capital of France?"}]
  }'

# Generate
curl -s http://127.0.0.1:11434/api/generate \
  -d '{
    "model": "oamazonasgabriel/qwen3.5-2b-mlx:fp16-8gbGPU",
    "prompt": "Write a short poem about artificial intelligence",
    "stream": false
  }'

# OpenAI-compatible endpoint
curl -s http://127.0.0.1:11434/v1/chat/completions \
  -d '{
    "model": "oamazonasgabriel/qwen3.5-2b-mlx:fp16-8gbGPU",
    "messages": [{"role": "user", "content": "Hello!"}],
    "stream": false
  }'

Python (ollama library)

pip install ollama

import ollama

# Chat with thinking
response = ollama.chat(
    model='oamazonasgabriel/qwen3.5-2b-mlx:fp16-8gbGPU',
    messages=[{'role': 'user', 'content': 'Solve: 7x + 3 = 24'}],
)
print(response.message.content)

JavaScript (ollama.js)

npm install ollama

import ollama from 'ollama'

const response = await ollama.chat({
  model: 'oamazonasgabriel/qwen3.5-2b-mlx:fp16-8gbGPU',
  messages: [{ role: 'user', content: 'Explain quantum entanglement' }],
})
console.log(response.message.content)

---

TOOL CALLING

This model supports structured tool/function calling. Example:

curl -s http://127.0.0.1:11434/api/chat \
  -d '{
    "model": "oamazonasgabriel/qwen3.5-2b-mlx:fp16-8gbGPU",
    "messages": [{"role": "user", "content": "What'\''s the weather in Tokyo?"}],
    "tools": [{
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Get the current weather",
        "parameters": {
          "type": "object",
          "properties": {
            "location": {"type": "string", "description": "City name"}
          },
          "required": ["location"]
        }
      }
    }]
  }'

---

THINKING MODE

Qwen3.5-2B supports a thinking/reasoning mode where the model internally reasons before producing its final answer. This is enabled by default and produces more accurate responses for math, logic, and complex reasoning tasks.

---

MODEL DETAILS

Sampling Parameters

These are baked into the model via its Modelfile:

Parameter	Value	Rationale
num_ctx	8192	Default for 8 GB GPU balance
temperature	0.7	Balanced creativity/determinism
top_p	0.8	Nucleus sampling
top_k	20	Focused token selection
repeat_penalty	1.1	Light repetition deterrence

💡 You can increase num_ctx up to 262K if you have sufficient memory or are running on CPU with enough RAM.

---

MEMORY & PERFORMANCE

VRAM Usage

Component	Size
Model weights (BF16)	~3.9 GB
KV cache (8K context)	~0.3 GB
Ollama overhead	~0.2 GB
Total	~4.4 GB ✅

Context Window Scaling

Context	Fits 8 GB?	Notes
8,192	✅ Plenty of headroom	~4.4 GB total
32,768	✅ Yes	~5.0 GB total
131,072	⚠️ Tight	~7.5 GB total
262,144 (native)	❌ May spill	Needs ~10 GB+

Performance

• Prompt processing: ~80-150 tok/s (on modern GPU)
• Text generation: ~30-70 tok/s (NVIDIA RTX 4060+)
• CPU-only: ~8-12 tok/s
• Model load time: ~8-15 seconds

---

Opencode Integration

Add to ~/.config/opencode/opencode.jsonc:

"oamazonasgabriel/qwen3.5-2b-mlx:fp16-8gbGPU": {
  "name": "oamazonasgabriel/qwen3.5-2b-mlx:fp16-8gbGPU",
  "options": {
    "supportsThinking": true,
    "contextWindow": 8192
  }
}

Use as agent:

"model": "ollama/oamazonasgabriel/qwen3.5-2b-mlx:fp16-8gbGPU"

---

LINKS

Resource	URL
This model on Ollama	https://ollama.com/oamazonasgabriel/qwen3.5-2b-mlx
Upstream model (HuggingFace)	https://huggingface.co/Qwen/Qwen3.5-2B
GGUF source (HuggingFace)	https://huggingface.co/bartowski/Qwen_Qwen3.5-2B-GGUF
Ollama documentation	https://docs.ollama.com