Qwen2.5-Coder-1.5B — Ollama Model (FP16, 8 GB GPU)

Model: oamazonasgabriel/qwen2.5-coder.1.5b-mlx:fp16-8gbGPU

---

DESCRIPTION

A code-specialized variant of Qwen2.5-Coder-1.5B optimized for MLX conversion workflows, using F16 (FP16) quantization for maximum quality. Designed to run on hardware with 8 GB GPU memory (e.g., RTX ³⁰⁷⁰⁄₄₀₆₀, Apple Silicon with 8 GB unified memory).

This model excels at code generation, code completion, bug fixing, and general programming tasks while remaining lightweight enough to fit comfortably on consumer-grade GPUs.

Key Features

• Code-First Training: Fine-tuned on an extensive corpus of code and code-related text
• Fast Generation: ~40-80 tok/s on compatible GPU hardware
• 32K Context Window: Sufficient for most code files and repositories
• Insert Mode: Supports fill-in-the-middle (FIM) for code infilling
• Lightweight: 3.1 GB — fits on virtually any modern GPU

Architecture

Property	Value
Architecture	Qwen2 (Decoder-only Transformer)
Parameters	1.5B
Layers	28
Attention Heads	12 (query), 2 (KV)
Embedding Size	1,536
Native Context	32,768 tokens
Modalities	Text only
Quantization	F16 (Float16)
Model Size	3.1 GB
License	Apache 2.0
Upstream	Qwen/Qwen2.5-Coder-1.5B-Instruct
GGUF Source	bartowski/Qwen_Qwen2.5-Coder-1.5B-Instruct-GGUF

---

REQUIREMENTS

Resource	Minimum	Recommended
GPU Memory	4 GB VRAM	8 GB+ VRAM
System RAM	8 GB	16 GB
Disk Space	4 GB free	8 GB+ free
Ollama Version	0.30.0+	Latest

Platform support: - NVIDIA GPU: Any GPU with 4 GB+ VRAM (GTX 1660, RTX 3060+, etc.) - Apple Silicon: Any Mac (M1+, 8 GB unified memory is plenty) - AMD GPU: ROCm-compatible with 4 GB+ VRAM - CPU-only: Works, but ~10-15 tok/s expected

---

QUICK START

1. Install Ollama

# macOS (Homebrew)
brew install ollama

# Linux
curl -fsSL https://ollama.com/install.sh | sh

# Windows
# Download from https://ollama.com/download

2. Pull & Run

# Pull the model (downloads ~3.1 GB)
ollama pull oamazonasgabriel/qwen2.5-coder.1.5b-mlx:fp16-8gbGPU

# Run interactively
ollama run oamazonasgabriel/qwen2.5-coder.1.5b-mlx:fp16-8gbGPU

# Single prompt
ollama run oamazonasgabriel/qwen2.5-coder.1.5b-mlx:fp16-8gbGPU \
  "Write a Python function to merge two sorted lists"

---

USAGE

CLI

# Interactive chat
ollama run oamazonasgabriel/qwen2.5-coder.1.5b-mlx:fp16-8gbGPU

# With system prompt
ollama run oamazonasgabriel/qwen2.5-coder.1.5b-mlx:fp16-8gbGPU \
  --system "You are an expert Rust developer. Provide concise, idiomatic solutions."

REST API

# Chat completion
curl -s http://127.0.0.1:11434/api/chat \
  -d '{
    "model": "oamazonasgabriel/qwen2.5-coder.1.5b-mlx:fp16-8gbGPU",
    "messages": [{"role": "user", "content": "Write a Go HTTP server"}]
  }'

# Generate (completion)
curl -s http://127.0.0.1:11434/api/generate \
  -d '{
    "model": "oamazonasgabriel/qwen2.5-coder.1.5b-mlx:fp16-8gbGPU",
    "prompt": "def quicksort(arr):",
    "stream": false
  }'

# OpenAI-compatible endpoint
curl -s http://127.0.0.1:11434/v1/chat/completions \
  -d '{
    "model": "oamazonasgabriel/qwen2.5-coder.1.5b-mlx:fp16-8gbGPU",
    "messages": [{"role": "user", "content": "Explain monads in Haskell"}],
    "stream": false
  }'

Python (ollama library)

pip install ollama

import ollama

# Chat
response = ollama.chat(
    model='oamazonasgabriel/qwen2.5-coder.1.5b-mlx:fp16-8gbGPU',
    messages=[{'role': 'user', 'content': 'Write a JavaScript debounce function'}],
)
print(response.message.content)

# Generate (code completion)
response = ollama.generate(
    model='oamazonasgabriel/qwen2.5-coder.1.5b-mlx:fp16-8gbGPU',
    prompt='function fibonacci(n) {',
)
print(response.response)

JavaScript (ollama.js)

npm install ollama

import ollama from 'ollama'

const response = await ollama.chat({
  model: 'oamazonasgabriel/qwen2.5-coder.1.5b-mlx:fp16-8gbGPU',
  messages: [{ role: 'user', content: 'Write a React custom hook' }],
})
console.log(response.message.content)

---

CODE COMPLETION (FIM)

This model supports fill-in-the-middle (FIM) / code infilling via the <|fim_prefix|>, <|fim_suffix|>, and <|fim_middle|> special tokens:

curl -s http://127.0.0.1:11434/api/generate \
  -d '{
    "model": "oamazonasgabriel/qwen2.5-coder.1.5b-mlx:fp16-8gbGPU",
    "prompt": "<|fim_prefix|>def hello():\n    <|fim_suffix|>\n    return greeting<|fim_middle|>",
    "options": {
      "stop": ["<|endoftext|>", "<|im_end|>"]
    }
  }'

---

MODEL DETAILS

Sampling Parameters

These are baked into the model via its Modelfile:

Parameter	Value	Rationale
num_ctx	32768	Full native context window
temperature	0.7	Balanced creativity/determinism
top_p	0.8	Nucleus sampling
top_k	20	Focused token selection
repeat_penalty	1.1	Light repetition deterrence

---

MEMORY & PERFORMANCE

VRAM Usage

Component	Size
Model weights (F16)	~3.1 GB
KV cache (16K context)	~0.5-1 GB
Ollama overhead	~0.2 GB
Total	~3.8-4.3 GB ✅

Context Window Scaling

Context	Fits 8 GB?	Notes
8,192	✅ Easy	~3.8 GB total
32,768	✅ Yes	~4.3 GB total
65,536	⚠️ Depends	May spill to CPU

Performance

• Prompt processing: ~100-200 tok/s (on modern GPU)
• Text generation: ~40-80 tok/s (NVIDIA RTX 4060+)
• CPU-only: ~10-15 tok/s
• Model load time: ~5-10 seconds

---

Opencode Integration

Add to ~/.config/opencode/opencode.jsonc:

"oamazonasgabriel/qwen2.5-coder.1.5b-mlx:fp16-8gbGPU": {
  "name": "oamazonasgabriel/qwen2.5-coder.1.5b-mlx:fp16-8gbGPU",
  "options": {
    "supportsThinking": false,
    "contextWindow": 32768
  }
}

Use as agent:

"model": "ollama/oamazonasgabriel/qwen2.5-coder.1.5b-mlx:fp16-8gbGPU"

---

LINKS

Resource	URL
This model on Ollama	https://ollama.com/oamazonasgabriel/qwen2.5-coder.1.5b-mlx
Upstream model (HuggingFace)	https://huggingface.co/Qwen/Qwen2.5-Coder-1.5B-Instruct
GGUF source (HuggingFace)	https://huggingface.co/bartowski/Qwen_Qwen2.5-Coder-1.5B-Instruct-GGUF
Ollama documentation	https://docs.ollama.com