Gemma 4 31B Vision — 16 GB VRAM, Q3_K_S

A compact multimodal build of Gemma 4 31B Instruct with vision support, prepared for local use with Ollama.

This model is optimized to fit within 16 GB of VRAM using Q3_K_S quantization. In tested configurations, the practical context window that still fits within 16 GB VRAM is approximately 36,000–40,000 tokens.

Model

byczech/gemma4-it-mmproj-16G-Q3_K_S:31b

Highlights

• Gemma 4 31B Instruct
• Text and image understanding
• Multimodal projector included
• Q3_K_S quantization
• Optimized for GPUs with 16 GB VRAM
• Practical context window of approximately 36K–40K tokens
• Suitable for image analysis, screenshots, documents, coding, summarization and general chat

Run

ollama run byczech/gemma4-it-mmproj-16G-Q3_K_S:31b

Recommended Ollama configuration

Flash Attention and a quantized KV cache can reduce memory usage and allow a larger practical context window:

OLLAMA_FLASH_ATTENTION=1 \
OLLAMA_KV_CACHE_TYPE=q4_0 \
OLLAMA_CONTEXT_LENGTH=36864 \
ollama serve

Then run the model in another terminal:

ollama run byczech/gemma4-it-mmproj-16G-Q3_K_S:31b

You can verify whether the model is fully loaded on the GPU with:

ollama ps

The PROCESSOR column should ideally report 100% GPU.

Vision example with Python

Install the Ollama Python package:

pip install ollama

Then pass an image path to the model:

from ollama import chat

response = chat(
    model="byczech/gemma4-it-mmproj-16G-Q3_K_S:31b",
    messages=[
        {
            "role": "user",
            "content": "Describe this image in detail.",
            "images": ["image.jpg"],
        }
    ],
)

print(response.message.content)

Context length

A context size of around 36,000–40,000 tokens can fit within 16 GB VRAM in suitable configurations. The exact maximum is not fixed and may vary depending on:

• Ollama version
• GPU architecture and driver
• Compute backend
• KV-cache type
• Number and resolution of input images
• Runtime overhead
• Other applications using GPU memory
• Parallel requests and loaded models

If the model does not fit fully on the GPU at 40K context, reduce the context size gradually, for example:

OLLAMA_CONTEXT_LENGTH=36000 ollama serve

You can also set the context explicitly per request with num_ctx.

Quality and memory trade-off

The model uses Q3_K_S quantization to make a 31B multimodal model practical on a 16 GB GPU. This significantly reduces memory requirements, but it may also reduce output quality compared with larger or less aggressively quantized variants.

Using a Q4_0 KV cache further reduces memory usage and helps preserve more VRAM for the context window. For higher KV-cache precision, use q8_0, but expect a smaller practical context window.

Notes

• The 36K–40K context range is a practical estimate, not a guaranteed limit.
• Image inputs also consume context and additional memory.
• Close other GPU-intensive applications when using the largest possible context.
• Keep OLLAMA_NUM_PARALLEL=1 for the lowest memory usage.
• Actual VRAM usage may change between Ollama releases.