A compact Qwen3.5-0.8B reasoning model improved through Weight-Space Programming. Features targeted weight modifications designed to reduce repetitive reasoning loops and improve mathematical reasoning.

Details

Updated yesterday

yesterday

90bcef69c496 · 737MB

model

archqwen35

parameters752M

quantizationQ4_K_M

529MB

projector

archclip

parameters101M

quantizationBF16

207MB

params

{ "num_ctx": 4096, "num_predict": 4096, "presence_penalty": 1.5, "temperature": 0.6,

101B

Qwen3.5-0.8B Weight-Space Programming

A compact reasoning model improved through Weight-Space Programming — modifying model weights directly without conventional fine-tuning or gradient descent.

This Ollama release packages the Qwen3.5-0.8B Weight-Space Programming checkpoint in a format ready for local inference.

Overview

Property	Details
Model family	Qwen3.5
Parameters	0.8B
Architecture	Qwen3.5
Method	Weight-Space Programming
Quantization	Q4_K_M (original GGUF source)
Model format	GGUF
License	Apache 2.0
Context window	Up to 256K (architecture/configuration)
Intended use	Local reasoning, mathematics, lightweight assistant tasks

What is Weight-Space Programming?

Unlike conventional fine-tuning, Weight-Space Programming modifies selected model weights directly to change model behaviour.

In this experiment, a frontier LLM identified a repetitive reasoning-loop pattern in the original Qwen3.5-0.8B model and modified a small subset of its weights.

Only 131,072 weights were modified.
Approximately 0.016% of the model’s weights.
Modifications were made to MLP down_proj tensors in layers 15 and 19.
No gradient descent or conventional training was used.

The objective was to reduce repetitive reasoning loops and improve mathematical problem-solving.

Reported Benchmarks

Results reported by the original research:

Benchmark	Base Qwen3.5-0.8B	Weight-Space Version
MATH-500	7.9%	48.1%
GSM8K	13.8%	67.1%
IFEval	21.0%	62.3%
Token limit reached	93.0%	33.1%

An independent reproduction reported 51.0% on MATH-500.

Benchmark disclaimer: These are upstream research results, not independently verified benchmarks for this Ollama distribution. The original evaluation used vLLM, thinking enabled, and specific sampling settings.

Installation

Install Ollama:

https://ollama.com/download

Pull the model:

ollama pull neelcontractor/qwen-weight-space

Run it:

ollama run neelcontractor/qwen-weight-space

API Usage

The model can also be used through Ollama’s local API.

curl http://localhost:11434/api/chat \
  -d '{
    "model": "neelcontractor/qwen-weight-space",
    "messages": [
      {
        "role": "user",
        "content": "Explain how binary search works."
      }
    ],
    "stream": false
  }'

Recommended Use Cases

Lightweight local reasoning
Mathematical problem-solving experiments
Instruction following
Small local AI assistant workflows
Experimenting with weight-space modifications

Limitations

A 0.8B model is still a small language model and may struggle with complex reasoning.
Benchmark improvements do not guarantee improvements across every task.
Ollama inference results may differ from the original vLLM evaluations.
Long context windows require additional memory.
Performance depends on hardware, quantization and inference configuration.

Credits and Attribution

This model is based on research and model weights published by:

Original weight-space experiment: RikiyaT
Reconstructed checkpoint: ML-Intern-lab
GGUF conversion: prithivMLmods
Base model: Qwen3.5-0.8B by Qwen

Research:

https://rikiyat.github.io/blog/weight-space-programming/

Original checkpoint:

https://huggingface.co/RikiyaT/Qwen3.5-0.8B-weight-space-programming

GGUF conversion:

https://huggingface.co/prithivMLmods/Qwen3.5-0.8B-weight-space-programming-GGUF

Reproduction code and weight diffs:

https://github.com/RikiyaT/lm_understand-logs

License

Apache 2.0. Refer to the original model and distribution repositories for licensing details.

Published on Ollama by neelcontractor.