hoodarunner/offline-agent

A tool-calling persona layer over qwen3:4b, tuned for a hand-written ReAct loop that runs with no API key, no cloud, and no network.

ollama pull hoodarunner/offline-agent

What this is

qwen3:4b with a system prompt and sampling parameters shaped for one job: choosing the right tool and actually calling it. The weights are unmodified — this is not a fine-tune. The value is in the prompt and the decoding settings, which are the two things that decide whether a 4B model chains tool calls or fakes them.

What this is not

It does not contain tools. A Modelfile can’t hold Python. This model expects six tools to exist — arithmetic, note read/write/list, current time, word count — and will ask to call them. Something has to be listening.

That something is the companion repo, which supplies the tools and the loop: https://github.com/yashhooda1/slm-offline-ai-agent

Pull this model, clone that repo, set OLLAMA_MODEL=hoodarunner/offline-agent, and you have a working agent. Pull this model alone and you have a model with opinions and no hands.

Why the prompt is shaped this way

Small models fail at tool use in a specific, repeatable way: they claim they did the thing. Ask a stock 4B model to read a file and count its words, and it will happily call read_note, look at the result, and then announce a word count it invented — without ever touching word_count.

So the system prompt states the failure mode as a rule:

A tool result answers ONLY the question that tool was called for. Never reuse one tool’s output as the answer to a different question.

Plus the two that matter most: never claim a result a tool didn’t confirm, and if steps remain after an observation, call the next tool immediately.

temperature 0.1 is doing quiet work here too. Tool calling is structured output, not creative writing — sampling entropy shows up as malformed arguments and skipped calls, not as interesting prose.

Honest limits

• It still skips tools sometimes. 4B is the floor for reliable multi-step tool use, and the floor is not the same as solid ground. If a task must touch a specific tool, enforce that in your loop, not in the prompt.
• Reasoning mode is on (qwen3 default). Those thinking tokens are regenerated every step of an agent loop. Pass think=False if you’d rather have the speed; benchmark it on your own tasks before deciding.
• Nothing here is safe by itself. The tools decide that. In the companion repo the calculator parses to an AST instead of calling eval(), and file tools are locked to a sandbox directory — because the model picks those arguments, and model output is untrusted input. If you wire this to your own tools, assume the same.

License

Inherits Apache 2.0 from qwen3:4b.