Andy

Andy picks the next tap, scroll or app to open on an Android screen, small enough to run on the phone itself.

Give Andy a goal, the screen’s accessibility tree and a list of allowed actions. It replies with the single best next action as JSON. Andy is a 2B fine-tune of Qwen3.5-2B, distilled on AndroidControl, and at 1.3 GB it beats models twice its size on choosing the right next step.

Models

Same 2,003 AndroidControl test steps: Qwen3.5-2B 33.5%, Gemma 4 E2B 44.4%, Qwen3.5-4B 50.1%.

Run

ollama run atharvavdeo/andy --think=false

Always run Andy with thinking turned off. That is how it was trained and evaluated, and with thinking off Ollama reproduces the evaluation prompt and answers exactly.

import ollama

screen = """Goal: Watch the video on how to make molten chocolate cake in the Kitchen Stories app.
Current app: com.ajnsnewmedia.kitchenstories
Previous actions:
1. opened app Kitchen Stories
Screen: change since the last action unknown · 6 elements
Screen elements (@x,y = centre, % of screen width,height):
[1] FrameLayout #content @50,50
  [2] View "Our Latest Recipes" @43,75
  [3] Button "See all" @89,75 (click)
  [4] ImageView "Today, Tab 1 of 3" @16,94 (click,selected)
  [5] ImageView "Search, Tab 2 of 3" @50,94 (click)
  [6] ImageView "Profile , Tab 3 of 3" @83,94 (click)
Allowed actions:
{"op":"tap","id":3}
{"op":"tap","id":4}
{"op":"tap","id":5}
{"op":"tap","id":6}
{"op":"scroll","dir":"up"}
{"op":"scroll","dir":"down"}
{"op":"back"}
{"op":"home"}
{"op":"wait"}
{"op":"done"}
{"op":"handoff"}"""

r = ollama.chat(model="atharvavdeo/andy", messages=[{"role": "user", "content": screen}], think=False)
print(r.message.content)   # {"op":"tap","id":5}

Andy answers with one action from the allowed list. Check the reply against the list before executing it.

Prompt format

Keep the layout above: goal, current app, previous actions, the screen line, the element tree with positions, then the allowed actions. Use andy:v2 with this layout. andy:v1 expects the same layout without the screen line and the positions.

Confidence and hand-off

Andy is calibrated, so an agent can act on confident steps and hand the rest to the user. At its default threshold v2 hands off 22% of steps and is 75% accurate on the rest. This needs a score for every allowed action, which the Ollama chat API does not provide. Use the GGUF files with llama.cpp for it.

Limitations

• Andy reads text only, so unlabelled icon buttons are hard for it.
• It decides one step at a time. Step accuracy is not the same as finishing a task.
• It was trained and tested on AndroidControl, in English.
• Andy only chooses actions. The app that executes them must enforce its own safety rules.

License

• Andy: Apache-2.0
• Base model Qwen3.5-2B: Apache-2.0
• Training data: AndroidControl by Google

Andy on Hugging Face