How I built phone control for my desktop AI companion: 34 tools into one, and a pairing code that never touches argv

Ankita AI is a desktop AI assistant by Krish Verma — a little companion that lives on your desktop and gets things done while you're away. Last week it learned to drive your phone. The problem is mundane and annoying: your phone is in your hand, your assistant is on your desktop, and bridging the two usually means the assistant handing you adb commands to paste into a terminal. That is the assistant giving up. So I built one curated phone tool over mobile-next/mobile-mcp, pinned at version 1.0.16, and the rule is simple: you read a six-digit code off your phone screen, and Ankita does everything else.

34 tools is a good MCP surface and a poor assistant surface

mobile-mcp exposes about 34 fine-grained tools, and every one demands an explicit device id and raw coordinates. Fine for a protocol, terrible for a model: every call repeats state the host already knows, and the model has 34 schemas to choose from on every turn. So the phone family is one tool with named actions — devices, see, tap, type, screenshot, install, batch, raw — that remembers the selected device. The full server stays behind it as an explicit raw escape hatch.

The constants in src/integrations/phone-control.mjs are the design in miniature. The bridge is pinned (@mobilenext/mobile-mcp@1.0.16) because an unpinned @latest would silently change the tool ceiling. Telemetry is off (MOBILEMCP_DISABLE_TELEMETRY: 1) — phone contents are personal and nothing leaves the machine. The cloud-device tools are refused outright, so a call can never reserve or bill a remote device. The bridge is hidden from the MCP server list and never starts on its own; only a phone action starts it.

The numbers that keep it cheap

A phone is a richer context sink than a desktop window, so the caps are explicit. Screenshots are capped at 3 per turn, mirroring the browser cap, each at most 15 MB at 1024 px on the longest edge. Log reads default to 60 entries and cap at 200 — the server's own cap is 10,000, and nobody needs that in a prompt. A batch holds at most 50 steps and 32 KB of serialized steps; a raw call's arguments cap at 8 KB. Timeouts are 60 seconds per request and 180 seconds for first run, because the first call downloads the bridge.

Reading the screen works on refs, not coordinates. see returns elements as @e5-style refs, and a tap takes a fresh ref or explicit x/y from the latest screenshot — never both, never neither, and never a guessed coordinate. The skill enforces the order: connect, then read, then act, re-reading refs instead of replaying stale taps.

Wi-Fi pairing without typing a port number

Wireless debugging is where assistants usually hand the user a command. Ankita finds the phone itself with adb mdns, pairs with the six-digit code you read off the screen, and connects — you never type an adb command or copy a port number. The code goes to adb over stdin, so it never appears in the process list, and it is hidden from the approval preview. If the phone drops off after it sleeps or the router changes, the next action reconnects from the remembered address, and the skill tells the assistant to rediscover a changed Wi-Fi port instead of reusing a stale one.

Ask once to connect, then only for the irreversible

The approval model is the decision this feature turns on. First use asks once and shows the exact command that will run — the only time third-party code starts. After that, reading the screen, tapping, typing, and launching apps run without a prompt. Four actions always ask, because they run new code or end/erase state: install, uninstall, clear_data, and stop_recording.

That "always" had a hole. The first version checked the top-level action name only, so install inside a batch or a raw call reached the phone without a prompt. The fix normalizes every step — case- and whitespace-insensitively, so Install or install can't slip past — checks each batch step against the same approval list, rejects raw nested batches outright, and requires curated paths for screenshot, log, and recording output. Screenshot receipts only accept captures this tool generated: the filename must match phone-<17 digits>-<id>.png inside downloaded-images/, the path must resolve inside the real artifact folder after symlink resolution, the byte count must match the receipt, and the file must start with a real PNG signature. A receipt is model-visible JSON; it is never trusted on its own.

What it doesn't do yet

This is on main in PR #79, not in a release yet — v2.7.5 is still the latest packaged version, so treat this as landed code, not a download. Android works on Windows; iOS needs a Mac with Xcode and a booted simulator. The first call is slow while the bridge downloads. The verification record is honest about its limits: 36/36 tests pass across the phone and wireless suites, and a live adb mdns scan correctly returned "no phone advertising" — but no real-phone acceptance run is recorded yet. Screen content is treated as untrusted data and can't authorize an action by itself, and logins, passwords, and payment screens stay yours to handle on the phone.

The project is open source at github.com/akyourowngames/A.N.K.I.T.A, and the site is ankita-sable.vercel.app.

Tell me I'm wrong

The line I drew is "ask once to connect, then trust taps" — but a tap inside an app can delete a photo, send a message, or confirm a purchase. A tap is free; uninstall always asks. Is that the right boundary, or is it a category error — should the approval question be about what the tap does, not which tool name carries it? And the receipt rule refuses any PNG the tool didn't generate, which keeps a poisoned path out but also means you can't hand Ankita a screenshot you took yourself. Where exactly would you draw it?

— Krish Verma, creator of Ankita AI