agent-desktop lets AI agents operate desktop apps through accessibility trees, not pixels
agent-desktop, from Vercel Labs, is a command-line tool and C library for macOS, Windows and Linux. A snapshot returns JSON with a ref for every element an agent can act on; the agent acts on a ref, then snapshots again to check the result, with no pixel matching. When the UI changes, an action returns a stale-ref error instead of hitting the wrong element. By default, actions don't move the pointer or take focus.