Agent Desktop gives any agent reliable computer use on the desktop. Built with Rust, it sees any app's real UI structure through OS accessibility trees and operates it — refs stay stable and actions stay safe to retry, instead of guessing from pixels.
agent-desktop is a command-line tool that lets AI agents inspect and control desktop applications through operating-system accessibility data instead of screenshots or pixel matching. It is used to automate applications such as Finder, Safari, Xcode, and Slack with structured JSON commands and results.
This repository also configures its own agents. See what agent-desktop tells them →
Skill Claude CodeCodex
C-ABI bindings over agent-desktop's PlatformAdapter. Consumers (Python ctypes, Swift, Node ffi-napi, Go cgo, C++, Ruby fiddle) link libagentdesktopffi.{dylib,so,dll} and call ad functions directly instead of spawning the CLI binary per call. The canonical observe-act workflow is: adinit → adadaptercreate[withsession]…
Skill Claude CodeCodex
Reliable computer use via native OS accessibility trees. Use when an AI agent needs to see and operate desktop applications (click buttons, fill forms, navigate menus, read UI state, toggle checkboxes, scroll, drag, type text, take screenshots, manage windows, use clipboard, manage notifications). Covers 59 command…
Skill Claude CodeCodex needs its repo
Drive a desktop application from one plain-language goal, or resolve one intent at a time, without ever reading the accessibility tree. Use when automating a desktop app and you want to keep a 150-element JSON tree out of your context. Hand run.mjs a whole goal and it observes, decides and acts until the goal is met…