agent-desktop is a command-line tool that lets AI agents inspect and control desktop applications through operating-system accessibility data instead of screenshots or pixel matching. It is used to automate applications such as Finder, Safari, Xcode, and Slack with structured JSON commands and results.
Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add lahfir/agent-desktop --skill agent-desktop-macosgit clone --depth 1 https://github.com/lahfir/agent-desktopWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/lahfir/agent-desktop/agent-desktop-macos)<a href="https://agentmods.dev/skills/lahfir/agent-desktop/agent-desktop-macos"><img src="https://agentmods.dev/badge/skills/lahfir/agent-desktop/agent-desktop-macos/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/lahfir/agent-desktop/agent-desktop-macos"><img src="https://agentmods.dev/badge/skills/lahfir/agent-desktop/agent-desktop-macos.svg" alt="Reviewed on agentmods" width="80" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00098 | $0.01509 |
| Opus 5.5 | $0.00039 | $0.00604 |
| Sonnet 5.5 | $0.00020 | $0.00302 |
| Haiku 4.5 | $0.00010 | $0.00151 |
Grade A, and why
agent-desktop-macos scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 4d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 67 lines — stays where its author put it; the contents beside it link to each section on GitHub.
agent-desktop on macOS
Read the core skill first (agent-desktop skills get desktop) for the observe, act, verify loop. This skill covers only what is different on macOS. Symptoms and fixes are in references/troubleshooting.md.
Requirements
- macOS 13 or later, Intel or Apple Silicon.
- Accessibility permission for the app that launches agent-desktop (Terminal, iTerm2, Warp, VS Code, Codex). If System Settings lists the
agent-desktopbinary separately, grant it too. - Screen Recording permission, only for
screenshotand forsession start --screenshotsartifacts. - Automation permission for System Events, only where a command reports it missing.
First contact
- Run
agent-desktop status. It includes the permission report and the state root. - Run
agent-desktop permissions. It reportsaccessibility,screen_recordingandautomation. Each isgranted,denied,not_requiredorunknown. A denied entry carries asuggestion. - If a permission is denied, run
agent-desktop permissions --request. It shows the system prompt. - Open System Settings > Privacy & Security > Accessibility. Turn on the launching app.
- Restart the launching app. A running process does not pick up a new grant.
- Run
agent-desktop snapshot --app Finder -ito confirm.
permissions does not prompt. permissions --request can prompt, including for Automation. Automation reports unknown when macOS would need to prompt.
What differs on macOS
Headless by default. Ref actions use accessibility actions only. They do not steal focus, move the cursor, send keys, or write the pasteboard. Add the global --headed flag only when you want physical input.
Headed is physical first for click, right-click, type, clear and scroll. Core focuses the exact window of the ref. The adapter then sends a real click, key or wheel event to the verified point. expand, collapse, set-value, select, toggle, check, uncheck, focus and scroll-to stay semantic under --headed. Double-click, triple-click, hover and drag need --headed. Raw --xy input never changes focus.
What ships with it
1 file beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 4d ago First seen · 67 lines · 98 tokens per session scan A 18a67d0f4c47
agent-desktop-macos is a skill published in the GitHub repository lahfir/agent-desktop (1,816 stars, last pushed today), licensed Apache-2.0. It adds 98 tokens to every session and 1,509 once invoked, about $0.0004 per session on Opus 5.5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-10-19.
Other skills, from other repositories
kwin-desktop-automation
Use when the user asks to launch, click, type, screenshot, or otherwise drive a Linux KDE Plasma / Wayland desktop app through the kwin-mcp MCP server. Trigger when kwin-mcp tools are available and the task involves desktop GUI automation, end-to-end GUI testing, kiosk / embedded device control, or live KDE Plasma…
tauri-pilot
Inspect, interact with, and test a running Tauri v2 app via CLI. Communicates over Unix socket using JSON-RPC 2.0. Use when testing UI, automating interactions, or debugging a Tauri app.
hatch3r-browser-verify
Opt-in browser verification skill — spec-run-first Playwright verification (assertions execute in the runner, agent reads only failures), axe-core a11y audits, toHaveScreenshot() regression diffs, E2E test scaffolds, and snapshot-mode exploratory driving. Default ON for UI-affecting agent invocations; disable globally…
computer-use
A set of rules for controlling Windows desktop applications through screen observation and simulated mouse and keyboard actions. It also describes how to find and launch installed applications and verify the results.
web-navigator
A browser-automation routing guide for web tasks. It directs navigation, page inspection, and actions such as clicking or filling forms to the appropriate browser tool.
pagouse
Observe and (with a grant) drive Chromium tabs via the pagouse CLI: snapshot the accessibility tree, click/fill by ref, navigate. Not a seat adapter. Trigger keywords: pagouse, browser tab, fill form, click the button on the page, snapshot page, AX tree.