agent-desktop is a command-line tool that lets AI agents inspect and control desktop applications through operating-system accessibility data instead of screenshots or pixel matching. It is used to automate applications such as Finder, Safari, Xcode, and Slack with structured JSON commands and results.
Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add lahfir/agent-desktop --skill agent-desktop-linuxgit clone --depth 1 https://github.com/lahfir/agent-desktopWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/lahfir/agent-desktop/agent-desktop-linux)<a href="https://agentmods.dev/skills/lahfir/agent-desktop/agent-desktop-linux"><img src="https://agentmods.dev/badge/skills/lahfir/agent-desktop/agent-desktop-linux/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/lahfir/agent-desktop/agent-desktop-linux"><img src="https://agentmods.dev/badge/skills/lahfir/agent-desktop/agent-desktop-linux.svg" alt="Reviewed on agentmods" width="80" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00132 | $0.03930 |
| Opus 5.5 | $0.00053 | $0.01572 |
| Sonnet 5.5 | $0.00026 | $0.00786 |
| Haiku 4.5 | $0.00013 | $0.00393 |
Grade B, and why
agent-desktop-linux scanned grade B with 1 finding against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 11d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Asks for rootmediumPrivilege escalation
A mod that escalates privileges can change anything on the machine, not only the project.
`INTERNAL`; run `chmod 700` on it or let the tool create it. Copies of this mod
1 near-identical copy found in the catalogue:
- agent-desktop-linux — 88% identical, 41 lines differ
How it starts
The opening of the file, as written. The whole thing — 230 lines — stays where its author put it; the contents beside it link to each section on GitHub.
agent-desktop-linux
The observe-act loop, refs, JSON envelope, sessions and tracing match every platform. This file covers only what differs on Linux.
Needs Ubuntu 24.04+ (x86_64 or aarch64), a graphical session of the signed-in
user and a running AT-SPI bus. Other distributions need the at-spi2-core bridge
fix dc0dc33; Ubuntu 22.04's bridge aborts on Action.GetActions. The archive
loader floor is glibc 2.35. Run the CLI inside the target apps' desktop session:
the session bus, display and runtime directory come from its environment. A TTY
login, an SSH shell without session variables, a container without a bus or
another user's session sees no apps. GNOME is the measured desktop; X11 and
Wayland sessions both work and differ in input, capture and window control.
Observation and semantic actions need no helper tool.
Capability Table
| Group | Commands | Status on Linux |
|---|---|---|
| Observation | snapshot, find, get, is |
Works on X11 and Wayland through AT-SPI. Supports window, sheet, alert and menu; GNOME adds shell surfaces. Virtual tray getters have limits; see the shell reference and check status |
| Surfaces | list-surfaces |
Lists an app's windows, sheets and open menus; shell reads use snapshot --surface |
| Ref interaction | click, right-click, type, set-value, clear, select, toggle, check, uncheck, expand, collapse, scroll-to |
Need a readable live identity and the required AT-SPI capability. See the shell reference for menu-ref resolution limits; X11 headed fallback also needs usable geometry and does not bypass identity checks |
| Wheel scrolling | scroll, mouse-wheel |
Physical wheel input; X11 and --headed only |
| Pointer gestures | double-click, triple-click, hover, drag |
--headed, X11 only. On Wayland the pointer cannot be aimed and these refuse with ACTION_NOT_SUPPORTED |
| Focus | focus |
Needs --headed for the focus permission. Calls AT-SPI Component.GrabFocus on a focusable node that is not yet focused, not the pointer, and verifies the state; a GTK4 app rejects the call, so focus ends ACTION_FAILED grab_focus_rejected there and focus-window is the way to focus its window |
| Keyboard and mouse | press, mouse-click, mouse-move, mouse-wheel |
press and text keys work on X11 and capability-detected Mutter Wayland sessions; pointer commands are X11 and --headed. See references/display-servers-and-input.md |
| Held input | key-down, key-up, mouse-down, mouse-up |
Fail closed on every platform: on Linux, key-down and key-up with ACTION_NOT_SUPPORTED; mouse-down and mouse-up with POLICY_DENIED headless and ACTION_NOT_SUPPORTED with --headed |
| App and window | launch, close-app, list-windows, list-apps, focus-window, move-window, resize-window, minimize, maximize, restore |
Launch and listing work on X11 and Wayland; focus-window requires X11. Other window control and graceful close need an X11 client. See references/windows-and-lifecycle.md |
| Displays | list-displays |
Works |
| Capture | screenshot |
X11: a window without permission; a display only under a non-compositing manager. Wayland: portal consent, with display cropping. No selector returns the primary display. See references/capture-and-clipboard.md |
| Clipboard | clipboard-get, clipboard-set, clipboard-clear |
Works; text, images and file lists. Wayland needs wl-clipboard |
| Wait | wait |
Works for ms, --element, --window, --text, --menu, --menu-closed, --event. Menu waits share the menu-surface detector; --notification waits for a new notification on GNOME |
| Notifications | list-notifications, dismiss-notification, dismiss-all-notifications, notification-action |
GNOME Shell only; list headless, mutations --headed on X11, and dismiss-notification and notification-action also need --expected-app or --expected-title. Fields are read from the shell's widget layout, verified on GNOME Shell 46.0 only, and each mutation is proven by its effect. Wayland refuses the shell pointer before input. A notification owner that is not the shell or its own helper returns PLATFORM_NOT_SUPPORTED. See the shell reference |
| Shell surfaces | open-system-surface |
Only system-tray and action-center can work. system-tray needs a StatusNotifierWatcher (else it refuses with watcher: absent); action-center needs proven role ancestry. quick-settings refuses with shell_container_unproven, taskbar always refuses, and start-menu and system-tray-overflow are not supported, though --help lists them. See the shell reference |
| Cursor overlay | cursor-overlay |
Requires an active session; see Cursor overlay below. X11 enable returns data.rendered: true; Wayland saves the setting with data.rendered: false |
| System | status, permissions, version, batch, skills, session, trace |
Works |
What ships with it
6 files beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 11d ago Changed · -1 lines 5091c0b4f80b
- 13d ago First seen · 231 lines · 132 tokens per session scan B 37ab5e00bc88
agent-desktop-linux is a skill published in the GitHub repository lahfir/agent-desktop (1,830 stars, last pushed 9d ago), licensed Apache-2.0. It adds 132 tokens to every session and 3,930 once invoked, about $0.0005 per session on Opus 5.5. A static security scan graded it B with 1 finding (asks for root). No closer match exists in the catalogue, so it is treated as the original; first seen 2026-10-26.
agent-desktop-linux compared with skills from other repositories
| Mod | Type | Stars | Last push | Licence | Scan | Tokens | Works with |
|---|---|---|---|---|---|---|---|
| agent-desktop-linuxlahfir/agent-desktop | Skill | 1.8k | 9d ago | Apache-2.0 | B | 132 | Claude Code, Codex |
| kmsgchannprj/kmsg | Skill | 283 | 1mo ago | MIT | A | 71 | Codex |
| kakaoclisilver-flight-group/kakaocli | Skill | 183 | 7mo ago | MIT | A | 14 | Claude Code, Codex |
| gws-calendargoogleworkspace/cli | Skill | 32k | 5d ago | Apache-2.0 | A | 12 | Claude Code, Codex |
| gws-gmail-sendgoogleworkspace/cli | Skill | 32k | 5d ago | Apache-2.0 | A | 13 | Claude Code, Codex |
| gws-gmailgoogleworkspace/cli | Skill | 32k | 5d ago | Apache-2.0 | A | 16 | Claude Code, Codex |
| gws-gmail-watchgoogleworkspace/cli | Skill | 32k | 5d ago | Apache-2.0 | A | 20 | Claude Code, Codex |