agent-desktop is a command-line tool that lets AI agents inspect and control desktop applications through operating-system accessibility data instead of screenshots or pixel matching. It is used to automate applications such as Finder, Safari, Xcode, and Slack with structured JSON commands and results.
Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add lahfir/agent-desktop --skill agent-desktop-windowsgit clone --depth 1 https://github.com/lahfir/agent-desktopWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/lahfir/agent-desktop/agent-desktop-windows)<a href="https://agentmods.dev/skills/lahfir/agent-desktop/agent-desktop-windows"><img src="https://agentmods.dev/badge/skills/lahfir/agent-desktop/agent-desktop-windows/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/lahfir/agent-desktop/agent-desktop-windows"><img src="https://agentmods.dev/badge/skills/lahfir/agent-desktop/agent-desktop-windows.svg" alt="Reviewed on agentmods" width="80" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00119 | $0.02145 |
| Opus 5.5 | $0.00048 | $0.00858 |
| Sonnet 5.5 | $0.00024 | $0.00429 |
| Haiku 4.5 | $0.00012 | $0.00215 |
Grade A, and why
agent-desktop-windows scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 4d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 118 lines — stays where its author put it; the contents beside it link to each section on GitHub.
agent-desktop-windows
The observe-act loop, ref system, JSON envelope, sessions and tracing are the same on every platform. This file covers only what differs on Windows.
Requires Windows 10 1809+ / Windows Server 2019+ (x64 or ARM64). Run it in the interactive desktop session of the signed-in user. Session 0 (services and SYSTEM tasks), Server Core, the secure desktop (UAC prompts, the sign-in screen), a locked desktop and another user's session are not supported: observation, actions and capture there fail or return nothing.
Capability Table
| Group | Commands | Status on Windows |
|---|---|---|
| Observation | snapshot, find, get, is, screenshot |
Works; surfaces are window, focused, a Chromium modal reached as sheet, and an open application menu reached as menu |
| Surfaces | list-surfaces |
Works; per-process inventory of window, focused and sheet surfaces plus a menu surface carrying item_count when a menu is open |
| Shell surfaces | open-system-surface |
Works; raises start-menu, taskbar, system-tray, system-tray-overflow or action-center and returns the window identity snapshot --surface <kind> consumes; quick-settings returns PLATFORM_NOT_SUPPORTED naming action-center |
| Ref interaction | click, right-click, type, set-value, clear, select, toggle, check, uncheck, expand, collapse, scroll, scroll-to |
Works; semantic delivery in headless and --headed modes alike, except right-click, which is a physical click and returns POLICY_DENIED unless --headed (type also requires focus permission and returns POLICY_DENIED under strict headless) |
| Multi-click and focus | double-click, triple-click, focus |
Works with global --headed; focus is headed-required (A3-4, A19-5) |
| Keyboard and mouse | press, hover, drag, mouse-click, mouse-move, mouse-wheel |
Works; cursor-moving input requires --headed; press --app synthesizes into the foreground queue after verification (fails closed headless if not already frontmost; non-interactive callers report delivered_unverified) |
| Held input | key-down, key-up, mouse-down, mouse-up |
Fails closed with ACTION_NOT_SUPPORTED on every platform until a daemon owns held-input lifetime |
| App and window | launch, close-app, list-windows, list-apps, focus-window, resize-window, move-window, minimize, maximize, restore |
Works; launch resolves an absolute path or a bare name, with or without .exe, under System32 or the Windows directory (A21-1), not display names |
| Displays | list-displays |
Works |
| Clipboard | clipboard-get, clipboard-set, clipboard-clear |
Works; typed text and image content |
| Wait | wait |
Works for ms, --element, --window, --text, --menu, --menu-closed, --event, and --notification predicates |
| Notifications | list-notifications, dismiss-notification, dismiss-all-notifications, notification-action |
Works over the Action Center; the commands that raise shell chrome take the foreground, so pass --headed when the center is closed |
| Cursor overlay | cursor-overlay |
Works; enable renders a click-through overlay and data.rendered is the renderer's own pipe acknowledgement, not merely a spawned process. disable carries no rendered field. See references/shell-and-overlay.md |
| System | status, permissions, version, batch, skills, session, trace |
Works |
What ships with it
4 files beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 4d ago First seen · 118 lines · 119 tokens per session scan A fbae73906803
agent-desktop-windows is a skill published in the GitHub repository lahfir/agent-desktop (1,816 stars, last pushed today), licensed Apache-2.0. It adds 119 tokens to every session and 2,145 once invoked, about $0.0005 per session on Opus 5.5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-10-19.
Other skills, from other repositories
kwin-desktop-automation
Use when the user asks to launch, click, type, screenshot, or otherwise drive a Linux KDE Plasma / Wayland desktop app through the kwin-mcp MCP server. Trigger when kwin-mcp tools are available and the task involves desktop GUI automation, end-to-end GUI testing, kiosk / embedded device control, or live KDE Plasma…
tauri-pilot
Inspect, interact with, and test a running Tauri v2 app via CLI. Communicates over Unix socket using JSON-RPC 2.0. Use when testing UI, automating interactions, or debugging a Tauri app.
hatch3r-browser-verify
Opt-in browser verification skill — spec-run-first Playwright verification (assertions execute in the runner, agent reads only failures), axe-core a11y audits, toHaveScreenshot() regression diffs, E2E test scaffolds, and snapshot-mode exploratory driving. Default ON for UI-affecting agent invocations; disable globally…
computer-use
A set of rules for controlling Windows desktop applications through screen observation and simulated mouse and keyboard actions. It also describes how to find and launch installed applications and verify the results.
web-navigator
A browser-automation routing guide for web tasks. It directs navigation, page inspection, and actions such as clicking or filling forms to the appropriate browser tool.
pagouse
Observe and (with a grant) drive Chromium tabs via the pagouse CLI: snapshot the accessibility tree, click/fill by ref, navigate. Not a seat adapter. Trigger keywords: pagouse, browser tab, fill form, click the button on the page, snapshot page, AX tree.