What is desktop-controller?

A guide for controlling Windows desktop applications with mouse, keyboard, browser, and screenshot automation. It chooses Playwright for web or Electron apps and Windows APIs for native desktop apps.

Use it to click buttons, type text, send messages, fill web forms, control native Windows programs, and take screenshots.

It provides a way to automate applications that may not offer an agent-specific interface. Selecting the appropriate control method helps web tasks use page elements and native tasks use desktop controls.

desktop-controller — Skill for Claude Code and Codex by 24kchengYe

Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

Any agent

npx skills add 24kchengYe/desktop-controller-skill --skill desktop-controller-skill

Clone the repo

git clone --depth 1 https://github.com/24kchengYe/desktop-controller-skill

Made for: Claude Code, Codex.

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

README.md

[![agentmods](https://agentmods.dev/badge/skills/24kchengye/desktop-controller-skill/desktop-controller-skill/github.svg)](https://agentmods.dev/skills/24kchengye/desktop-controller-skill/desktop-controller-skill)

Your own site

<a href="https://agentmods.dev/skills/24kchengye/desktop-controller-skill/desktop-controller-skill"><img src="https://agentmods.dev/badge/skills/24kchengye/desktop-controller-skill/desktop-controller-skill/github.svg" alt="Measured on agentmods" height="20"></a>

Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.

Your own site · 80×15

<a href="https://agentmods.dev/skills/24kchengye/desktop-controller-skill/desktop-controller-skill"><img src="https://agentmods.dev/badge/skills/24kchengye/desktop-controller-skill/desktop-controller-skill.svg" alt="Reviewed on agentmods" width="80" height="20"></a>

Per session 304 Skills are progressive disclosure: only the name and description are preloaded; the body loads when the skill is used.

When invoked 3,574 The whole file, excluding the scripts and references it only reads on demand.

Security scan A 1 finding. A grade says what 26 rules found in the file — not that it is safe.

Origin original No closer match found in the catalogue.

Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

Model	Per session	Once invoked
Fable 5.1	$0.00304	$0.03574
Opus 5.5	$0.00122	$0.01430
Sonnet 5.5	$0.00061	$0.00715
Haiku 4.5	$0.00030	$0.00357

Measured 2mo ago against content hash cba34b20c0bb, method: parsed. Prices are Anthropic first-party input rates as of 2026-10-07, from the pricing page.

Security

Grade A, and why

desktop-controller scanned grade A with 1 finding against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 2mo ago.

The scan reads SKILL.md. This mod also ships 3 executable files (scripts/app_registry.py, scripts/desktop_control.py, scripts/playwright_control.py), listed below but not scanned — reading those needs a real analyzer, not pattern matching.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Makes network callslowCapability

Not a fault in itself. Listed so you know the mod talks to something, and to what.

curl --noproxy localhost -s http://localhost:9222/json/version

SKILL.md · 331 lines

How it starts

The opening of the file, as written. The whole thing — 331 lines — stays where its author put it; the contents beside it link to each section on GitHub.

Universal Desktop Controller

Automate any Windows desktop application using a combination of Win32 API, Playwright (for web/Electron apps), and screenshot-based visual feedback. Inspired by OpenAI's playwright-interactive Codex skill, adapted for Claude Code.

Smart Routing: Choose the Right Engine

CRITICAL DECISION: Before any action, determine which engine to use:

User Request → Is it a web/browser/Electron app?
                 ├── YES → Playwright (CSS selectors, instant, precise)
                 │         Examples: email webmail, Slack, browser tasks, web forms
                 │         ✅ page.click('#compose')  — instant, 100% accurate
                 │
                 └── NO  → Win32 API (native desktop automation)
                           Examples: WeChat, QQ, DingTalk, Notepad, File Explorer
                           ✅ SetCursorPos + mouse_event — works for any native app

Why this matters: For the Tsinghua email (a web app), using Playwright CSS selectors is 10-50x faster than screenshot→analyze→click. OpenAI's playwright-interactive proved this: DOM selectors beat vision-based coordinate guessing every time for web content.

Engine	Speed	Precision	Use For
Playwright	<1s per action	100% (DOM selector)	Web apps, Electron apps, browser tasks
Win32 API	2-3s per action	95% (coordinate-based)	Native desktop apps (WeChat, QQ, etc.)
Screenshot+Vision	10-15s per action	80% (AI guessing)	Last resort / unknown UI

Architecture

┌──────────────────────────────────────────────────┐
│             Desktop Controller Skill              │
├──────────────┬──────────────┬────────────────────┤
│  Win32 Layer │ Playwright   │  Visual Feedback   │
│  (Native)    │ (Web/Electron)│  (Screenshot+AI)   │
├──────────────┼──────────────┼────────────────────┤
│ FindWindow   │ Browser      │ CaptureScreen      │
│ SendKeys     │ Page.click   │ CaptureWindow      │
│ SetCursorPos │ Page.fill    │ → Claude Vision    │
│ mouse_event  │ Page.goto    │ → Verify State     │
│ Clipboard    │ Locator      │ → Decide Next Step │
└──────────────┴──────────────┴────────────────────┘

Read the full file on GitHub · 331 lines

Files

What ships with it

3 files beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

2mo ago First seen · 331 lines · 304 tokens per session scan A cba34b20c0bb

Subscribe to this mod's changes

desktop-controller is a skill published in the GitHub repository 24kchengYe/desktop-controller-skill (12 stars, last pushed 7mo ago), licensed MIT. It adds 304 tokens to every session and 3,574 once invoked, about $0.0012 per session on Opus 5.5. A static security scan graded it A with 1 finding (makes network calls). No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-31.

Browser automation agent-skills ai-agent ai-coding-agent anthropic browser-automation claude

desktop-controller next to 6 skills from other repositories
Mod	Type	Stars	Last push	Licence	Scan	Tokens	Works with
desktop-controller24kchengYe/desktop-controller-skill	Skill	12	7mo ago	MIT	A	304	Claude Code, Codex
stealth-browser-mcpvibheksoni/stealth-browser-mcp	Skill	3.3k	yesterday	MIT	A	76	Codex
browser-automation-expertpersonamanagmentlayer/pcl	Skill	44	11d ago	Apache-2.0	A	94	Claude Code
mapbgmacris/quimera-ai	Skill	12	2d ago	MIT	A	98	Claude Code
chromeboostlordamdal/chromeboost	Skill	8	7d ago	MIT	A	338	Claude Code
tandembgmacris/quimera-ai	Skill	12	2d ago	MIT	A	68	Claude Code
chromeflowNeoDrew/chromeflow	Skill	3	today	MIT	A	273	Claude Code

desktop-controller