What is winui_mcp_server AGENTS.md?

Instructions for controlling supported Windows desktop applications through their accessibility interface, OCR, and an MCP tool. They describe how to inspect an app's controls before taking actions.

Use them to inspect, describe, and automate Windows desktop app interfaces through the command-line gateway.

They provide a repeatable way to discover visible controls and use text recognition when an app does not expose enough accessibility information.

How whyb/winui_mcp_server configures Codex and OpenCode

Reuse

Borrowing it

Nothing to install: this file belongs to whyb/winui_mcp_server. Take a copy, put it at the same path in your own repository, and replace the rules that are about this project with yours.

Copy the file

curl -O https://raw.githubusercontent.com/whyb/winui_mcp_server/main/AGENTS.md

Clone the repo

git clone --depth 1 https://github.com/whyb/winui_mcp_server

Made for: Codex, OpenCode.

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

README.md

[![agentmods](https://agentmods.dev/badge/instructions/whyb/winui_mcp_server/agents-md/github.svg)](https://agentmods.dev/instructions/whyb/winui_mcp_server/agents-md)

Your own site

<a href="https://agentmods.dev/instructions/whyb/winui_mcp_server/agents-md"><img src="https://agentmods.dev/badge/instructions/whyb/winui_mcp_server/agents-md/github.svg" alt="Measured on agentmods" height="20"></a>

Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.

Your own site · 80×15

<a href="https://agentmods.dev/instructions/whyb/winui_mcp_server/agents-md"><img src="https://agentmods.dev/badge/instructions/whyb/winui_mcp_server/agents-md.svg" alt="Reviewed on agentmods" width="80" height="20"></a>

Per session 1,442 This file is loaded in full into every session.

When invoked 1,442 The same file — it is already loaded in full.

Security scan A 0 findings. A grade says what 26 rules found in the file — not that it is safe.

Origin original No closer match found in the catalogue.

Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

Model	Per session	Once invoked
Fable 5.1	$0.01442	$0.01442
Opus 5.5	$0.00577	$0.00577
Sonnet 5.5	$0.00288	$0.00288
Haiku 4.5	$0.00144	$0.00144

Measured 1mo ago against content hash f4f2874e9449, method: parsed. Prices are Anthropic first-party input rates as of 2026-10-07, from the pricing page.

Security

Grade A, and why

winui_mcp_server AGENTS.md scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 1mo ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

Origin

Copies of this mod

1 near-identical copy found in the catalogue:

winui_mcp_server CLAUDE.md — 100% identical, 0 lines differ

AGENTS.md · 140 lines

How it starts

The opening of the file, as written. The whole thing — 140 lines — stays where its author put it; the contents beside it link to each section on GitHub.

WinUI Automation Agent

Your Role

You are the automation agent for Windows desktop applications. Control any supported app's GUI through the packaged UIA, OCR, and MCP tooling in this repository.

Environment

Package manager: uv - all Python execution must use uv run or uvx.
Run command pattern: uv run python cli_gateway.py <command>
Working directory: D:\codes\github\winui_mcp_server
Accessibility Insights for Windows: AccessibilityInsights\1.1\AccessibilityInsights.exe when present; launch with uv run python cli_gateway.py inspect.

Control Workflow

1. Discover the UIA tree

Use the CLI gateway with --window (title) or --processname (process).

uv run python cli_gateway.py --window "Notepad" state
uv run python cli_gateway.py --window "Notepad" describe
uv run python cli_gateway.py --window "Notepad" dump-tree --depth 4
uv run python cli_gateway.py --processname notepad describe

Prefer dump_tree or discover before acting. Detailed nodes include ref, class, name, value (when available), control type, patterns, visibility, and rectangle.

2. Use OCR when accessible text is missing

When Name/Value is empty or unreliable, use the bundled PP-OCRv6 tiny pipeline:

uv run python cli_gateway.py --window "MyApp" ocr-scan --depth 5 --max-nodes 500

Each control node receives:

ref - stable tree reference such as 0.3.2.1
uia_text - text exposed by UI Automation
ocr_text - OCR text spatially bound to that control
ocr_confidence - average recognition confidence
effective_text - best available label
text_source - uia, ocr, uia+ocr, or none

OCR and screenshots run locally. Models are packaged under winui_mcp/models/PP-OCRv6; no external service is used.

3. Act on the exact node

Use the OCR-derived ref whenever possible:

uv run python cli_gateway.py --window "MyApp" click --ref "0.3.2.1"
uv run python cli_gateway.py --window "MyApp" double-click --ref "0.3.2.1"
uv run python cli_gateway.py --window "MyApp" hover --ref "0.3.2.1"

Read the full file on GitHub · 140 lines

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

1mo ago First seen · 140 lines · 1,442 tokens per session scan A f4f2874e9449

Subscribe to this mod's changes

winui_mcp_server AGENTS.md is an instructions file published in the GitHub repository whyb/winui_mcp_server (2 stars, last pushed 1mo ago), licensed BSD-3-Clause. It adds 1,442 tokens to every session, about $0.0058 per session on Opus 5.5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-10-08.

winui_mcp_server AGENTS.md next to 6 instructions from other repositories
Mod	Type	Stars	Last push	Licence	Scan	Tokens	Works with
winui_mcp_server AGENTS.mdwhyb/winui_mcp_server	Instructions file	2	1mo ago	BSD-3-Clause	A	1,442	Codex, OpenCode
vscode buildNext.instructions.mdmicrosoft/vscode	Instructions file	195k	today	MIT	A	9,719	GitHub Copilot
next.js AGENTS.mdvercel/next.js	Instructions file	143k	today	MIT	A	7,461	Codex, OpenCode
vscode oss-third-party-notices.instructions.mdmicrosoft/vscode	Instructions file	195k	today	MIT	A	5,649	GitHub Copilot
langchain AGENTS.mdlangchain-ai/langchain	Instructions file	149k	today	MIT	A	4,680	Codex, OpenCode
vscode copilot-instructions.mdmicrosoft/vscode	Instructions file	195k	today	MIT	A	3,093	GitHub Copilot
vscode model-prompts.instructions.mdmicrosoft/vscode	Instructions file	195k	today	MIT	A	2,022	GitHub Copilot

winui_mcp_server: Instructions file for Codex