openeyes: Instructions file for Codex

AGENTS.md

openeyes AGENTS.md is an instructions file for Codex, OpenCode from yangpeng366/openeyes. It costs 2,675 tokens per session, scanned A, original, MIT.

Repository handover instructions for OpenEyes, a Windows desktop automation system driven through a command line or MCP server. It finds interface elements through accessibility data, web-page structure, or vision, then performs or records actions.

In plain words
What is it for?
Use them when changing or testing OpenEyes desktop automation, its MCP tools, element detection, action execution, recording and replay, or quality checks.
Why use it?
They explain the system's safety rules, including dry runs, explicit execution, fallback sources, and replay behavior. This helps agents understand whether an action was only planned, simulated, or really performed.

Instructions file for CodexOpenCode

Written for Codex and OpenCode: the file is AGENTS.md. Also seen: mentions AGENTS.md; mentions Codex.

This is yangpeng366/openeyes's own configuration. It tells Codex and OpenCode how to work on openeyes itself, so it is not a mod to install elsewhere. Copy it as a starting point and replace the rules that are about this project. Everything openeyes configures →

Reuse

Borrowing it

Nothing to install: this file belongs to yangpeng366/openeyes. Take a copy, put it at the same path in your own repository, and replace the rules that are about this project with yours.

Copy the file
curl -O https://raw.githubusercontent.com/yangpeng366/openeyes/main/AGENTS.md
Clone the repo
git clone --depth 1 https://github.com/yangpeng366/openeyes

Made for: Codex, OpenCode.

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

agentmods badge for openeyes AGENTS.md

README.md
[![agentmods](https://agentmods.dev/badge/instructions/yangpeng366/openeyes/agents-md/github.svg)](https://agentmods.dev/instructions/yangpeng366/openeyes/agents-md)
Your own site
<a href="https://agentmods.dev/instructions/yangpeng366/openeyes/agents-md"><img src="https://agentmods.dev/badge/instructions/yangpeng366/openeyes/agents-md/github.svg" alt="Measured on agentmods" height="20"></a>

Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.

agentmods 80×15 button for openeyes AGENTS.md

Your own site · 80×15
<a href="https://agentmods.dev/instructions/yangpeng366/openeyes/agents-md"><img src="https://agentmods.dev/badge/instructions/yangpeng366/openeyes/agents-md.svg" alt="Reviewed on agentmods" width="80" height="20"></a>
Per session 2,675 This file is loaded in full into every session.
When invoked 2,675 The same file — it is already loaded in full.
Security scan A 0 findings. A grade says what 26 rules found in the file — not that it is safe.
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5.1 $0.02675 $0.02675
Opus 5.5 $0.01070 $0.01070
Sonnet 5.5 $0.00535 $0.00535
Haiku 4.5 $0.00267 $0.00267

Measured today against content hash fc8deaed55ba, method: parsed. Prices are Anthropic first-party input rates as of 2026-10-07, from the pricing page.

Security

Grade A, and why

openeyes AGENTS.md scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured today.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

AGENTS.md · 126 lines

How it starts

The opening of the file, as written. The whole thing — 126 lines — stays where its author put it; the contents beside it link to each section on GitHub.

AGENTS.md — OpenEyes 交接要点

面向接手的 agent / 人类。目标:读完这页就能动手改,不必修完再回头翻长文。

这项目是干什么的

AI 友好的 Windows 桌面操作原语:see → resolve → act。 任何大模型 agent 通过 CLI(eyes)或 MCP server 驱动桌面 GUI。 核心资产是结构化 element schema 与降级链,不是模型。

六条主要思路

  1. 结构化优先:uia → cdp → vision。能走无障碍树 / DOM 就不看像素; 截图是最后手段,且走解析式 vision(OmniParser v2:先解析成结构化元素再决策)。 每次结果都带 source,说明这答案从哪来。
  2. 易点击:UIA 命中点中心 → 未命中点 vision bbox 中心 → 都不命中走字母 hint overlay。 任何一步都要有可解释的降级原因。
  3. 默认 dry-run:所有改变状态的操作先 dry-run,显式 --go 才真执行。 验收门禁据此判定「真跑了」还是「只是演练」。
  4. MCP 优先:23 个工具的契约是一份稳定接口,改内部不许改契约; examples/mcp-stdio-probe.py 就是为了守住它。
  5. Record & Replay 无需 vision:录的是逻辑步骤(UIAEvent)而非原始事件流, 回放翻译成 openeyes.actuate() 调用;backend 可注入(auto / jev 真模型 / mock), 真模型缺凭据时回落 mock,不静默卡住。
  6. 判断交给模型,阈值留在代码:openeyes/decision/ 的 decide() 把打分分四带 (ACT / FALLBACK / UNCERTAIN / FLAG_FOR_REVIEW),只有 FLAG_FOR_REVIEW 这一带 会置 needs_attention=True——人的注意力只花在真需要人看的地方。
  7. 验收要能判红:浏览器四路探针(type / shot / 双跑 / 失败面)统一一套 verdict 模型, 双跑比对要真哈希;判据必须「有可能失败」,否则门禁形同虚设。

改动规矩

  • 跑 pwsh -NoProfile -File examples\quality-gate.ps1:ruff、UTF-8 无 BOM、MCP 契约 23 工具、pytest 全套。 CI 跑的是同一串步骤,别只跑局部。
  • 全量测试目前 1336 passed / 8 skipped(2026-11-18 实测);改到公共路径就把全量跑一遍。
  • 真机验收用可见窗口(真实 Edge profile),不用无头。
  • 源文件 UTF-8 无 BOM(examples/bom-sweep.py 会扫)。
  • 提交直接推 origin main(单人维护,不走 PR、不 force push)。

接口速查(agent 接入只看这一层)

  • CLI:eyes <command> --json,字段固定;子命令先 dry-run 再 --go。
  • MCP:openeyes-mcp,工具名与 CLI 一一对应。
  • 退出码:0 成功 / 1 失败 / 2 参数错 / 3 没匹配到目标 / 4 后端不可达 / 5 动作没生效。 JSON 走 stdout,人读错误走 stderr。
  • 元素字段:bbox / center / control_type / name / automation_id / class / state。
  • 要更深:README 的 Architecture 与 Record & Replay 两节。

已知边界

  • 只有 Windows 的 UIA 后端真正可用;macOS / Linux 在路线图上。
  • 远程浏览器走 SSH 端口转发 + Host 头改写,隧道断了就是断了,不自动修。
  • vision backend 依赖外部模型权重,离线环境只到 uia / cdp 这层。

实践与调试

每条都是实测过的坑:症状 → 原因 → 对策。命令照仓库实情写,先说「跑什么、看什么」。

1. 中断与回执(四路验收探针):Ctrl+Break → exit 130 + verdict=INTERRUPTED,且 --out 目录必须为空(不写回执);taskkill 走 Windows STATUS_CONTROL_C_EXIT(0xC000013A); TerminateProcess 硬杀 → exit 1 且无回执——只存在 PNG 不算通过,回执是唯一凭证。 对策:探针一律用空的 --out 起步,跑完先看回执再看产物。

Read the full file on GitHub · 126 lines

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. today Changed · +4 lines · +139 tokens per session fc8deaed55ba
  2. 3d ago Changed · +15 lines · +473 tokens per session 3996186b9b05
  3. 4d ago First seen · 107 lines · 2,063 tokens per session scan A b9e4a798a11f

Subscribe to this mod's changes

openeyes AGENTS.md is an instructions file published in the GitHub repository yangpeng366/openeyes (2 stars, last pushed today), licensed MIT. It adds 2,675 tokens to every session, about $0.0107 per session on Opus 5.5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-11-13.

Related

openeyes AGENTS.md compared with instructions from other repositories

openeyes AGENTS.md next to 6 instructions from other repositories
ModTypeStarsLast pushLicenceScanTokensWorks with
openeyes AGENTS.mdyangpeng366/openeyesInstructions file2todayMITA2,675Codex, OpenCode
vscode buildNext.instructions.mdmicrosoft/vscodeInstructions file196ktodayMITA9,719GitHub Copilot
next.js AGENTS.mdvercel/next.jsInstructions file144ktodayMITA8,130Codex, OpenCode
vscode oss-third-party-notices.instructions.mdmicrosoft/vscodeInstructions file196ktodayMITA5,685GitHub Copilot
langchain AGENTS.mdlangchain-ai/langchainInstructions file150ktodayMITA4,680Codex, OpenCode
vscode copilot-instructions.mdmicrosoft/vscodeInstructions file196ktodayMITA3,153GitHub Copilot
generative-ai-for-beginners AGENTS.mdmicrosoft/generative-ai-for-beginnersInstructions file124ktodayMITA2,981Codex, OpenCode