computer-use

A set of rules for controlling Windows desktop applications through screen observation and simulated mouse and keyboard actions. It also describes how to find and launch installed applications and verify the results.

In plain words
What is it for?
It is for tasks such as opening applications, searching within them, changing settings, playing music, and saving screenshots.
Why use it?
It provides a defined process for safely carrying out desktop tasks when working directly with native Windows apps. The instructions require checking system information and the visible screen before acting.

Skill for Claude CodeCodex

Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

agentmods
npx agentmods add skills/plocr/reasonix-computer-use/computer-use
Any agent
npx skills add Plocr/Reasonix-computer-use --skill computer-use
Clone the repo
git clone --depth 1 https://github.com/Plocr/Reasonix-computer-use

Made for: Claude Code, Codex.

Per session 23 Skills are progressive disclosure: only the name and description are preloaded; the body loads when the skill is used.
When invoked 2,505 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 0 findings. Scan, not verified.
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5 $0.00023 $0.02505
Opus 5 $0.00012 $0.01252
Sonnet 5 $0.00005 $0.00501
Haiku 4.5 $0.00002 $0.00250

Measured 2d ago against content hash c04dad308fcd, method: parsed. Prices are Anthropic first-party input rates as of 2026-08-30, from the pricing page.

Security

Grade A, and why

computer-use scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 2d ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

skills/computer-use/SKILL.md · 161 lines

How it starts

The opening of the file, as written. The whole thing — 161 lines — stays where its author put it; the contents beside it link to each section on GitHub.

Computer Use — 原生桌面操作

你是 Reasonix 桌面自动化 Operator。按以下契约执行任务。

工具

工具 作用
screen_interactor(observe) 观察窗口,返回结构化元素(ID、bbox、text、a11y 来源)
screen_interactor(execute) 执行最多 5 步操作:click_reftypepressscrolldrag
computer_app 启动/搜索应用。插件已预先索引系统所有应用路径
computer_system 系统画像、诊断、窗口列表、文件搜索(Known Folders)

执行流程

核心流程:先读记忆 → 再分析 → 再执行 → 后验证。绝不在分析前做任何操作。

第零步:读取系统记忆(必须首先执行,禁止跳过)

⚠️ 严格禁止调用 computer_system(operation="profile") 重新生成画像——画像已存在!

读取插件记忆目录中的系统画像文件,了解已安装应用和系统环境:

  1. 读取 memory/system.md — 获取硬件摘要、已安装应用列表、常用目录
  2. 读取 memory/system-index.json — 获取应用精确路径、Known Folders、显示器信息

从记忆中找到与任务相关的应用名称和路径。如果记忆文件不存在或缺少目标应用, 使用 computer_app(operation="search") 重新扫描。

降级策略(按顺序尝试)

  1. 记忆中有目标应用 → computer_app(launch, query="应用名")
  2. 记忆中没有 → computer_app(search, query="应用名")
  3. 搜索也找不到 → press + keys: ["win"] 打开开始菜单,然后 type 搜索
  4. 系统中确实没有 → 考虑用浏览器打开网页版

第一步:任务分析

拿到指令后,必先拆解为原子步骤,列清单,再动手。

用户指令 拆分步骤
"放首歌" ① 查系统画像找音乐软件 → ② 有则启动/无则开浏览器 → ③ observe → ④ 搜索框输入 → ⑤ 点播放
"QQ换主题" ① 启动QQ → ② 登录 → ③ observe → ④ 找设置 → ⑤ 个性化 → ⑥ 选主题色
"截图保存" ① observe → ② 隐藏工具截图 → ③ 验证文件

第二步:执行

按分析好的步骤逐步执行,每步一个原子操作:

  • 启动应用 — 按优先级:① computer_app(launch)→画像解析 ② computer_app(search)→重扫 ③ press:["win"]→type 搜索
  • 观察screen_interactor(mode="observe") 获取元素
  • 操作screen_interactor(mode="execute", actions=[{element_ref, type}])

第三步:验证

操作后检查 execute 返回的 after 快照的 element_count 变化确认生效。blocked=true 时停止汇报。

任务分解

收到用户指令后,先将任务拆解为原子步骤,再逐步执行。每步执行后用 after 快照验证。

示例:

用户指令 拆分步骤
"帮我放首歌" 1. 检查是否安装音乐软件(查系统画像)→ 2. 有则启动 → 3. observe 找到搜索框 → 4. 输入歌名 → 5. 点击播放。如无音乐软件,改用浏览器打开网页版
"打开QQ换主题" 1. 启动 QQ → 2. 登录(如需)→ 3. observe 找到设置入口 → 4. 进入个性化/主题 → 5. 选择目标主题色 → 6. 确认应用
"截图保存桌面" 1. observe 确认当前窗口 → 2. 调用隐藏工具截图 → 3. 验证文件已生成

原则:

  • 每步只做一个原子操作(一次点击 / 一次输入 / 一次观察)
  • 操作后检查 after 快照的 element_count 是否变化以确认生效
  • 遇到 blocked=true 停止并汇报

Read the full file on GitHub · 161 lines

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. 2d ago First seen · 161 lines · 23 tokens per session scan A c04dad308fcd

Subscribe to this mod's changes

computer-use is a skill published in the GitHub repository Plocr/Reasonix-computer-use (40 stars, last pushed 28d ago), licensed MIT. It adds 23 tokens to every session and 2,505 once invoked, about $0.0001 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.

Related

Other skills, from other repositories

clawdcursor

FALLBACK ONLY — do not invoke unless you have already ruled out (1) a native API (Gmail API, GitHub API, Slack API …), (2) a CLI (git, gh, aws, npm, curl …), (3) direct file editing, and (4) existing browser automation (Playwright, Puppeteer). Only when all four are unavailable or have already failed should you use…

AmrDab/clawdcursor · 265 tokens

nova-grounding

AX-first reading and grounded activation workflow for Nova desktop automation on macOS and Windows. Use for reading UI, clicking controls, OCR fallback, screenshots, and coordinate input without guessing.

bigduu/Nova · 40 tokens

edit-video

Edit video in DaVinci Resolve. Color grade footage, edit timelines, add transitions, render projects, apply LUTs, add nodes, navigate pages. Use when: "color grade", "edit timeline", "render in DaVinci", "add node", "apply LUT", "cut footage", "DaVinci Resolve", "video editing", "color correction", "deliver"…

manushi4/Screenhand · 86 tokens

macos-harness

Control a whole Mac from one persistent Python session with screenshots, PID-targeted input, an animated virtual pointer, targeted Apple Accessibility, Apple Events, Browser Harness CDP, and filesystem access. Use for native, Electron, browser, dialog, file, or cross-app tasks without moving the physical cursor or…

browser-use/macos-harness · 71 tokens

video

Generate videos with MiniMax H3 — text-to-video, image-to-video, first/last-frame, and reference-driven generation (subject images, motion clips, voice/music audio). Async task flow with a live task card in chat.

thewolffish/wolffish-app · 49 tokens

python

Hermetic Python runtime (uv-managed CPython) for native Python plugins.

thewolffish/wolffish-app · 16 tokens