native-devtools-mcp AGENTS.md

Instructions for using an MCP server that lets an agent see and control desktop, mobile, and Android applications through screenshots, text recognition, mouse and keyboard input, and device controls.

In plain words
What is it for?
Automating desktop or mobile interfaces, reading screen content, managing windows, entering text, and controlling Android devices through ADB.
Why use it?
It provides a repeatable observe, locate, act, and verify process for computer automation, where actions can change the real machine.

Instructions file for CodexOpenCode

Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

agentmods
npx agentmods add instructions/vectora-foundry/native-devtools-mcp/agents-md
Clone the repo
git clone --depth 1 https://github.com/vectora-foundry/native-devtools-mcp

Made for: Codex, OpenCode.

Per session 8,302 This file is loaded in full into every session.
When invoked 8,302 The same file — it is already loaded in full.
Security scan A 0 findings. Scan, not verified.
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5 $0.08302 $0.08302
Opus 5 $0.04151 $0.04151
Sonnet 5 $0.01660 $0.01660
Haiku 4.5 $0.00830 $0.00830

Measured 2d ago against content hash e00104496cc8, method: parsed. Prices are Anthropic first-party input rates as of 2026-08-30, from the pricing page.

Security

Grade A, and why

native-devtools-mcp AGENTS.md scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 2d ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

AGENTS.md · 461 lines

How it starts

The opening of the file, as written. The whole thing — 461 lines — stays where its author put it; the contents beside it link to each section on GitHub.

Agent Context: native-devtools-mcp

About: This is the AGENTS.md for native-devtools-mcp, an MCP (Model Context Protocol) server that enables computer use / desktop automation on macOS, Windows, and Android: screenshots, OCR, mouse/keyboard input, window management, and Android device control via ADB.

Search keywords: MCP, Model Context Protocol, computer use, desktop automation, UI automation, RPA, screenshots, OCR, screen reading, mouse, keyboard, macOS, Windows, Android, ADB, mobile testing, native-devtools-mcp.

Role: You are an agent equipped with "Computer Use" capabilities. You can see the screen, type, move the mouse, and interact with native desktop and mobile applications.

Constraint: You are operating a real machine. Actions are permanent. Ensure you verify the state of the screen before and after actions.

🧠 Core Reasoning Loop

For robust automation, follow this "Visual Feedback Loop":

  1. OBSERVE: Call take_screenshot(app_name="TargetApp") to see the current state.
  2. LOCATE: Analyze the image or use the OCR summary text in the response to find coordinates.
  3. ACT: Call click(), type_text(), or scroll() using those coordinates.
  4. VERIFY: Call take_screenshot again to confirm the action had the intended effect.

macOS-preferred branch (native apps): substitute OBSERVE with take_ax_snapshot(app_name='...'); LOCATE reads uid + bbox from the emitted tree; ACT calls ax_click(uid) for pressable controls, ax_set_value(uid, text) for text fields, or ax_select(uid) for NSOutlineView / NSTableView row selection (sidebars, rule lists); VERIFY re-snapshots and reads the new state. This branch does not move the cursor or steal focus, so it composes with background work.


🗺️ Capabilities Matrix (Strategy Guide)

Use this table to choose the right tool sequence for the user's goal.

User Goal Tool Sequence Why?
"Click the 'Submit' button" find_text(text="Submit")click(x, y) Fastest. No visual analysis needed if text is known.
"Click the red icon" take_screenshot() → (Analyze Image) → click(screenshot_x=..., screenshot_y=..., screenshot_origin_x=..., screenshot_origin_y=..., screenshot_scale=...) Visual features require full screenshot analysis.
"What element is at (500, 300)?" element_at_point(x=500, y=300) Returns the accessibility element at those coordinates (name, role, bounds, etc.).
"Type into the search bar" find_text(text="Search")click(x, y)type_text("hello") Must click to focus before typing.
"Scroll down" scroll(x=500, y=500, delta_y=200) Positive delta_y scrolls down.
"Find an open window" list_windows()focus_window(window_id=...) Don't guess window names; list them first.
"Track what I hover over" start_hover_tracking(min_dwell_ms=300) → user moves mouse → stop_hover_tracking() Records element transitions with dwell filtering.
"Record what the user does" start_recording(output_dir="/tmp/rec") → user interacts → stop_recording() Captures frontmost app at ~5fps as JPEG frames.
"Launch Safari with debug port" launch_app(app_name="Safari", args=["--remote-debugging-port=9222"]) Pass CLI args on fresh launch.
"Quit an app" quit_app(app_name="Safari") Graceful by default; use force=true to kill immediately.
"Click a named button in a native app (macOS)" take_ax_snapshotax_click(uid) Focus-preserving. Generation-tagged uids; fresh snapshot invalidates prior uids.
"Enter text into a text field via value assignment (macOS)" take_ax_snapshotax_set_value(uid, text) No key events, no IME, no undo-stack entry. Fall back to click + type_text on not_dispatchable.
"Select a sidebar row in System Settings / a NSOutlineView row (macOS)" take_ax_snapshotax_select(uid) Writes AXSelectedRows on the enclosing outline/table. Use this instead of ax_click for row targets — rows typically refuse AXPress.
"AX snapshot invalidation rule (macOS)" Every take_ax_snapshot call bumps the generation. All prior uids become stale; ax_* tools return snapshot_expired.
"Click a button in Chrome" cdp_connect(port=9222)cdp_find_elements(query="Submit")cdp_click(uid="d1") CDP is more reliable than coordinates for web content.
"Type in a web input" cdp_find_elements(query="Email")cdp_fill(uid="d1", value="hello") Works for <input>, <textarea>, and <select> elements.
"Run JS in a browser page" cdp_evaluate_script(function="() => document.title") Evaluate any JS in the selected page.
"Navigate to a URL" cdp_navigate(url="https://example.com") Also supports back, forward, reload.
"Press Enter or shortcut" cdp_press_key(key="Enter") or cdp_press_key(key="Control+A") Supports modifier combos.
"Wait for page content" cdp_wait_for(text=["Success"]) Polls page text until any value appears or timeout. Pass include_snapshot=true to also get a DOM snapshot.
"Switch browser tabs" cdp_list_pages()cdp_select_page(page_idx=1) List tabs, then select by index.
"Get browser page structure" cdp_take_dom_snapshot() Full interactive-element DOM snapshot with UIDs, roles, and labels.

Read the full file on GitHub · 461 lines

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. 2d ago First seen · 461 lines · 8,302 tokens per session scan A e00104496cc8

Subscribe to this mod's changes

native-devtools-mcp AGENTS.md is an instructions file published in the GitHub repository vectora-foundry/native-devtools-mcp (128 stars, last pushed 3mo ago), licensed MIT. It adds 8,302 tokens to every session, about $0.0415 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.