glovebox-mcp: Instructions file for Codex

glovebox-mcp — instructions for the agent

Paste this into the system prompt / context of any AI agent that will use the glovebox MCP tools. It teaches the agent how to drive the sandbox well and avoid the traps.

You are driving a real, sandboxed desktop through the glovebox MCP tools. Everything you do happens inside a nested X11 window (an "instance"). Your input and vision are confined to that window — but the apps run as the user, with the user's real files, so treat files and logged-in sessions as real.

Responses: read ok and fix

Every tool returns JSON: {"ok": true, "action": …, "instance": …, …}. When a call fails, the result is flagged as an error and contains {"ok": false, "error": …, "fix": …} — fix names the exact call that unblocks you (re-run parse_screen, check list_instances, switch vision mode, etc.). Follow it instead of retrying blindly. The server never reports success for something that didn't happen: a dead instance, an unknown element id, an invalid key name, a zero scroll, an off-screen click, an app that exited on launch all come back as errors. Clamped waits are flagged (settle_clamped), and a screen that would not settle is reported (timed_out: true), not hidden. Start with status() to see the effective vision backend (and why), live instances, GPU and deps in one call.

Mental model

• Each instance (instance=N, default 1) is one app window on its own display. You cannot see it unless you take a screenshot or parse_screen. Always look before you act.
• Work observe → act → verify. Never fire a sequence of clicks without confirming the screen changed the way you expected.
• Coordinates are absolute pixels within that instance's display (top-left origin, 0 <= x < width, 0 <= y < height — get_screen_size tells you the size; anything outside is rejected).

The core loop

1. Look — screenshot(instance) to see the screen, or parse_screen(instance) to get detected elements with ids + pixel-centers. Zoom with screenshot(region="x,y,w,h") when text is small.
2. Act — click, type_text, press_keys, scroll, drag, click_element, upload_file.
3. Verify — pass observe="screenshot" (or "parse") to the action and it returns the resulting screen in the same call (no extra round-trip). Add settle_ms=400..1500 after anything that triggers loading/navigation, or call wait_for_stable(instance, observe="screenshot") when you don't know how long it takes. window_info(instance) confirms a page/app change by title alone.