oculos: Instructions file for Codex

OculOS — AI Agent Instructions

You have access to OculOS, a local REST API that lets you read and control any desktop application through its UI Automation tree.

Base URL: http://127.0.0.1:7878

Auth: if the server was started with a token (or bound to a network address), send it on every request: X-OculOS-Token: <token> (or Authorization: Bearer <token>). GET /health returns auth_required and never needs the token. A 401 with code unauthorized means the token is missing or wrong — ask the user for it.

---

Core Workflow

1. list_windows          → find the target application (get pid / hwnd)
2. find_elements / tree  → locate the elements you need (get oculos_id)
3. interact              → act on those elements
4. wait / find again     → confirm the UI reached the state you expect

Element ids are stable. An oculos_id is 16 hex characters. Finding the same element again returns the same id, so you can re-run find as often as you like. An id stops working when the element disappears or after 30 minutes without use — you then get a not_found error: run find again and use the fresh id.

---

Discovery

List all windows

GET /windows

Returns: [{ pid, hwnd, title, exe_name, rect, visible }]

Search for elements (preferred — fast)

GET /windows/{pid}/find?q=Submit&type=Button&interactive=true
GET /hwnd/{hwnd}/find?q=Search&interactive=true

Parameters:

• q — case-insensitive substring match on label or automation_id
• type — element type filter, case-insensitive (see the list below; an unknown type → 400 invalid_input listing the valid ones)
• interactive=true — return only elements that have at least one action

Element types: Window, Button, SplitButton, Edit, Text, CheckBox, RadioButton, ComboBox, ListBox, ListItem, TreeView, TreeItem, Menu, MenuBar, MenuItem, TabControl, TabItem, ToolBar, StatusBar, ScrollBar, Slider, Spinner, ProgressBar, Image, Link, Group, Pane, Dialog, Document, DataGrid, DataItem, Header, HeaderItem, Table, TitleBar, ToolTip, Separator, Calendar, Thumb, Custom, Unknown.