macbot-mcp CLAUDE.md

macbot-mcp

macOS GUI automation via the Accessibility API, AppleScript, and Quartz.

When to use which tool

Prefer accessibility tools over screenshots. They're 5-15x cheaper in tokens and return structured, searchable data.

Goal	Tool	Tokens
Read UI state (windows, buttons, tabs, text)	mac_ui_tree	~200-700
Find a specific element (button, link, field)	mac_ui_find	~100-300
Get the browser URL	mac_ui_url	~50
See what something looks like visually	mac_screenshot + Read	~3000

Use screenshots only when you genuinely need visual information — layout, colors, images, charts, or anything the accessibility tree can't describe.

Common workflows

Press a button without a screenshot (or coordinates)

mac_ui_press(app="Firefox", name="Submit", role="AXButton")

One call: finds the element and presses it through the Accessibility API. No mouse movement, no focus change — works even if the window is occluded or the element is scrolled out of view. This beats find+click for buttons, links, tabs, and checkboxes; use index= to pick among multiple matches.

Click when press isn't supported (canvas, custom widgets)

1. mac_ui_find(app="Firefox", name="Submit")
   -> {"role": "AXButton", "title": "Submit", "x": 450, "y": 320, "width": 80, "height": 32}
2. mac_click(x=490, y=336, app="Firefox")

Pass app= to mac_click: a click lands on whatever window is at those coordinates, so an occluded target means the wrong window gets it.

Navigate browser tabs

1. mac_ui_tree(app="Firefox", max_depth=5, max_nodes=50)
   -> Find AXRadioButton elements in the tab group with titles and coordinates
2. mac_click at the tab's coordinates
3. mac_ui_url to confirm you're on the right page

Type into an app

1. mac_type_text(text="Hello world", app="TextEdit")
2. mac_key_press(key="return", app="TextEdit")

Always pass app= when typing. Do NOT split focus and typing into separate tool calls (mac_focus_app then mac_type_text) — seconds of model thinking elapse between calls, the terminal running the agent reclaims focus, and the text lands in the wrong window (usually your own input box). The app= param focuses and types atomically inside one call.