ios-simulator-interaction

iOS Simulator Interaction

Use iosef (via Bash) as the primary tool for all iOS simulator interactions. Compared to idb and simctl, it:

• Lets you interact by AX-tree selectors (--name, --role, --identifier) instead of only bare coordinates
• Scales screenshots so pixel coordinates match iOS point space — no translation needed
• Infers which simulator to use from VCS root or session state
• Provides dedicated assert commands (exists, count, text, wait) with exit codes for scripting

Getting Started

Session lifecycle:

# Create a local session (scoped to this directory)
iosef start --local --device "my-sim"

# ... do work (tap, type, describe, view, etc.) ...

# Tear down — shuts down simulator, deletes device, removes .iosef/
iosef stop

Use --local to avoid interfering with simulators used by other agents. Session state lives in .iosef/state.json — ensure .iosef/ is in .gitignore.

For existing simulators: iosef connect "iPhone 16" --local

Check session state anytime: iosef status (or iosef status --json)

Core Workflow

Inspect: describe and view

Always start with iosef describe to get the accessibility tree. The format is:

AXButton "Label" (center_x±half_width, center_y±half_height)

The center values are the tap targets. Example: AXButton "Start" (197±160, 270±22) → tap at (197, 270).

iosef describe                        # Full AX tree
iosef describe --depth 2              # Limit tree depth
iosef describe --x 200 --y 400       # What's at this coordinate?
iosef describe --json | jq '.. | objects | select(.role == "button")'

Use iosef view for screenshots (not simctl or idb — iosef aligns coordinate spaces):

iosef view                            # Screenshot to temp file (prints path)
iosef view --output /tmp/screen.png   # Screenshot to specific path

Interact: tap, type, swipe

Prefer selectors over coordinates — selectors survive layout changes and scroll position shifts.