ios-sim-mcp: Instructions file for Claude Code

ios-sim-mcp

MCP server for driving the iOS Simulator the way Chrome MCP drives a browser: semantic AX tree with stable refs, fast snapshot/tap/type cycles, no screenshot-and-click loops.

Architecture

Layer 1 (current, shipping): wraps idb (Facebook's iOS Device Bridge — talks to CoreSimulator's private framework, no WebDriverAgent bridge). Each call is ~100ms.

Layer 2 (in progress): DYLD_INSERT_LIBRARIES dylib injected at app launch via SIMCTL_CHILD_DYLD_INSERT_LIBRARIES (passed to xcrun simctl launch — idb's launch path doesn't forward env). Opt-in per launch (launch_app({inject: true})).

• 2a ✅ Proof-of-life — constructor runs, emits os_log lifecycle line. See dylib/ios_sim_mcp_dylib.m + dylib/build.sh. Subsystem: com.hmbsoftware.ios-sim-mcp, category: lifecycle. Smoke-tested via test/inject.ts.
• 2b ✅ Unix socket + JSON-Lines RPC. Socket at /tmp/ios-sim-mcp-<sanitized-bundle-id>.sock, perms 0600. Per-connection serial read loop on a concurrent dispatch queue. Methods registered via ism_register_method (ping, info, paste_text). MCP side: src/dylib_client.ts with line-buffered protocol, pending-call map, lazy connect with 4s deadline, auto-reconnect after relaunch. Tools: dylib_ping, dylib_info, dylib_call, paste_text. type_text auto-routes through paste_text when dylib is loaded. Measured: ~1ms per call once connected, 822ms cold first call (mostly socket-existence wait after launch). Smoke-tested via test/dylib_2b.ts + test/paste_perfect.ts.
  • paste_text uses UIPasteboard.generalPasteboard.string = text + first-responder [responder paste:nil], bouncing to the main queue. First-responder lookup walks UIWindowScene.windows (iOS 13+) preferring isKeyWindow, then falls back to UIApplication.windows. Solves three real input bugs: (1) iOS first-letter autocapitalization mangling lowercase emails ([email protected] → [email protected] → login fails), (2) Unicode silently dropped through HID translation (café → nothing), (3) smart-quote substitution chewing a"b'c into A. Verified byte-perfect for emails, passwords, Unicode, and quote-containing strings.
• 2c ✅ view_tree + view_hit_test. Walks UIWindowScene.windows (with UIApplication.windows fallback) → recursive subviews, on the main thread (dispatch_sync(main)). Per node: class, window-coord frame, alpha/hidden/interactive, accessibility id+label+value, text content for UILabel/UIButton/UITextField/UITextView, and vc_class when this view is a UIViewController's viewIfLoaded. Caps: default max_nodes=1500, max_depth=30; hit_cap: true in response when truncated. Skips invisible (hidden / α<0.01 / zero-size) unless include_invisible: true. Renderer: src/view_tree.ts with class_filter / ax_id_contains / text_contains filters (subtree ancestor preservation). Tools: view_tree, view_hit_test. Smoke-tested via test/view_tree.ts. Sub-5ms response for a 105-node SwiftUI Settings tree.
• 2e ✅ JavaScriptCore eval bridge. JSContext singleton (lazily created on first eval_js, persists across calls, reset via eval_js_reset). All evaluation runs on the main queue via dispatch_sync — same discipline as view_tree/paste_text — so JS code can safely touch UIKit. ism_install_js_bridges sets up globals: app/defaults/pasteboard/bundle/process/notif_center as direct Apple objects, plus block-callable helpers key_window()/all_windows()/first_responder()/find_view_by_ax_id/find_view_by_class/find_vc_by_class/post_notification/cls/log. JSExport categories declared on NSBundle, NSProcessInfo, NSUserDefaults, UIPasteboard, UIApplication, UIView, UIWindow, UIViewController, UILabel, UIButton, UITextField, UITextView — all getters declared with @property syntax so JS sees them as value access (bundle.bundleIdentifier), not function calls (bundle.bundleIdentifier()); methods declared with - (return)method: become camelCase JS functions following JSC's selector→name rules (setObject:forKey: → setObjectForKey(value, key)). ism_coerce_js_value maps results to {kind, value} with kinds string|number|boolean|null|undefined|array|object|array_described|objc|other — JSON-able objects pass through; ObjC-bridged objects get {class, description}. Tools: eval_js, eval_js_reset. Smoke-tested via test/eval_js.ts: 13/13 covering arithmetic, string ops, bridged property access, ObjC bridged return, state persistence, syntax/runtime exceptions, NSUserDefaults round-trip, helper functions.
  • Caveat: long-running JS blocks the main queue (dispatch_sync). 5s default MCP-side timeout protects the client but the host app's UI freezes for that duration. Don't run infinite loops.
• 2g ✅ WKWebView eval bridge + open_url deep-link tool. ism_find_webviews() walks all windows DFS and collects every WKWebView. webview_list reports {index, ax_id, url, title, frame, hidden} per webview. webview_eval_js picks one (priority: ax_id > index > first visible > first) and calls -[WKWebView evaluateJavaScript:completionHandler:] on the main queue; the completion handler runs on a background semaphore-waited thread with a 4s cap (below the MCP-side 5s default). Result coercion mirrors eval_js ({kind, value}). MCP-side ergonomic wrappers webview_fill / webview_click / webview_find inline params as JSON and run hardened JS — webview_fill uses Object.getOwnPropertyDescriptor(HTMLInputElement.prototype, 'value').set.call(el, text) + dispatchEvent(new Event('input'/'change')) so React's synthetic value tracker and Angular's NgModel actually pick up the change. open_url tool wraps xcrun simctl openurl <udid> <url> for delivering deep links / OAuth callback URLs / http(s) URLs. Builds with -framework WebKit.
  • Auth flow recipe — out-of-process web auth (the AppAuth / ASWebAuthenticationSession case). The login UI runs in SafariViewService / AuthenticationServicesAgent, a separate system process. The dylib cannot reach it — no DOM injection, no view tree, no URL-protocol interception (URLSession swizzle is host-process only). Don't try to drive the sheet. Instead, short-circuit it: (1) network_stub the token endpoint with a canned token JSON response, (2) open_url("yourapp://oauth/callback?code=test&state=...") to deliver the redirect URL directly to the app — AppAuth's URL-handler resumes the flow and calls the (now stubbed) /token, login completes without ever rendering the web sheet.
  • Auth flow recipe — in-process WKWebView. If the app uses a custom WKWebView-based external user agent (e.g. OIDExternalUserAgentIOSCustomBrowser, or any embedded-browser sign-in), use webview_list to confirm presence, webview_find('input[name=email]') to verify the field is rendered, then webview_fill({selector:'input[name=email]', text:'…'}) + webview_click({selector:'button[type=submit]'}).
  • Limitation: WKWebView itself runs networking in its own process — the host-process URLProtocol interceptor still won't see resource loads from inside the webview. JS eval against the DOM works fine.
• 2f ✅ Network stubbing. Extends ISMURLProtocol.startLoading with a stub-match check before the forwarding path. ism_find_matching_stub does first-match-wins substring lookup on URL + optional case-insensitive method filter, guarded by ism_net_stubs_lock. On match, synthesizes NSHTTPURLResponse with the stub's status/headers, feeds the body through didLoadData, calls URLProtocolDidFinishLoading, and records the response via ism_net_record_completed with the request marked stubbed:true + stub_id. delay_ms honored via dispatch_after. Stub body accepted as body (UTF-8 string) or body_base64 (binary). Methods: network_stub_add/list/remove/clear. Tools: network_stub, network_stubs, network_unstub, network_unstub_all. Smoke-tested via test/network_stubs.ts: 13/13 covering synthesis with custom headers + JSON body, tail marking, method filter (POST stub doesn't catch GET), delay_ms timing, single + bulk removal.
• 2d ✅ Network interception. ISMURLProtocol subclasses NSURLProtocol; we install via [NSURLProtocol registerClass:] (legacy URLConnection path) + swizzle the class methods +[NSURLSessionConfiguration defaultSessionConfiguration] and +ephemeralSessionConfiguration to prepend our protocol class to protocolClasses (modern URLSession path). In startLoading we forward via a fresh ephemeral session whose config is fetched via the ORIGINAL ephemeral IMP (saved in ism_orig_ephemeralConfig_imp) — bypasses the swizzle to break recursion. Re-entry guard: [NSURLProtocol setProperty:@YES forKey:kISMHandledKey inRequest:forward], declined by +canInitWithRequest:. Ring buffer (default 500 records) + side dictionaries for full request/response bodies (default cap 256KB; full bodies retrievable by id). Methods: network_start, network_stop, network_status, network_tail, network_get_body, network_clear, network_self_test. Tools mirror those. Verified end-to-end against httpbin.org (200/405/418), URL filter works, sub-5ms tail latency. Coverage caveats (must be honest about): catches URLSession-based code only — misses raw CFNetwork / nw_connection_t, background URLSessions, gRPC libraries that bypass URLSession, and WKWebView resource loads (separate process). HTTPBodyStream request bodies are noted but not captured. Sessions constructed before network_start are not retro-fitted (relaunch with inject:true to catch app-startup traffic).
  • Critical SwiftUI caveat: accessibilityIdentifier set in SwiftUI (.accessibilityIdentifier("foo")) lives on the SwiftUI AX tree, NOT on the bridged UIView.accessibilityIdentifier. So view_tree's ax_id_contains filter won't find SwiftUI-set ids. Use the AX snapshot tool for SwiftUI ax-id lookups; use view_tree for SwiftUI when you need view class hierarchy, frames, or hit-testing. Native UIKit apps with explicit accessibilityIdentifier on UIView are found correctly by both.
  • view_hit_test returns the topmost hit via [keyWindow hitTest:p withEvent:nil] plus the full nextResponder chain (typically reaches UIApplication / AppDelegate). View controllers in the chain are tagged is_vc: true (rendered as [VC]).
• 2d URLProtocol-based network interception with bodies (the big unlock).
• 2e JSContext eval bridge.