AgentDeck CLAUDE.md

AgentDeck

Stream Deck+ controller for AI coding agents — a bidirectional local control system.

Monorepo

• bridge/ — Node.js server: Daemon hub + Session Bridge (PTY, hook HTTP, state machine). src/apme/ — APME eval module (SQLite store, collector, deterministic+LLM judge runner, category-aware rubrics, turn-level mid-session eval, Pareto recommender, daemon HTTP API). Canonical SessionSample (shared/src/sample.ts + sample_events table) is the SSOT: a bounded unit with a typed trajectory (user/assistant/model/tool/state) that both the timeline and the eval derive from. Per-unit cost via shared/src/pricing.ts (override-loadable), Pareto-frontier recommender (apme/pareto.ts) surfaced via the dashboard Recommend tab, trajectory scorers (apme/scorers/). Eval results are off the device timeline (timeline = activity log only) — they flow via the apme_eval WS event + scorecard/SQLite. Tasks segment on explicit boundaries (/task close, /clear), session_end, or a 15-minute generic idle gap armed at turn close (AGENT_IDLE_GAP_MS, measured: 96% of real claude inter-turn gaps are shorter, and before it existed 100% of codex tasks reached the judge only via the 30-min orphan reaper — hours late or never); TodoWrite-all-completed is a non-segmenting soft hint. A task's display title is its first user prompt, and the judge summary outranks it once it exists (deriveTaskTitle SSOT shared/src/task-title.ts; the Swift copy is GENERATED — pnpm generate-apme-display-rules → TaskTitleRules.generated.swift — with behavior pinned by the shared vector file shared/task-title-vectors.json both suites replay: a markup-first prompt derives null, a slash PATH is not a slash command, all index math is code points): the intent title fills the one-line slot until judged, the outcome sentence wins after, and the Task N fallback stays the display SSOT's "meaningless" shape. The /apme default tab is a task-first Work board (lifecycle buckets + attention-first sort; bucket SQL is single-source in the store so filter, row flag and badge cannot disagree — attention is 7-day windowed because the unwindowed bucket measured 93% of all history). Every row exposes its project + native session as a contextual drill-down; the exact session_id scope gathers that session's tasks and changes ordering to recency so its work reads as one chronology, with the same entry point in Run/Task detail. A turn closes at its Stop, and records WHICH signal closed it (turns.end_source: stop / synthetic_stop / next_prompt / interrupted / session_end / run_close / clear) — closing at the next prompt instead put the user's typing time inside the turn's duration, and without the attribution a dropped Stop hook is byte-identical to a normal close, so its rate was unmeasurable. agentdeck apme stop-health reads it; pre-column rows are never backfilled. A user cancel is not a lost hook: ESC emits no hook at all, so no Stop was ever due — interrupted is its own bucket, outranks synthetic when both are set (a cancel can only ever arrive AS a synthetic Stop), and stays out of the loss ratio. It is detected two ways because one is not enough: the watchdog sees the marker while it is still the transcript tail, and the next-prompt close runs readInterruptSince for the commoner cancel-then-retype shape that buries it. The marker predicate is single-source in bridge/src/claude-interrupt-marker.ts (observer + watchdog + collector) and reads text blocks only — a tool_result quoting the same sentence is routine and would otherwise invent a cancel. Missed-Stop recovery is two wirings, not one — claude-turn-watchdog.ts per PTY bridge, observed-turn-watchdogs.ts per observed session inside the daemon (1.0.20 shipped only the first, which covered none of the real traffic). Recovery re-enters the real Stop path (the daemon self-POSTs its own /hooks/Stop) rather than reimplementing it, and a synthetic Stop must never drain a queue whose delivery needs a listener (takeDirectiveForStop). Task rows render one-per-task: task_end is data-only (spinner stop, judge upsert, reaper) and never a standalone row — the task_start header folds the closure in (SSOT shared/src/timeline-task-display.ts, mirrored Swift/Kotlin/TUI; glance surfaces exclude task rows entirely). Env AGENTDECK_TIMELINE_PROJECTION=1 (default off) flips the device timeline to the sample projection (suppresses adapters' direct chat/tool rows). The default judge chain is mlx → foundationModels, and the FM leg is a FLOOR, not a peer: on-device FM measured 0.580 on the judge-fidelity rubric (vs 0.86–1.00 for the MLX tier) and its window is a hard, measured 4,096 tokens that refuses 4.2% of real task_rollup judge prompts — but flipping the default without keeping FM as the fallback would have removed evaluation outright from every machine with no MLX server. The fallback rides the DEFAULT only (fallbackToFoundationModels): a backend the user NAMED, offline, still skips visibly. Two consequences: anything that must decide before the call — the REVIEW tier and its diff budget — resolves the leg that will actually answer first (resolveJudgeBackend / ReviewRunner.resolveBackend), because the legs' windows differ 4×; and a prompt budget for the on-device leg is sized in MEASURED tokens, never characters (the basic tier's own caps used to build a 5,161-token prompt for a 4,096-token window, since the activity block is the user's own Korean prompts at ~1.15 chars/token). See docs/apme.md
• plugin/ — Stream Deck SDK v2 plugin for macOS and Windows. Five actions: session-slot (keypad) + four SD+ encoders — E1 Volume (utility-dial), E2 Claude Usage (option-dial), E3 Codex Usage (iterm-dial), E4 Launcher (launcher). UUIDs are immutable post-distribution so several no longer match their display name; the mapping is in docs/streamdeck-layout.md. Host controls dispatch per-platform through plugin/src/system/ (darwin/win32 backends behind one facade): macOS uses osascript/open, Windows uses a persistent PowerShell CoreAudio coprocess for volume, rundll32 for URLs, and a safe Get-StartApps lookup for desktop apps. SDKVersion: 3 is mandatory — Maker Console rejects 2 ("Minimum Manifest SDK version must be 3 or later") and DRM follows from the SDK version; verify the DRM-processed build's encoders through the review loop before publishing (see streamdeck-layout)
• plugin-ulanzi/ — Ulanzi Studio plugin for the D200H Deck Dock and D200X LCD keys (official UlanziDeckPlugin-SDK). One dynamic keypad action + session-centric two-level UX, shares the @agentdeck/shared buildSessionDeck layout engine. D200X encoders are a separate, not-yet-shipped action/UX; keypad support must not imply encoder support. Connects to the daemon over WS like the SD plugin and is the sole Ulanzi deck driver — Node direct-HID was deleted (2026-07-08), Swift direct-HID and the legacy zkswe/ research tree were deleted (2026-07-14). The Studio handshake does not identify D200H versus D200X, so the daemon preserves the historical d200h health wire identity and reports connectivity from ulanzi-plugin WS presence. The Marketplace bundle ships NO native binary, and that is enforced at package time, not intended. Through 1.0.4 the SVG rasterizer was @resvg/resvg-js, which needs one .node per architecture: a single Marketplace bundle must run everywhere, so it carried five — 18.5 MB of a 20 MB plugin, four of them unusable on any given machine — and macOS raised "Apple could not verify" on resvgjs.darwin-arm64.node (Ulanzi Studio team, 2026-08-25). Signing was never the fix: a loose native module inside a folder Studio downloads and unpacks has no owner who can sign it for the process that loads it — we do not build Studio, and Studio does not build our dependency. @resvg/resvg-wasm is the same resvg version and the same Rust core (pin the two in lockstep — the identity is the whole argument), ships as one file identical on every OS/CPU, and renders byte-identically: measured over 34 tiles spanning both raster sizes, every session state and feGaussianBlur, same RGBA and same PNG hash. Three consequences. initWasm is async while every tile render is sync, so app.ts awaits initRaster() before the Studio bridge opens, i.e. before any onAdd can ask for a key image. Fonts became load-bearing: the WASM build has no filesystem and therefore no loadSystemFonts fallback, so a bundle missing them renders every tile textless rather than merely differently. And the packaging verify walks the tree for .node/.dylib/.so/.dll rather than checking a list of package names, so a dependency that grows an optional native accelerator later cannot slip one back in. See plugin-ulanzi/VERIFY.md
• shared/ — TypeScript types/utils shared between bridge & plugin (protocol, states, timeline, adapter interfaces, session-utils)
• hooks/ — Claude Code CLI hook installer for ~/.claude/settings.local.json (the App Store opt-in installer writes the user-selected ~/.claude/settings.json), Codex lifecycle hook installer for ~/.codex/config.toml, and OpenCode observer plugin installer for ~/.config/opencode/plugins/agentdeck.js (standalone opencode sessions POST opencode_* lifecycle hooks to the daemon; self-disables in managed PTYs via AGENTDECK_PORT)
• config/ — Default settings and prompt templates
• setup/ — npm setup package (npx @agentdeck/setup)
• android/ — Jetpack Compose launcher app (CremaS, Onyx, Kobo, tablets)
• apple/ — SwiftUI Multiplatform app (iOS/iPadOS/macOS). macOS includes in-process Swift daemon (apple/AgentDeck/Daemon/, no Node.js dependency) — mDNS, device modules (ADB/Serial/Pixoo/Timebox/iDotMatrix), Gateway proxy, HTTP+WS server
• esp32/ — PlatformIO Arduino firmware (LVGL touch displays + WS2812B matrix + InkDeck e-ink). InkDeck is a Seeed TRMNL 7.5" OG DIY Kit (XIAO ESP32-S3 Plus + 800×480 UC8179 e-ink), always USB-powered, driven by custom AgentDeck firmware (PlatformIO env inkdeck, WiFi/WS to the daemon like other ESP32 boards). The allocation-free responsive e-ink geometry SSOT is esp32/src/ui/eink/eink_dashboard_layout.h; InkDeck consumes it directly, scripts/sync-xteink-eink-dashboard.sh mirrors it into the external crosspoint-agentdeck X3/X4 fork (whose GfxRenderer remains hardware-specific), and InkDeckPreview.swift pins it. Its Density bands key off the short edge only — orientation is the separate Layout::portrait field, so a panel can be Compact in either orientation (InkDeck 800×480 and XTeink X4 480×800 both are). Firmware is hardware-verified and ships via WiFi OTA (agentdeck esp32-ota inkdeck). Formerly the "TRMNL" commercial BYOS e-ink device — that HTTP pull integration was removed (Node commit c71044bd). See docs/devices.md