pr-cockpit AGENTS.md

pr-cockpit / PR Cockpit

What this project is

• This is the user's local batch PR review workbench. The product name is PR Cockpit; the repo usually lives at /Users/openstudio/work/products/tools/pr-cockpit.
• Core flow: pull the GitHub PR list, give each PR an isolated worktree, have the AI produce a structured review, and let the human vet it in the web UI before posting line-level/summary comments to GitHub.
• Stack: Nuxt 4 + @nuxt/ui/Tailwind v4, better-sqlite3 + drizzle, Nitro server/api/, the local gh CLI, @anthropic-ai/claude-agent-sdk and @openai/codex (the Codex CLI binary, driven through codex app-server).
• Main directories: core/ is the business logic and agent engine, server/api/ is the Nitro API, app/ is the Vue UI, tests/ holds lightweight contract/regression tests, data/ is the local SQLite plus the migration source for the old central worktrees.

Running locally

• Usual checks: pnpm typecheck, pnpm test; also run pnpm build for riskier UI/bundling changes.
• Dev server: pnpm dev, README defaults to http://localhost:3001.
• The long-running pr-cockpit instance on the user's machine is usually on port 5332; 4737 is a different project. Confirm the port before touching processes — don't kill the wrong project's server by matching on the .output/server/index.mjs process name.
• SQLite has no formal drizzle migration flow. Tables are actually created by ensureSchema() and ensureColumns() in core/db/client.ts; core/db/schema.ts only provides query types. When changing the DB, update both places and keep them idempotent, so that re-running them on every startup stays safe.
• Observability tables (phase 0 of the session-host rework, 2026-08): every agent execution writes a runs row (review / guided / recheck / skillgen get their own row per execution; fix / feature / global sessions reuse the entity id as the run id and append per turn) plus run_usage rows per model. Cost comes from the Claude result's modelUsage / total_cost_usd; Codex only reports tokens, so its cost is estimated from data/codex-rates.json (see docs/codex-rates.example.json) or left NULL — never write 0 as a placeholder. Skill text is versioned in skill_versions (core/skillVersions.ts; edits insert a new version, skills.content mirrors the current one) and reviews record skill_version_id. findings.checked_by distinguishes human clicks from the drawer's auto-adjust and the automation engine; the dashboard's precision metric only counts human. Dashboard: app/pages/dashboard.vue ← /api/metrics/overview (aggregates) + /api/metrics/runs (the run list, offset/limit ≤ 100 + total) ← core/metrics/queries.ts. Pages that list things page client-side with app/components/PagerBar.vue (UPagination, 1-based) unless the list is unbounded (the run list pages server-side).
• Session host (phase 1, 2026-08): Claude chats no longer spawn claude -p per turn — core/host/claudeHost.ts keeps ONE long-lived Agent SDK query() per session (streaming-input mode, core/host/queue.ts), fed a user message per turn. core/host/options.ts is the only place SDK options are assembled: settings are loaded like the CLI (no settingSources: []), Claude Code's own system prompt is kept and ours appended, permissionMode is per session (default / acceptEdits / plan / bypassPermissions, switchable mid-session), allowDangerouslySkipPermissions is always on so the switch works later. Safety = the permission bridge (core/host/permissions.ts: canUseTool → a permission_requests row + permission_request RunEvent → the UI answers via POST /api/runs/:id/prompts/:pid → the parked promise resolves; AskUserQuestion = kind question, ExitPlanMode = kind plan) + a PreToolUse hook that turns dangerous Bash (isDangerousCommand in core/agent/dangerGuard.ts) into an ask in every mode unless the run's allowDanger switch is on. All SDK messages are normalized (core/host/normalize.ts) into RunEvents, fanned out on bus channel run:<id> (SSE /api/runs/:id/stream, and the global session stream) and persisted to run_events (text deltas are live-only). Result cost/usage is cumulative per query lifetime — diffCumulativeUsage turns it into per-turn deltas. Idle sessions close after HOST_IDLE_MS (20 min) and resume via resume on the next message; server/plugins/recover.ts expires pending prompts on restart; shutdown.ts closes live queries. Every session run (PR / feature branch / cwd) runs on the host through core/runs/session.ts (worktree kinds default to bypassPermissions + the danger hook, cwd to default; the UI can change the mode per session; pipeline events share the run:<id> channel). Codex sessions run on the Codex host (next bullet) behind the same SessionHost interface — the session pipeline picks a host with hostFor(provider) and the /api/runs/:id/* endpoints with hostOf(runId) (core/host/index.ts). The UI side is shared: app/composables/useRunHost.ts (RunEvent reducer, pending prompts, mode, context meter, cost) + RunPromptCards.vue + RunHostStrip.vue, used by session/SessionView.vue; core/host/pending.ts decodes pending rows for the detail endpoints. Verified live: Write → permission card → allow; AskUserQuestion; plan card → approve → CLI leaves plan mode. Note the CLI auto-allows some read-only commands (e.g. echo) in default mode without a prompt.
• Review family on the host option factory (phase 2, 2026-08): review / guided / recheck / skillgen build their SDK options with buildReviewOptions() in core/host/options.ts — the user's configuration (CLAUDE.md, rules, skills, plugins) is loaded like the CLI and the operating contract + review skill are appended to the Claude Code preset. Read-only is enforced by three independent layers in core/host/readonly.ts: a PreToolUse hook (a hook deny beats every allow rule, and settings.disableAllHooks does NOT switch off SDK callback hooks — verified live), inline settings deny rules + disableAllHooks: true (the user's own hooks never run in a review worktree), and disallowedTools + canUseTool. All Bash decisions go through isDangerousBash in core/agent/guard.ts — a blacklist that can never be complete (variables, backslashes, exotic redirects); treat it as defence in depth and extend tests/host-readonly.test.ts when touching it (write primitives are matched only in command position so paths like patch.ts stay allowed). MCP servers are not even connected unless the owner's "let reviews use MCP" switch is on (core/agent/settings.ts, meta keys agent.chrome / agent.reviewMcp, edited on /agent-config; on = every configured server is callable, like a session, and Claude in Chrome comes along when agent.chrome is set — the review verdict takes a boolean, there is no per-server list any more). Servers the PR branch itself declares are never enabled: buildReviewOptions lists <cwd>/.mcp.json keys in settings.disabledMcpjsonServers (the CLI would otherwise auto-approve and spawn them at init — a server SPAWN is invisible to all three read-only layers), and the Codex host sends mcp_servers: {} to an unattended thread whose worktree has a .codex/config.toml. agent.chrome applies to every Claude session kind (cwd / PR / branch worktree), not only directory sessions. A running review can be stopped (core/agent/reviewAborts.ts, POST /api/reviews/:id/stop, also wired into the Codex runner's stop handle). One-shot text helpers (commit message, feature title, comment rewrite, JSON repair) use runHelperText in core/host/helpers.ts — SDK, no settings, no tools, one turn, explicit cwd — instead of claude --print. CLAUDE_CODE_PROJECT_DIR_NAME pins the memory/transcript directory of worktree runs to the project's main clone (projectDirNameFor).
• Codex on codex app-server (phase 4, 2026-08): @openai/codex-sdk is gone. core/codex/appServer.ts keeps ONE codex app-server --listen stdio:// process per Nitro process (JSON-RPC over NDJSON in core/codex/rpc.ts; never a ws:// listener or the daemon — no second locally reachable control surface; the user's shell aliases codex to bypass sandboxing, so never shell: true), started lazily, restarted with backoff after a crash (live threads get a crashed callback and their turns fail). core/codex/codexHost.ts implements the same SessionHost surface as the Claude host: a thread per run (thread/start / thread/resume, stale ids fall back to a fresh thread + a note event), turn/start per message with the sandbox/approval policy of the CURRENT mode (core/codex/policy.ts: plan → read-only sandbox; default/acceptEdits → workspace-write + on-request; bypassPermissions → never; the danger switch = full access + network), turn/interrupt, /compact → thread/compact/start. Approvals (item/commandExecution|fileChange|permissions/requestApproval, legacy execCommandApproval/applyPatchApproval) and item/tool/requestUserInput go through the shared permission bridge (core/host/permissions.ts rows + permission_request events, answered by the same endpoint/cards; "always" → acceptForSession or the proposed execpolicy amendment). Notifications map to RunEvents in core/codex/mapEvents.ts (pure, fixture-testable); token usage is per-thread cumulative (thread/tokenUsage/updated.total) and differenced per turn; USD is still the rate-table estimate or null. Review / guided / recheck / skillgen use runCodexReadonly in core/codex/oneshot.ts: an ephemeral thread with readOnly sandbox + untrusted approvals, so EVERY command is submitted before it runs and isForbiddenRemoteOrGitMutation declines git/GitHub mutations pre-execution (verified live: git push declined, outputSchema honoured). Sessions keep the post-execution guard (shouldBlockCodexCommand on completed commands → the turn is interrupted and errors) because in-sandbox git commit never asks. Binary resolution lives in core/codex/bin.ts (env → packaged .output/vendor/codex/bin → pnpm vendored → PATH, logged as unpinned); scripts/prepare-electron-codex.mjs copies the vendored binary into .output for packaging (macOS signing of that nested binary is unverified). codexStatus.ts / codexModels.ts read getAuthStatus / model/list from the live server; the transparency page's Codex section is core/codex/describe.ts. Tests: tests/codex-host.test.ts drives the host against tests/helpers/mockCodexAppServer.mjs (set CODEX_EXECUTABLE to a .mjs file and the RPC layer runs it under node). The user's Codex hooks/plugins fire inside threads (a stop hook was observed) — they are part of the loaded configuration, not something we disable.
• Verify-before-post + eval replay (phase 5, 2026-08): projects.verify_before_post (project config switch) makes a fresh review run a second read-only pass (core/agent/verify.ts, same read-only policy as reviews; Codex via runCodexReadonly with an output schema) whose only job is to refute each finding; verdicts land in findings.verify_status / verify_note (refuted findings stay visible but unchecked, the drawer shows a tag) and the pass has its own runs row (subkind = 'verify'). A failed verify never fails the review. Eval replay: pnpm eval run --golden eval/golden/<name>.json --project <id|name> [--provider] [--model a,b] [--effort] [--skill-version|--skill|--methodology] [--verify] (scripts/eval.ts → core/eval/runner.ts) replays labelled PRs at a fixed head sha (prepareWorktree({ checkoutSha, prNumber }) falls back to refs/pull/<n>/head when the branch moved or is gone; no merge of the default branch so the input is exactly the labelled head), scores findings against labels with path + title/problem token matching (core/eval/judge.ts, greedy one-to-one, no LLM judge in v1), reports precision / recall / F1 / cost with and without the verify pass, writes eval_runs / eval_cases / eval_findings and a markdown report under eval/reports/ (git-ignored). It never posts and never writes to git/GitHub. Golden format: eval/golden/example.json.
• Unified session runs (phase 3 completion, 2026-08): the fix / feature / global chat stacks are ONE thing now. A session is a runs row (kind = 'session') bound to a workspace — pr_worktree (a PR branch worktree; edits stay uncommitted until the upload path commits+pushes), branch_worktree (a fresh branch cut from the default branch; the agent may open the PR) or cwd (any directory). Turns live in run_turns, events in run_events, prompts in permission_requests; the workspace state that used to sit on fixes/feature_tasks/global_sessions (base/fix/push shas, pushed_at, reviews_at_push, pr_url, upload_state, busy_action, description) is on runs. core/runs/migrate.ts copied the legacy tables in once (same ids; marker runs.migrated.v1 in meta; the old tables stay as a rollback net and nothing reads them). core/runs/session.ts is the single turn pipeline (runSessionTurn, isRunBusy, stopRun, fixStatusOf = the legacy open/ready/pushing/pushed/error status derived from upload_state/busy_action, used by automation and the PR list). API: POST/GET /api/runs, GET/PATCH/DELETE /api/runs/:id, POST /api/runs/:id/{messages,stop,push,fork,open}, DELETE /api/runs/:id/workspace, plus the host endpoints (stream, events, interrupt, mode, prompts/:pid). Provider follows the project/runtime defaults until a native session exists, then the run's own row pins it; before a provider takes over, the other host's live session for that run is closed (hostOf must never route to a stale one). UI: app/components/session/SessionView.vue is the one chat surface (turns, host cards, ask-user card, slash palette, danger/mode/ultracode switches, upload preview, open/update PR, worktree tools, open in VS Code/Cursor/Terminal); PrDetailDrawer (fix tab) and GlobalChat (the project assistant: FAB + slideover on project pages ONLY, workspace picker cwd / new branch worktree for a new session, per-project history of both kinds with rename / delete / fork, ?session=<id> deep link via useOpenGlobalSession) are thin shells around it — the separate feature tab / SessionsTab was removed 2026-08-27; sessions without a projectId are adopted into a project's history when their path lies under its clone (GET /api/runs). History handoff between providers is /api/agent/history/run/:id (core/agent/historyAccess.ts). Automation dispatches to POST /api/runs + /messages + /push; server/plugins/recover.ts reconciles busy_action = pushing and streaming run_turns on boot.
• Housekeeping that closed the plan (2026-08): the per-turn claude -p chat runner (runClaudeAgentChat, claudeCli.ts, the danger hook file writer, ChatRunner) is deleted — only the system prompt builders remain in core/agent/{fixer,featureChat,globalChat}.ts. Sessions in worktrees pin CLAUDE_CODE_PROJECT_DIR_NAME to the project clone (one memory dir per project). REVIEW_MAX_BUDGET_USD caps a review-family execution (unset = no cap). core/host/recover.ts is the boot-time host recovery (tested). The session stream renders host events as cards (session/RunEventCard.vue: tool call + result, Edit/Write change, thinking, subagent, compaction, denial). Codex: core/codex/protocol/ holds the generated app-server bindings for the pinned @openai/codex (pnpm codex:types regenerates; EXPECTED_CODEX_VERSION in core/codex/bin.ts must match package.json — a test checks it, and the handshake warns on a different binary); the RPC layer retries -32001 (overloaded) with backoff; /fork is a local slash command; stopping a PR session reports that the PR's automation was paused. tests/host-config.probe.ts is the manual CLI probe (plugins / Chrome / connectors / memory files).
• Session composer extras (2026-08, the items rescued from the plan's §6 cut list): (1) Slash palette — core/host/commands.ts classifies the CLI's command list (user/project skills by their "(user)"/"(project)" description suffix, plugin commands by namespace, the rest built-in; only CURATED_BUILTINS show without "show all", HIDDEN_COMMANDS never; MCP prompts lose their display-only " (MCP)" suffix). The catalogue comes from GET /api/agent/commands?provider&cwd|projectId (the probe cache) and is replaced by a live commands_changed push (RunEvent commands). Matching is PREFIX matching on the name, any ./:/__ segment and aliases (/plan → speckit.plan, Notion:tasks:plan). session/CommandPalette.vue is the dropdown + grouped browser; cockpit-side commands (/clear /new /resume /fork /cd /copy /model /effort /stop /push /pr) shadow same-named built-ins and are intercepted in SessionView.handleSlash (/model /effort → POST /api/runs/:id/settings → persisted on the run + host.setModel). Codex: skills/list feeds the same palette and /name args becomes a {type:'skill'} input item in codexHost.startTurn. (2) Message queue — a message sent while a turn runs is a run_turns row with status queued (submitSessionTurn / cancelQueuedTurn in core/runs/session.ts; DELETE /api/runs/:id/queue/:turnId withdraws it); the next queued turn starts when the running one ends, Stop drops the queue, recover.ts marks leftovers stopped. (3) File rewind — sessions run with enableFileCheckpointing; the SDK user-message uuid is stored on the user turn (run_turns.message_uuid) and POST /api/runs/:id/rewind {turnId} resumes the live query if needed, dry-runs for the file list (a real rewind returns no counts) and restores the tracked files; the conversation is kept. Claude only. (4) pnpm eval golden-from-reviews --project <id|name> bootstraps a golden set from human-accepted / posted findings (goldenFromReviews in core/eval/golden.ts) — review the labels before trusting scores. Deliberately still out: message priorities, the override editor on the transparency page, Codex profile pools, Electron packaging.
• Agent configuration transparency: one provider at a time (Claude / Codex picker in the first block's header). core/host/config.ts probes Claude Code without running a turn (initializationResult / mcpServerStatus with tool annotations / getContextUsage with per-file, per-tool and per-skill tokens / reloadPlugins / the undeclared getSettings for layer names; 5-minute cache) and marks the disk scan of candidate files with what the CLI reports as loaded (this CLI version never loads AGENTS.md; nested rules count recursively). core/codex/describe.ts reads the live app-server (config/read layers, mcpServerStatus/list with tools, skills/list, hooks/list, plugin/installed, and an ephemeral MCP-less thread/start for instructionSources); the protocol has NO startup-context token figure, so the Codex tab shows instruction-file sizes instead. /api/agent/config (GET/PATCH), app/pages/agent-config.vue; MCP servers, commands and skills render through app/components/CatalogList.vue (grouped, filterable, collapsed by default, first sentence until a row is opened). Never call app/list / plugin/list for the page (multi-MB catalogues).
• Inbox (phase 3, 2026-08): core/inbox/queries.ts ← /api/inbox ← app/pages/inbox.vue lists what waits for the human (pending prompts, drafts with findings, author updates, runs that failed in the last 24 h, automation notes); the sidebar badge polls it every 30 s. Deep links: /projects/:id?pr=<n>&review=<id> opens the PR drawer, useOpenGlobalSession() opens the global chat drawer on a session.
• Smoke-testing a built server: runtimeConfig values are baked at build time, so DB_PATH=… node .output/server/index.mjs silently uses the production data/cockpit.db. Override with Nuxt's runtime names (NUXT_DB_PATH, NUXT_REPOS_DIR, NUXT_WORKTREE_LOCATION=central, NUXT_AUTOMATION_ENABLED=false) on a copy of the DB, and confirm with lsof -p <pid> | grep .db before running anything that talks to an agent or GitHub.
• The default worktree location is .pr-cockpit-worktrees/<taskId> inside each project's local clone; that directory is written into the target repo's .git/info/exclude (local only, not committed) — do not touch the target repo's shared .gitignore. That exclude line does not stop IDEs from discovering those worktrees — editors find repos by scanning the filesystem, not by reading gitignore/exclude; what actually decides discovery is the editor's own scan depth setting (in VS Code, git.repositoryScanMaxDepth, default 1, needs to be ≥2). On startup, recovery moves any still-existing old ./data/worktrees/<taskId> persistent fix/feature worktrees over with git worktree move, and clears paths pointing at directories that are gone. Only WORKTREE_LOCATION=central keeps using REPOS_DIR.