cockpit: Skill for Claude Code

Integration-test a cockpit runtime path against the mock CLI

Unit tests prove a helper returns the right value. Static review proves the diff looks correct. Neither proves the feature actually works end to end. For anything whose value is runtime behaviour — the cockpit agent calling a tool, a session spawning with the right flags, a job posting to the inbox, the permission gate allowing/denying — the only honest evidence is driving the real Claude CLI and watching it happen. This harness does that with a scripted mock Anthropic API so it is deterministic and offline.

When to use

• A behavioural acceptance criterion: "the assistant calls a tool and it succeeds", "asking X returns Y", "a cockpit-agent tool is not denied".
• A spawn-time contract: the CLI receives --system, --mcp-config, a model, a permission mode.
• A regression you can only see at the wire level (what the CLI sent to the API, what came back, what rendered).

If you only need to assert a pure function's output, write a normal tests/*.test.ts vitest unit test instead. Use this harness when the proof requires a running CLI.

The harness (tests/integration/)

startHarness() (in tests/integration/harness.ts) boots three things in isolated tmpdirs: a mock Anthropic API on a random port that replays scripted SSE, a seeded COCKPIT_CONFIG_DIR + CLAUDE_CONFIG_DIR (password, a mock provider pointing the CLI at the mock, default model), and a cockpit server (node dist/server.js) on a random port. The Playwright fixture in tests/integration/fixtures.ts exposes a per-test harness and an already-authenticated page (it injects the cockpit_session cookie), so you navigate straight to authenticated routes.

Write a test

Model on tests/integration/hello.spec.ts. Import the fixture, skip when no CLI is present, script the mock, create a session, drive the UI, assert both the rendered output and what the CLI sent.

import { execSync } from "node:child_process";
import { textResponse, toolUseResponse } from "../mock-api/builder";
import { expect, test } from "./fixtures";

const CLAUDE_BIN = process.env.CLAUDE_BIN ?? "claude";
const CLAUDE_AVAILABLE = (() => {
  try { execSync(`${CLAUDE_BIN} --version`, { stdio: "ignore" }); return true; } catch { return false; }
})();
test.skip(!CLAUDE_AVAILABLE, `claude binary not found at ${CLAUDE_BIN} (set CLAUDE_BIN env)`);

test("the cockpit agent calls a config tool and it is not denied", async ({ page, harness }) => {
  // Script the mock: turn 1 emits a tool_use, turn 2 a final text answer.
  harness.mock.setScript([
    { events: toolUseResponse("list_jobs", {}) },
    { events: textResponse("You have 2 scheduled jobs.") },
  ]);

  const res = await page.request.post(`${harness.cockpitUrl}/api/sessions`, {
    data: { cwd: harness.configDir, cockpitAgent: true, runtime: "pty" },
  });
  const { sessionId } = await res.json();

  await page.goto(`${harness.cockpitUrl}/sessions/${sessionId}?cwd=${encodeURIComponent(harness.configDir)}`);
  const input = page.getByTestId("message-input");
  await expect(input).toBeVisible();
  await page.waitForTimeout(5000); // let the eager PTY spawn settle before sending
  await input.fill("list my jobs");
  await page.getByTestId("btn-send").click();

  await expect(page.getByText("You have 2 scheduled jobs.")).toBeVisible({ timeout: 30_000 });

  // Assert what reached the API: the system prompt and the tool result round-tripped,
  // so the tool was NOT denied. getRequests() returns { url, body } for each call.
  const calls = harness.mock.getRequests().filter((r) => r.url.split("?")[0] === "/v1/messages");
  expect(calls.length).toBeGreaterThanOrEqual(2); // tool turn + follow-up turn
  expect(calls[0].body).toContain("Cockpit Assistant"); // --system prompt arrived
});