karimo-overview: Command for Claude Code

Run an automated test suite against the BOS MCP server. This validates environment, authentication, all 10 tools, the resource endpoint, asset proxy, and error sanitization.

Setup

1. Determine the base URL:

   • If the dev server is running: http://localhost:3000
   • Otherwise use NEXT_PUBLIC_APP_URL or https://bos-3-0.vercel.app
   • Verify the base URL is reachable with a quick curl -s -o /dev/null -w "%{http_code}" {BASE_URL}

2. Get the MCP API key:

   • Check if MCP_TEST_API_KEY env var is set
   • If not, ask the user for their MCP API key (generated in BOS settings)

3. Initialize results tracking:

   • Track pass/fail/skip counts
   • Store failure details for summary

Test Execution

Run tests sequentially. For each test, report the result inline: PASS, FAIL (with reason), or SKIP (with reason).

Phase 1: Environment Check

Check that required env vars are set (read their existence, not their values):

# Check each var exists (non-empty)
[ -n "$NEXT_PUBLIC_SUPABASE_URL" ] && echo "PASS: NEXT_PUBLIC_SUPABASE_URL" || echo "FAIL: NEXT_PUBLIC_SUPABASE_URL not set"
[ -n "$SUPABASE_SERVICE_ROLE_KEY" ] && echo "PASS: SUPABASE_SERVICE_ROLE_KEY" || echo "FAIL: SUPABASE_SERVICE_ROLE_KEY not set"
[ -n "$OPENAI_API_KEY" ] && echo "PASS: OPENAI_API_KEY" || echo "FAIL: OPENAI_API_KEY not set"

Phase 2: Auth Flow

Test authentication against the MCP endpoint:

# A1: No token - expect 401
curl -s -o /dev/null -w "%{http_code}" -X POST {BASE_URL}/api/mcp \
  -H "Content-Type: application/json" \
  -d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-03-26","capabilities":{},"clientInfo":{"name":"test","version":"1.0"}}}'

# A2: Invalid token - expect 401
curl -s -o /dev/null -w "%{http_code}" -X POST {BASE_URL}/api/mcp \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer fake_invalid_token_xyz" \
  -d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-03-26","capabilities":{},"clientInfo":{"name":"test","version":"1.0"}}}'

# A3: Valid token - expect 200
curl -s -o /dev/null -w "%{http_code}" -X POST {BASE_URL}/api/mcp \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer {API_KEY}" \
  -d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-03-26","capabilities":{},"clientInfo":{"name":"test","version":"1.0"}}}'

# A5: OAuth metadata
curl -s -o /dev/null -w "%{http_code}" {BASE_URL}/.well-known/oauth-authorization-server