posthog-foss: Skill for Claude Code

Auditing warehouse source endpoint coverage

Finds endpoint gaps in sources that already work: the vendor exposes an object users want, and we never added a table for it.

This is the reverse of implementing a new source. Nothing here is about the ~646 scaffolded stubs.

Output goes in products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS.md. Read it first: it records what a previous audit already found, so you extend it rather than rediscover it.

Why this needs a method

There are ~586 implemented sources. You cannot diff all of them against vendor docs in one pass, and a flat list of "source X has N tables" tells you nothing, because N should be 3 for some vendors and 40 for others.

So the audit is: establish our real inventory, rank by who actually uses it, diff the top of that ranking by hand, and sweep the long tail with a batched workflow. Depth over breadth on the sources that matter; breadth via fan-out for the rest. Ten spec-verified sources beat 200 guesses.

Both halves have been run once. COVERAGE_GAPS.md holds the hand-audited high-adoption sources and COVERAGE_GAPS_APPENDIX.md holds the swept remainder, so a re-run is a refresh, not a cold start.

Sweeping the long tail with a workflow

The tail is too big to audit inline but parallelizes perfectly, since sources are independent. What worked: one parallel() fan-out, batches of 8 sources per agent, 69 agents for 547 sources. It cost about 6.8M subagent tokens and 3,000 tool calls, and returned 4,540 findings with zero agent errors, so budget accordingly before starting. Requires explicit user opt-in to run a workflow at that scale.

Design notes that mattered:

• Pass the payload by file, not inline. Write [{s, l, t, d}] (source type, label, current tables, known docs URLs) to a scratchpad JSON, give each agent an index range, and have it read its own slice. Inlining 136KB of source data into 69 prompts is pure waste.
• Give agents our table list up front. It comes from get_schemas() and is authoritative, so agents spend their budget on vendor research instead of re-reading our code.
• Force structured output with a schema, including a verified boolean and the exact doc_url diffed against. That URL is what makes the result auditable afterwards.
• Make "could not verify" an explicit, blessed answer. Tell agents plainly that a fabricated endpoint wastes an implementer's day and is worse than reporting nothing. On the first run this held perfectly: all 542 gaps/thin/adequate results were verified: true and only the 5 genuine failures came back could-not-verify.