tabular-cleanup

Tabular Cleanup Loop

A single agent that takes a messy data dump (<artifact>) to the cleanest defensible state, no human in the loop once running. The objective is a checklist, not a score: the agent infers a data contract, compiles it into deterministic binary checks (each reports a violation count, never a weighted float), then each iteration profiles the table, picks the worst open check, applies one pandas transform to resolve it, and keeps it only if that check's violations strictly drop with no collateral damage. Every accepted transform appends to a replayable pipeline.py; every attempt logs to the ledger. The work decomposes into structure (parse correctly, one tidy table, sane types) → contract synthesis (turn every observed anomaly into a check) → the fix loop. Contract synthesis is where quality is won or lost: an issue the profiler notices but never compiles into a check (classically, many spellings of one category) silently survives — a green checklist over dirty data. Checks read the stored value, so canonicalization is real work the loop must do, not a check-time trick.

When to use

Use this to autonomously clean a messy table to an inferred, confirmed contract where every defect is a deterministic check the loop must drive to zero or to an honest residual. Default to strict contract inference (a lenient contract that lets dirty data go "all green" fast is the primary failure mode); the only human checkpoint is confirming the contract at setup, after which the loop runs to a stop condition. Not for open-ended discovery over an already-clean dataset, diagnosing one known anomaly, or checking an external claim against sources — those are analytical loops; this one rewrites the data to match a contract.

Setup

Resolve bindings interactively. If loop.run.yaml exists in the working dir, load it, confirm the values in one line, and skip to the loop. Otherwise: on Claude Code (the AskUserQuestion tool is available) infer a likely value for each binding and present it as the recommended option; on other hosts ask each as a quoted plain-text prompt. Then write loop.run.yaml (format: examples/run.example.yaml) and confirm the values before creating any other files. The contract (below) is the one decision the user must actively approve — infer it, then get explicit sign-off; everything after is autonomous.