aidp-acceptance-contract

aidp-acceptance-contract — convergence verification for batch/streaming jobs

A "no exception" pass is not enough for streaming or backfill pipelines. They might still be processing rows that haven't drained yet. This skill wires up a YAML-driven contract: PASS only after K consecutive empty-pending windows.

When to use

• Source pipeline is structured streaming (the migrator's Pass-2 returns when the trigger completes, but the stream itself hasn't necessarily drained).
• Source pipeline is a slowly-draining batch backfill (each task processes a partition, the workflow PASSes when the partition is processed but a separate retry queue holds the failed rows).
• User asks "wait for it to settle", "convergence", "acceptance test on convergence".

Not needed for plain ETL where success = "the saveAsTable finished".

The pattern

1. Migrator runs Pass-2 to completion.
2. Acceptance contract starts probing periodically (every `sleep_between_s` seconds).
3. Each probe runs `pending_count_sql` and checks `pending = 0`.
4. After `zero_window` consecutive zero-probes → declare PASS.
5. If we hit `max_attempts` without convergence → declare ACCEPTANCE_CONTRACT_VIOLATED.
   Overall migration result is demoted to FAIL.

YAML contract format

Save at reports/<MyJob>_acceptance.yaml:

# Acceptance contract for <task_key>
task_key: "<task_key>"
description: "Wait until the retry queue drains"
pending_count_sql: "SELECT COUNT(*) AS pending FROM <sandbox_schema>.<retry_queue_table> WHERE status = 'PENDING'"
zero_window: 3                # K consecutive zero windows required for PASS
sleep_between_s: 30           # seconds between probes
max_attempts: 60              # ceiling; 60 × 30s = 30-min hard cap

# Optional: override the cluster (default = use job_migrate's cluster)
cluster_id: null

# Optional: notify-on-violation hook (called once if max_attempts hit before convergence)
on_violation_log_path: "/tmp/<MyJob>_acceptance.log"

Wire into a migration