reviewing-dataflow-pipeline

Review a DataFlow Pipeline

Use this skill after a candidate pipeline has produced trial artifacts and before reporting mode=trial_run. This is a release gate, not a substitute for pipeline generation.

Read references/rubric.md completely before starting.

Required evidence

Review the candidate pipeline source, trial input and output, intermediate operator outputs or counters, logs, target benchmark metadata and evaluation code, and at least five official benchmark examples. Build a stratified audit sample of three input records per selected source dataset. Preserve dataset and sample_id provenance in every table and finding.

If the existing trial does not cover three records from every selected dataset, do not extrapolate from it. Extend the trial input and rerun the same candidate pipeline on the missing audit records. Do not run the full dataset.

Delegated review

Dispatch six independent subagents, one for each D1-D6 rubric dimension. Give each reviewer only the dimension it owns plus paths to the raw artifacts and the downstream target. Do not give reviewers a proposed score or another reviewer's conclusions. Run reviewers concurrently when capacity permits.

Each reviewer must return structured JSON containing:

{
  "dimension": "D1",
  "raw_score": 0,
  "weighted_score": 0.0,
  "blocked": false,
  "redlines": [],
  "evidence": [{"claim": "...", "numbers": "...", "sample_ids": ["..."]}],
  "findings": ["..."],
  "required_fixes": ["..."]
}

Every raw score is an integer from 0 through 4. Missing evidence scores zero; reviewers must not assume compliance. Evidence must cite measured counts, rates, distributions, or concrete sample IDs. A statement without either is not evidence.

The main agent must not silently change reviewer scores. It checks arithmetic, deduplicates findings, and applies every redline. Resolve a factual conflict by reading the cited artifact; if it cannot be resolved, use the lower supported score and record the conflict.