csv-wave-pipeline

Auto Mode

When --yes or -y: Auto-confirm task decomposition, skip interactive validation, use defaults.

CSV Wave Pipeline

Usage

$csv-wave-pipeline "Implement user authentication with OAuth, JWT, and 2FA"
$csv-wave-pipeline -c 4 "Refactor payment module with Stripe and PayPal"
$csv-wave-pipeline -y "Build notification system with email and SMS"
$csv-wave-pipeline --continue "auth-20260228"

Flags:

• -y, --yes: Skip all confirmations (auto mode)
• -c, --concurrency N: Max concurrent agents within each wave (default: 4)
• --continue: Resume existing session

---

Overview

Wave-based batch execution using spawn_agents_on_csv with cross-wave context propagation. Tasks are grouped into dependency waves; each wave executes concurrently, and its results feed into the next wave.

Core workflow: Decompose → Compute Waves → Execute Wave-by-Wave → Aggregate

Phase 1: Requirement → CSV
   ├─ Parse requirement into subtasks (3-10 tasks)
   ├─ Identify dependencies (deps column)
   ├─ Compute dependency waves (topological sort → depth grouping)
   ├─ Generate tasks.csv with wave column
   └─ User validates task breakdown (skip if -y)

Phase 2: Wave Execution Engine
   ├─ For each wave (1..N):
   │   ├─ Build wave CSV (filter rows for this wave)
   │   ├─ Inject previous wave findings into prev_context column
   │   ├─ spawn_agents_on_csv(wave CSV)
   │   ├─ Collect results, merge into master tasks.csv
   │   └─ Check: any failed? → skip dependents or retry
   └─ discoveries.ndjson shared across all waves (append-only)

Phase 3: Results Aggregation
   ├─ Export final results.csv
   ├─ Generate context.md with all findings
   ├─ Display summary: completed/failed/skipped per wave
   └─ Offer: view results | retry failed | done

Context Propagation

Two context channels flow across waves:

1. CSV findings (structured): context_from column → prev_context injection — task-specific directed context
2. NDJSON discoveries (broadcast): discoveries.ndjson — general exploration findings available to all