estuary-task-stats

flowctl raw stats - Task Processing Statistics

Concepts: Stats show document and byte counts per transaction for a task. They answer "is data actually moving, and how much?" — separate from flowctl catalog status, which only shows control-plane state. A task can show OK status while having zero recent stats (stalled, no new source data, or sync schedule delay).

IMPORTANT: The command is flowctl raw stats, not flowctl stats.

Prerequisites

flowctl must already be authenticated — see the estuary-flowctl-setup skill.

Total Data Over a Period

Materialization — bytes read from source collections ("Data Read" in UI):

flowctl raw stats --task <task> --since 24h | \
  jq -s '{
    total_gb:     ([.[] | select(.materialize != null) | .materialize | to_entries[] | .value.right.bytesTotal // 0] | add // 0 | . / 1073741824 * 10 | round / 10),
    total_docs:   ([.[] | select(.materialize != null) | .materialize | to_entries[] | .value.right.docsTotal  // 0] | add),
    transactions: ([.[] | select(.materialize != null)] | length)
  }'

Capture — bytes written to Estuary collections ("Data Written" in UI):

flowctl raw stats --task <task> --since 24h | \
  jq -s '{
    total_gb:     ([.[] | select(.capture != null) | .capture | to_entries[] | .value.out.bytesTotal // 0] | add // 0 | . / 1073741824 * 10 | round / 10),
    total_docs:   ([.[] | select(.capture != null) | .capture | to_entries[] | .value.out.docsTotal  // 0] | add),
    transactions: ([.[] | select(.capture != null)] | length)
  }'

Hourly Breakdown

Useful for spotting gaps, spikes, or batch patterns. Matches the hourly bar chart in the Estuary UI.

Materialization:

flowctl raw stats --task <task> --since 48h | \
  jq -s '[.[] | select(.materialize != null) | {
    hour:  .ts[0:13],
    bytes: ([.materialize | to_entries[] | .value.right.bytesTotal // 0] | add),
    docs:  ([.materialize | to_entries[] | .value.right.docsTotal  // 0] | add)
  }] | group_by(.hour) | map({
    hour: .[0].hour,
    gb:   (([.[].bytes] | add) / 1073741824 * 10 | round / 10),
    docs: ([.[].docs]  | add)
  })'