event-log-and-reducer

event-log-and-reducer

When N subagents are working in parallel on shared state, the naive approach (each subagent writes to a shared catalog file with a lock) is what KC (a sibling verification harness) learned the hard way is fragile at scale. John uses event log + reducer instead — same shape that React/Redux and event-sourced systems use, ported to filesystem.

The pattern in one diagram

Subagent A ── events/extract/chunks/A-001.json (append-only, A's own file) ─┐
Subagent B ── events/extract/chunks/B-002.json                              │
Subagent C ── events/extract/chunks/C-003.json                              ├──► reducer ──► .john/checkpoints/extract/state.json
...                                                                         │              (canonical)
Subagent N ── events/extract/chunks/N-200.json                             ─┘

• Each subagent invokes emit_event.py; the writer assigns a unique filename and atomic envelope. Zero contention and retries never overwrite history.
• The reducer reads all event files for a phase, in deterministic order, and produces canonical state.
• Running the reducer twice yields the same output (idempotent).
• Canonical state is the read source for the next phase.

Where files live

In the user's project, under <project>/.john/:

• <project>/.john/events/<phase-name>/<work-unit-type>/<subagent-id>-<sequence>.json — events. One file per subagent emission.
• <project>/.john/checkpoints/<phase-name>/state.json — canonical reduced state. Written by reducer.

Conventions:

• Phase, work-unit, agent, and audit-run IDs use letters, digits, _, and -, beginning with a letter or digit.
• Event files are JSON; canonical state is JSON. Markdown if the canonical state is human-facing.
• Producers do not choose filenames. Pipe one JSON object through the shipped atomic writer:

printf '%s\n' '{"event_type":"chunk_complete","chunk_id":"chunk-042"}' | \
  python3 "${CLAUDE_PLUGIN_ROOT}/scripts/emit_event.py" \
    --phase extract --work-unit-id chunk-042 \
    --agent-id extractor-7 --audit-run-id run-20260709