resonate-recursive-fan-out-pattern-python

Resonate Recursive Fan-Out Pattern — Python

Overview

Recursive fan-out is when a durable function spawns child invocations (either of itself or of siblings), optionally waits for them in parallel, and possibly continues recursing. Each child is its own Resonate promise; if a worker crashes, each child resumes independently.

The pattern is expressed in Python by launching multiple ctx.run(...) or ctx.rpc(...) calls before awaiting them — collect the futures first, then await them. This is different from TS's map-over-promises shape only in syntax; the semantics are identical.

When to use

• Batch processing where items are independent
• Web crawling / tree traversal with dynamic depth
• Map-reduce style workflows
• Any fan-out where each leaf is a discrete, retryable unit of work

Don't use for parallel I/O within a single step (use async clients directly inside a ctx.run envelope) or for a tight inner loop (overhead of a promise per item dominates).

Parallel fan-out in the same process

Launch children without blocking; collect futures; await them:

from __future__ import annotations
import asyncio, os, time
from typing import TYPE_CHECKING
from resonate.resonate import Resonate

if TYPE_CHECKING:
    from resonate.context import Context

r = Resonate(url=os.environ.get("RESONATE_URL", "http://localhost:8001"))

async def enrich_batch(ctx: Context, order_ids: list[str]) -> list[dict]:
    # Launch all children (returns futures immediately)
    futures = [ctx.run(enrich_one, oid) for oid in order_ids]

    # Await them all; order preserved
    results = [await f for f in futures]
    return results

async def enrich_one(ctx: Context, order_id: str) -> dict:
    # Enrichment logic — this is a leaf
    return {"order_id": order_id, "enriched": True}

Each enrich_one call is an independent durable promise. If the parent worker crashes after launching children but before awaiting, the children continue; on parent replay, awaiting the future hits the stored promise value.