factory-llm-workflows

Factory LLM workflows

Each section leads with Principle (one sentence, stack-agnostic), then Why (constraint → option → tradeoff), then Recipe (the LangGraph / FastAPI / SSE shape we use), and Failure mode when there's one to name. Sections that are pure style with no deeper truth are marked Recipe only.

State shape — TypedDict, not Pydantic

Principle. LangGraph state is a TypedDict, not Pydantic. The state library's merge semantics dictate the shape.

Why. LangGraph merges state between nodes by shallow dict update — the framework expects a dict-like object whose fields are independently updatable. Pydantic validates on construction; every partial update fails validation or requires .model_copy(update=...), which loses the simplicity. TypedDict matches the framework's semantics: it's a dict, fields are optional via total=False, the type annotations are documentation that the type checker enforces at call sites.

Recipe.

from typing import TypedDict
from typing_extensions import NotRequired

class ChatState(TypedDict, total=False):
    """Documented fields. total=False makes everything optional."""
    user_query: str
    intent: NotRequired[str]
    rewritten_query: NotRequired[str]
    retrieved_chunks: NotRequired[list[RetrievedChunk]]
    response: NotRequired[str]
    rag_fallback_attempted: NotRequired[bool]  # one-attempt loop guards

Nested TypedDicts (RetrievedChunk, EvidenceChunk, ClaimVerdict) for complex types.

Failure mode. Reaching for Pydantic state because "Pydantic is more rigorous" — every node update became a .model_copy(update=...) dance, and the graph wiring drowned in validation noise.

Graph composition — node factory closures

Principle. Nodes are produced by factory functions that close over their dependencies; the graph wires the result.

Why. A node that imports its dependencies (LLM client, prompt template, retriever) at module scope is hard to test and impossible to swap. A factory function takes dependencies as parameters and returns a callable; the graph passes the factory the wired-up dependencies. Testing is "construct the node with mocks"; swapping is "construct the node with the alternative."