middleware

Add Middleware to a Railtracks Agent or Node

The user wants to add middleware to a railtracks node/agent: $ARGUMENTS

Quick reference

• All five accept optional keyword-only name= and work bare or called: @rt.pre_llm, @rt.post_node(name="print_after").
• wrap_node/wrap_llm require async def (TypeError otherwise). pre_llm/post_llm/post_node accept sync or async.
• middleware= wraps the whole node call (an agent's full tool-calling loop, including every tool node it invokes). model_middleware= wraps one raw LLM call inside that loop.
• Attach at creation — rt.agent_node(..., middleware=[...], model_middleware=[...]) — or after the fact without mutating the original — rt.couple(node, middleware=[...], model_middleware=[...]).
• List order = wrapping order: middleware=[A, B, C] → A wraps B wraps C wraps the node. rt.couple-ing more onto an already-middlewared node adds the new ones innermost; existing middleware stays outermost.

Best practices

1. Exception handling & retries

• pre_llm, post_llm, post_node only run on success — an exception skips them entirely. If you need to react to failure (cleanup, translation, fallback), use wrap_node/wrap_llm and put your own try/except around await call(...).
• Where you put a retry changes what it re-executes:
  • model_middleware= retry only re-issues the raw LLM call — safe by default, no side effects beyond the model provider request.
  • middleware= retry on an agent re-runs the entire node, including every tool call that already succeeded in that attempt. Only retry at this slot if the tools involved are safe to re-run (idempotent, or guarded by an idempotency key) — otherwise a transient failure after a non-idempotent tool call (e.g. "send email", "charge card") causes it to fire twice.
  • Default to retrying in model_middleware= for provider flakiness (rate limits, timeouts). Reach for middleware= retry only when you've confirmed the tools are safe to repeat.
• Prefer railtracks.prebuilt.middleware.Retry(max_tries=..., approach=..., retry_on=...) over hand-rolling — it raises once attempts are exhausted rather than swallowing the failure, and lets you scope retry_on to specific exception types instead of catching everything.
• Don't swallow exceptions silently inside a wrap_node/wrap_llm middleware (except Exception: return default) — that hides the failure from callers and from anything else in the chain. If you must return a fallback, do it deliberately (log it, tag it in the result) rather than as an unannounced side effect.