agento11y-prod-setup

Agent Observability — production evals & guards setup

The production counterpart to agento11y-test-starter (which runs pre-ship, on code alone, producing an offline test suite). This skill runs after ship, when the agent has real traffic, and sets up the two production surfaces the starter deliberately leaves out:

• Online eval rules — evaluators that score ingested live conversations, so regressions surface without hand-reviewing every conversation.
• Guards (hook-rules) — policies on the request path that warn (and can later be promoted to deny) in real time. A guard decides via one of three shapes: evaluator_ids (an evaluator judges), redact (regex redaction), or tool_filter (block tool calls). See Step 4.

What this skill does that agento11y doesn't

The sibling agento11y skill is the mechanics layer: exact CLI flags, evaluator/rule YAML shapes, create-or-update semantics, the online-eval setup steps. It assumes you already know what to create.

This skill is the judgment layer. It answers which rules and guards this specific agent needs, by grounding in two evidence sources a generic checklist can't use:

1. The agent's code — its system prompt, tools, and how it handles user data. Half the value is here; read and cite it (file:line).
2. The agent's real traffic — because it's deployed, you can see what it actually does in prod, not just what the code says it might.

Two gaps this skill fills beyond agento11y:

• Recommendation from evidence — agento11y starts once you know what to create; this decides.
• Guards — agento11y documents evaluators and rules but not guards (hook-rules), even though gcx agento11y guards exists. This skill carries the guard shapes (evaluator_ids / redact / tool_filter, plus action_on_fail) itself.

For any mechanical detail — exact flags, evaluator/rule YAML fields, the setup flow — defer to the agento11y skill and to gcx agento11y <sub> --help rather than restating it here.