MLOps-Courses/agentops-open-course

MLOps-Courses/agentops-open-course

Skill Claude CodeCodex

Prioritize open incidents deterministically. Use when the engineer asks what to investigate first, requests a queue ranking, or needs an evidence-backed triage summary.

MLOps-Courses/agentops-open-course

Skill Claude CodeCodex

Propose and verify safe, runbook-backed incident remediation. Use when the engineer asks how to fix a known incident or initiate or approve a guarded mock action.

MLOps-Courses/agentops-open-course

Skill Claude CodeCodex

Build offline checks and model-backed evidence for an LLM agent — trajectory scoring, groundedness/citation coverage, a token-cost regression tripwire, and side-by-side prompt A/B. Use when a prompt or model change might silently regress behavior, cost, or grounding, or when "it looked fine" is your only test.

MLOps-Courses/agentops-open-course

Skill Claude CodeCodex

Add layered safety to an LLM agent — PII redaction, prompt-injection spotlighting, human approval on state-changing tools, and a configuration-controlled write kill-switch. Use when an agent can take consequential actions, when tool output could carry injected instructions, or when user data must be redacted before it…

MLOps-Courses/agentops-open-course

Skill Claude CodeCodex

Run the operational loop for an LLM agent that is itself a production workload — detect, triage, mitigate with existing controls, review blamelessly, and convert each incident into a durable regression check. Use when an agent in production burns its error budget, regresses latency or cost, or misbehaves, and you need…

MLOps-Courses/agentops-open-course

Skill Claude CodeCodex

Split an LLM agent into a coordinator plus least-privilege specialists so that a prompt injection cannot reach a tool it was never given, containing blast radius by construction rather than by instruction. Use when one agent holds both untrusted-content readers and consequential write tools, or when you are deciding…

MLOps-Courses/agentops-open-course

Skill Claude CodeCodex

Make an LLM agent's tool and model calls survive flaky and dead dependencies with deadlines, bounded retries, a circuit breaker, and a validated model fallback. Use when an agent hangs on a slow tool, storms a down gateway with retries, or has no failover when its model endpoint is unreachable.

MLOps-Courses/agentops-open-course

Skill Claude CodeCodex

Bound an LLM agent's per-session token spend and attribute cost, so a runaway loop ends with an actionable message instead of an open-ended bill. Use when an agent's multi-step loop can spend without limit, when you need per-session token accounting, or when a delegation chain could multiply model calls.

MLOps-Courses/agentops-open-course

Skill Claude CodeCodex

Index of the AgentOps patterns for operating LLM agents in production — telemetry, guardrails, resilience, evaluation, token budgets, least privilege, and incident response — with a pointer to the full open-source course. Use when you want an overview of how the AgentOps skills fit together, or where to start…

MLOps-Courses/agentops-open-course

Skill Claude CodeCodex

Instrument an LLM agent with OpenTelemetry traces, metrics, and logs so you can see why a turn behaved as it did, with user content kept out of spans by default. Use when an agent is a black box in production, when you need per-turn traces of model and tool calls, or when wiring an agent to MLflow, Prometheus…