38 tagged llm observability, measured the same way as everything else here.
Browse within: agent-monitoring 30agent-observability 30ai-error-monitor 30ai-monitoring 30ai-observability 30opik 8
Skill Claude CodeCodex
Adding or changing API operations in @repo/operations. One source of truth (defineOperation + a Zod schema) becomes an HTTP endpoint, an OpenAPI operation, an MCP tool, TS + Python SDK methods, a latitude CLI command, and an in-process agent tool — descriptions and contracts must be written with all of these readers…
Skill Claude CodeCodex
Find, triage, and fix production errors captured by Datadog Error Tracking, then open a PR. Use when asked to look at "Datadog issues/incidents/errors", "find and fix bugs from Datadog", investigate the most-frequent or newest production errors, or work a specific Datadog Error Tracking issue.
Skill Claude CodeCodex
When to use: apps/web UI — routes, @repo/ui, TanStack Start server functions and collections, navigation (Link vs useNavigate), forms (useForm with createFormSubmitHandler + fieldErrorsAsStrings when Zod validation errors should appear on fields), Tailwind layout rules, design-system updates, and useEffect /…
Skill Claude CodeCodex
Surface the Opik traces worth a developer's attention, ranked by signal — errors, failed tool calls, latency, regressions, and low online-eval scores — plus Diagnostics issues. Reads live/production traces via the SDK (searchtraces and agentinsights) and works with no MCP; uses the MCP issue entity when connected.…
Skill Claude CodeCodex
Root-cause a specific Opik trace, or a pattern across traces, and return a grounded explanation. Uses the hosted Opik MCP when it is connected, and falls back to SDK scripting otherwise. Returns the root cause, the evidence spans as clickable Opik UI links, and one suggested next step. Use for "why did this trace…
Skill Claude CodeCodex
Add Opik tracing to an existing app and verify a real trace lands. Installs the Opik package, detects the language and LLM framework, adds the minimum tracing, runs a safe representative path, confirms a trace in Opik, and returns the trace link. Use for "instrument my code", "add opik tracing", "add observability"…
Skill Claude CodeCodex
Build an LLM evaluation and run it against your app, returning an experiment with scores. Covers datasets, LLM judges, RAG evaluation, synthetic data, error analysis, and validating evaluators against human labels. Use when the user wants to measure or improve AI product quality, or asks about evals, judges, or…
Skill Claude CodeCodex
Add Opik tracing to an existing app and verify a real trace lands. Installs the Opik package, detects the language and LLM framework, adds the minimum tracing, runs a safe representative path, confirms a trace in Opik, and returns the trace link. Use for "instrument my code", "add opik tracing", "add observability"…
Skill Claude CodeCodex
Reference for the Opik SDK — tracing, span types, framework integrations, threads, and the prompt library (Python, TypeScript, REST). Use for "what span types exist", "how do I flush", "trackopenai", "add OpikTracer", "version a prompt". To instrument a repo end to end, use the opik-instrument skill.