147 tagged ai observability, measured the same way as everything else here.
Browse within: ai-monitoring 76aiengineering 39datasets 39Evaluation 30agent-monitoring 30agent-observability 30ai-error-monitor 30llm-observability 30awesome 26awesome-list 26genai 26Cost Optimization 20agentic-devops 20ai-credits 20
Skill Claude CodeCodex
Debug LLM applications using the Phoenix CLI. Fetch traces, analyze errors, structure trace review with open coding and axial coding, inspect datasets, review experiments, query annotation configs, and use the GraphQL API. Use whenever the user is analyzing traces or spans, investigating LLM/agent failures, deciding…
Skill Claude CodeCodex
Manage GitHub issues, labels, project boards, sprint operations, and roadmap health for the Arize-ai/phoenix repository. Use when filing roadmap issues, triaging bugs, applying labels, running sprint close-out and rollover, auditing board hygiene, checking ticket-load balance across the team, keeping roadmap epics up…
Skill Claude CodeCodex
Generate synthetic evaluation datasets for the PXI eval harness (evals/pxi/). Use whenever the user asks to create, author, draft, expand, or audit an eval dataset for a PXI tool, skill, or behavior — including phrases like "write evals for ", "test PXI behavior", "synthetic dataset for PXI", "cover this tool with…
Skill Claude CodeCodex
Adding or changing API operations in @repo/operations. One source of truth (defineOperation + a Zod schema) becomes an HTTP endpoint, an OpenAPI operation, an MCP tool, TS + Python SDK methods, a latitude CLI command, and an in-process agent tool — descriptions and contracts must be written with all of these readers…
Skill Claude CodeCodex
Find, triage, and fix production errors captured by Datadog Error Tracking, then open a PR. Use when asked to look at "Datadog issues/incidents/errors", "find and fix bugs from Datadog", investigate the most-frequent or newest production errors, or work a specific Datadog Error Tracking issue.
Skill Claude CodeCodex
When to use: apps/web UI — routes, @repo/ui, TanStack Start server functions and collections, navigation (Link vs useNavigate), forms (useForm with createFormSubmitHandler + fieldErrorsAsStrings when Zod validation errors should appear on fields), Tailwind layout rules, design-system updates, and useEffect /…
Skill Claude CodeCodex
Configure feedback functions and selectors for TruLens evaluations.
Skill Claude CodeCodex
Instrument LLM apps with TruLens OTEL-based tracing - from setup to debugging and optimization.
Skill Claude CodeCodex
Creates, runs, and analyzes Arize experiments for evaluating and comparing model performance. Covers experiment CRUD, exporting runs, comparing results, and evaluation workflows using the ax CLI. Use when the user mentions create experiment, run experiment, compare models, model performance, evaluate AI, experiment…
Skill Claude CodeCodex
INVOKE THIS SKILL for Arize Prompt Hub and ax prompts workflows: author or import templates and save (Workflows A–B), label/promote (C), or list/get/edit/delete/duplicate (D). Use when the user mentions ax prompts, Prompt Hub, creating/editing/saving a prompt, {variable} placeholders, or production/staging labels. For…
Skill Claude CodeCodex
Downloads, exports, and inspects existing Arize traces and spans to understand what an LLM app is doing or debug runtime issues. Covers exporting traces by ID, spans by ID, sessions by ID, and root-cause investigation using the ax CLI. Use when the user wants to look at existing trace data, see what their LLM app is…
ContextJet-ai/awesome-llm-observability
Skill Claude CodeCodex
Use this when adding evaluation to an LLM/agent app - measuring output quality (correctness, faithfulness, relevance, safety) rather than just watching traces. Trigger on "add evals", "test my prompt", "is my RAG accurate", "catch regressions", "score outputs", or setting up an eval suite in CI. Covers offline (CI)…
ContextJet-ai/awesome-llm-observability
Skill Claude CodeCodex
Use this to make an LLM app resilient to provider failures, rate limits, timeouts, and outages. Trigger on "handle LLM API errors", "add retries/fallbacks", "the app breaks when OpenAI is down", "rate limit errors", "make my LLM calls reliable", "timeout handling". Add retries, timeouts, and model/provider fallbacks…
ContextJet-ai/awesome-llm-observability
Skill Claude CodeCodex
Use this to add safety and security guardrails to an LLM/agent app - blocking prompt injection, PII leakage, jailbreaks, toxic output, off-topic responses, or invalid structured output. Trigger on "add guardrails", "prevent prompt injection", "stop PII leaks", "validate the model's output", "make this safe for…
paulasilvatech/frontier-cockpit
Skill Claude CodeCodex
Creates professional Markdown documents with YAML frontmatter, versioning, author attribution, table of contents, and consistent formatting. Use for Markdown documents, README, ADR, spec, guide, changelog, runbook, RFC, technical docs, and converting PPTX or PowerPoint files to narrative Markdown reading editions with…
paulasilvatech/frontier-cockpit
Skill Claude CodeCodex
Guide for creating high-quality MCP (Model Context Protocol) servers that enable LLMs to interact with external services through well-designed tools. Use when building MCP servers to integrate external APIs or services, whether in Python (FastMCP) or Node/TypeScript (MCP SDK).
paulasilvatech/frontier-cockpit
Skill Claude CodeCodex
Create, audit, or consolidate agent skills following the Agent Skills open standard (agentskills.io). Interviews the user relentlessly about intent, scope, and edge cases before drafting. Covers SKILL.md structure, frontmatter, progressive disclosure, description optimization, script bundling, sub-command…
Skill Claude CodeCodex
Deep-dive into backtracking patterns from distilled clens sessions.
Skill Claude CodeCodex
Analyze cross-session development lifecycle and generate a journey report.
Skill Claude CodeCodex
Schema knowledge and interpretation guide for clens distilled session data. Auto-loads when reading .clens/distilled/ files.
Skill Claude CodeCodex
Deep-dive into backtracking patterns from distilled clens sessions.
Skill Claude CodeCodex
Analyze cross-session development lifecycle and generate a journey report.
Skill Claude CodeCodex
Compare two distilled clens sessions side-by-side to identify performance differences.