Plugin Claude Code
Orq plugins for Claude Code — tracing, MCP tools, and agent skills.
6 5d ago A
tokens not measured
original MIT
orq
02Plugin Claude Code
Agent skills for building, deploying, evaluating, and monitoring LLM pipelines on the orq.ai platform.
6 5d ago A
tokens not measured
original MIT
Instructions file
Instructions for orq-ai/assistant-plugins, covering orq-skills — maintainer notes, versioning, when to bump, how to bump and plugin manifest rules.
6 5d ago A 2,554 tokens
original MIT
Agent
orq.ai workspace assistant — routes to skills and commands for building, evaluating, and monitoring LLM pipelines.
6 5d ago A 26 tokens
original MIT
analytics
05Command
Show workspace analytics — requests, cost, tokens, errors, top models, and drill-down trends.
6 5d ago A 21 tokens
original MIT
models
06Command
List available AI models and their capabilities.
6 5d ago A 9 tokens
original MIT
Command
Manage orq.ai Skills — list, get, create, update, retire (tag as retired), or delete Skills (the platform entity, formerly Snippets) and find the prompts/agents that reference them.
6 5d ago A 48 tokens
original MIT
quickstart
08Command
Interactive onboarding guide — set up credentials, connect to orq.ai, and learn every command and skill.
6 5d ago A 23 tokens
original MIT
traces
09Command
Query and summarize traces with filters — debugging entry point before orq-analyze-trace-failures.
6 5d ago A 23 tokens
original MIT
workspace
10Command
Show a workspace overview — agents, deployments, prompts, datasets, experiments, projects, and knowledge bases.
6 5d ago A 22 tokens
original MIT
orq-trace
11Plugin Claude Code
Automatically trace Claude Code sessions to Orq. Captures sessions, turns, tool calls, and LLM responses as hierarchical OTLP spans.
6 5d ago A
tokens not measured
original MIT
evaluatorq
12Skill Claude CodeCodex
Write and run evaluatorq evaluation scripts (Python or TypeScript) for a single agent or deployment — custom scorers, built-in evaluators, and dataset-driven evaluation. For CLI workflows, use the companion skills: orq-red-team for eq redteam adversarial testing and orq-simulate-agent for eq sim multi-turn user…
6 5d ago A 117 tokens
original MIT
Skill Claude CodeCodex
Read production traces, identify what's failing, and build failure taxonomies using open coding and axial coding methodology. Use when debugging agent or pipeline quality, investigating "why are my outputs bad?", or before building any evaluator — error analysis must come first. Do NOT use when you already have…
6 5d ago A 94 tokens
original MIT
Skill Claude CodeCodex
Design, create, and configure orq.ai Agents with tools, instructions, knowledge bases, and memory stores. Use when building new agents, attaching KBs or memory, writing system instructions, selecting models, or setting up RAG pipelines. Do NOT use for debugging existing agents (use orq-analyze-trace-failures) or…
6 5d ago A 86 tokens
original MIT
Skill Claude CodeCodex
Create validated LLM-as-a-Judge evaluators following best practices — binary Pass/Fail judges with TPR/TNR validation for measuring specific failure modes. Use when you need to automate quality checks, build guardrails, or measure a specific failure mode identified during trace analysis. Do NOT use when failures are…
6 5d ago A 100 tokens
original MIT
orq-cli
16Skill Claude CodeCodex
Drive the orq command-line interface — check the install, authenticate, select a workspace, and run read and write commands against any orq.ai resource (traces, agents, deployments, evals, prompts, datasets, projects, skills). Use when a task needs shell access to orq.ai, when a script or CI job must read workspace…
6 5d ago C 125 tokens
original MIT
Skill Claude CodeCodex
Run cross-framework agent comparisons using evaluatorq from orqkit — compares any combination of agents (orq.ai, LangGraph, CrewAI, OpenAI Agents SDK, Vercel AI SDK) head-to-head on the same dataset with LLM-as-a-judge scoring. Use when comparing agents, benchmarking, or wanting side-by-side evaluation. Do NOT use…
6 5d ago A 100 tokens
original MIT
Skill Claude CodeCodex
Align, calibrate, or improve an existing LLM-as-a-judge (orq evaluator) so its verdicts match human judgment — boolean, categorical, or numeric judges. Use when the user wants to "align my evaluator", "improve my eval", "my judge keeps changing its mind", "find ambiguous cases", or "annotate an evaluator" — i.e. they…
6 5d ago A 207 tokens
original MIT
Skill Claude CodeCodex
Generate and curate evaluation datasets — structured generation via dimensions-tuples-NL, quick from description, expansion from existing data, plus dataset maintenance through deduplication, rebalancing, and gap-filling. Use when creating eval data, expanding test coverage, or cleaning datasets. Do NOT use when…
6 5d ago A 100 tokens
original MIT
Skill Claude CodeCodex
Invoke orq.ai deployments, agents, and models via the Python SDK or HTTP API. Use when a user wants to call a deployment with prompt variables, invoke an agent in a conversation, or call a model directly through the AI Router. Do NOT use for creating or editing deployments/agents (use orq-optimize-prompt or…
6 5d ago A 93 tokens
original MIT
Skill Claude CodeCodex
Manage orq.ai Skills (the platform entity, formerly called Snippets) end-to-end — list, get, create, update, and delete Skills, plus authoring guidance (display name, description, tags, project scoping, path placement), and how Skills get consumed (the {{skill. }} template placeholder inside prompts and agent…
6 5d ago A 103 tokens
original MIT
Skill Claude CodeCodex
Analyze and optimize system prompts using a structured prompting guidelines framework — AI-powered analysis and rewriting. Use when a prompt needs improvement, experiment results show quality gaps, or you want a structured review of an existing system prompt. Do NOT use when production traces show failures (use…
6 5d ago A 89 tokens
original MIT
orq-red-team
23Skill Claude CodeCodex
Invoke the evaluatorq red teaming CLI to run adversarial attacks against deployed agents or static datasets. Use when asked to "run a red team", "red team this deployment", "test my agent for vulnerabilities", "OWASP red team", or "check how my agent handles adversarial inputs". Do NOT use when you only need to build…
6 5d ago A 100 tokens
original MIT
Skill Claude CodeCodex
Create and run orq.ai experiments — compare configurations against datasets using evaluators, analyze results, and generate prioritized action plans. Use when evaluating LLM agents, deployments, conversations, or RAG pipelines end-to-end. Do NOT use without a dataset and evaluators. Do NOT use for cross-framework…
6 5d ago A 79 tokens
original MIT