skills-arena: Instructions file for Claude Code

Skills Arena - Claude Code Instructions

Project Overview

Skills Arena is an SDK for benchmarking and optimizing AI agent skill descriptions - essentially "SEO for agent skills." It answers the question: "When an agent has multiple skills available, will it pick yours?"

Quick Reference

from skills_arena import Arena, Config

# Evaluate a single skill
results = Arena().evaluate("./skill.md", task="web search")

# Compare skills head-to-head
results = Arena().compare(["./a.md", "./b.md"], task="web search")

# Battle royale with ELO rankings
results = Arena().battle_royale(skills, task="web search")

Architecture

Arena (entry point)
├── Parser      → Parses skill files (.md, JSON, etc.)
├── Generator   → Creates test scenarios via LLM
├── Runner      → Executes scenarios against agent frameworks
└── Scorer      → Calculates metrics (selection rate, ELO)

Key Patterns

1. Pydantic Models Everywhere

All data structures use Pydantic BaseModel for validation:

from pydantic import BaseModel, Field

class Skill(BaseModel):
    name: str
    description: str = ""

2. Async-First with Sync Wrappers

Core methods are async, with sync wrappers for convenience:

async def evaluate_async(self, ...) -> EvaluationResult:
    ...

def evaluate(self, ...) -> EvaluationResult:
    return asyncio.run(self.evaluate_async(...))

3. Abstract Base Classes for Extensibility

Each component has a base class for new implementations:

• BaseParser → ClaudeCodeParser, OpenAIParser, MCPParser
• BaseGenerator → LLMGenerator, MockGenerator
• BaseAgent → ClaudeCodeAgent, MockAgent

4. Mock Classes for Testing

Every component has a mock for testing without API calls:

# Use MockGenerator instead of LLMGenerator
# Use MockAgent instead of ClaudeCodeAgent

Directory Structure

src/skills_arena/
├── __init__.py      # Public exports
├── arena.py         # Main Arena class
├── config.py        # Config dataclass
├── models.py        # Core data models
├── exceptions.py    # Custom exceptions
├── parser/          # Skill format parsers
├── generator/       # Scenario generation
├── runner/          # Agent runners
├── scorer/          # Metrics & ELO
├── insights/        # AI analysis (future)
└── reporter/        # Output formatting (future)