skill-evolver

Skill Evolver

Analyze skill execution traces to discover issues, identify improvement opportunities, and apply fixes to skill files.

Trace Format

Traces are JSON with this structure:

{
  "id": "uuid",
  "request": "user's original request",
  "skills_used": ["skill-name"],
  "success": true/false,
  "total_turns": 2,
  "total_input_tokens": 5000,
  "total_output_tokens": 200,
  "duration_ms": 7000,
  "steps": [
    {"role": "assistant", "content": "...", "tool_name": null},
    {"role": "tool", "tool_name": "...", "tool_input": {}, "tool_result": "..."}
  ],
  "llm_calls": [
    {"turn": 1, "stop_reason": "tool_use", "input_tokens": 2500, "output_tokens": 50}
  ]
}

Workflow

This skill can receive two types of input (at least one required):

• Traces: Execution trace data from real skill runs — provides data-driven problem discovery
• Feedback: User-written improvement suggestions — provides directed guidance for changes

When both are provided, combine insights: use traces to validate/discover issues and feedback to prioritize and guide fixes.

Step 1: Analyze Inputs

If traces are provided, run the analysis script:

scripts/analyze_traces.py <traces.json> [--skill <name>] [--format json|text]

Output includes:

• Success rate
• Average turns, duration, tokens
• Common issues and warnings
• Recommendations

If feedback is provided, identify the user's improvement goals and map them to actionable changes.

If both are provided, cross-reference: does the feedback align with trace-discovered issues? Use feedback to prioritize which trace-identified problems to fix first.

Step 2: Extract Issue Details

For failed or problematic traces, extract full context:

scripts/extract_issue_context.py <traces.json> --failed
scripts/extract_issue_context.py <traces.json> --trace-id <id> --show-llm
scripts/extract_issue_context.py <traces.json> --high-turns

Skip this step if only feedback was provided (no traces).