skill-creator

Skill Creator

A skill for creating new skills and iteratively improving them.

At a high level, the process of creating a skill goes like this:

• Decide what you want the skill to do and roughly how it should do it
• Write a draft of the skill
• Create a few test prompts and run claude-with-access-to-the-skill on them
• Help the user evaluate the results both qualitatively and quantitatively
  • While the runs happen in the background, draft some quantitative evals if there aren't any. Then explain them to the user
  • Use the eval-viewer/generate_review.py script to show the user the results, and also let them look at the quantitative metrics
• Rewrite the skill based on feedback from the user's evaluation
• Repeat until satisfied
• Expand the test set and try again at larger scale

Your job is to figure out where the user is in this process and help them progress. Maybe they want to make a skill from scratch, or maybe they already have a draft and want to iterate.

Be flexible -- if the user says "I don't need to run a bunch of evaluations, just vibe with me", do that instead.

After the skill is done (order is flexible), run the skill description improver to optimize triggering.

Communicating with the user

Pay attention to context cues to understand how to phrase communication. In the default case:

• "evaluation" and "benchmark" are borderline, but OK
• for "JSON" and "assertion" you want to see cues from the user that they know what those things are before using them without explaining

It's OK to briefly explain terms if you're in doubt.

---

Creating a skill

Capture Intent

Start by understanding the user's intent. The current conversation might already contain a workflow the user wants to capture (e.g., they say "turn this into a skill"). If so, extract answers from the conversation history first -- the tools used, the sequence of steps, corrections the user made, input/output formats observed. The user may need to fill gaps, and should confirm before proceeding.