skill-creator

Skill Creator

A skill for creating new skills and iteratively improving them.

CastFlow catalog

For castflow generate skills, a queue item, or programmer-*-skill in this repo: follow SKILL_ITERATION.md One generation pass only, including its prose language (config or queue-card language; missing means English). A queue card and a named skill are the same pass. Write SKILL.md and ITERATION_GUIDE.md, then stop. Write EXAMPLES.md only when this pass found a feature in a script file. No such feature means do not create that file, and do not write it empty. Write SKILL_MEMORY.md only when this pass found a rule to record, including an implicit rule or a convention. No such rule means do not create that file, and do not write it empty. Stop means do not enter Creating a skill, Description Optimization, or the eval loop. It does not mean stop before a locator or a real rule this pass would write. ITERATION_GUIDE.md does not: it follows One generation pass for when a locator's script file is added, removed, renamed, or the feature moves, and when the yield changes. Do not update it because the implementation inside an existing script changed. Do not invent a product feature as that trigger. Do not leave ITERATION_GUIDE.md empty. Do not invent a script path, a rule, or a duty to fill the other files. Copy the name plus a short description (module name or id, and one NOT). Do not fill a domain template or a README. ignore this bullet later that says to make descriptions pushy — false recall against sibling modules is worse than a miss.

Freeform skills (not CastFlow catalog)

At a high level, the process of creating a skill goes like this:

• Decide what you want the skill to do and roughly how it should do it
• Write a draft of the skill
• Create a few test prompts and run claude-with-access-to-the-skill on them
• Help the user evaluate the results both qualitatively and quantitatively
  • While the runs happen in the background, draft some quantitative evals if there aren't any (if there are some, you can either use as is or modify if you feel something needs to change about them). Then explain them to the user (or if they already existed, explain the ones that already exist)
  • Use the eval-viewer/generate_review.py script to show the user the results for them to look at, and also let them look at the quantitative metrics
• Rewrite the skill based on feedback from the user's evaluation of the results (and also if there are any glaring flaws that become apparent from the quantitative benchmarks)
• Repeat until you're satisfied
• Expand the test set and try again at larger scale