shinka-run

Shinka Run CLI Skill

Run a batch of program mutations using ShinkaEvolve's CLI interface.

When to Use

Use this skill when:

• evaluate.py and initial.<ext> already exist
• The user wants to run code evolution using the ShinkaEvolve/Shinka library
• You want configurable program evolution runs using explicit CLI args

Do not use this skill when:

• You need to scaffold a new task from scratch (use shinka-setup)

What is ShinkaEvolve?

A framework developed by SakanaAI that combines LLMs with evolutionary algorithms to propose program mutations, that are then evaluated and archived. The goal is to optimize for performance and discover novel scientific insights.

Repo and documentation: https://github.com/SakanaAI/ShinkaEvolve Paper: https://arxiv.org/abs/2212.04180

Workflow

1. Inspect task directory

ls -la <task_dir>

Confirm evaluate.py and initial.<ext> exist.

1. Inspect CLI reference quickly

shinka_run --help

1. Check model availability before proposing a run

shinka_models
shinka_models --verbose

Validate the exact run config against shinka_models:

• Mutation models: every entry in evo.llm_models must appear in the llm list.
• Meta recommendation models: if evo.meta_rec_interval is set and evo.meta_llm_models is set, every meta model must appear in the llm list.
• Prompt evolution models: if evo.evolve_prompts=true, use evo.prompt_llm_models when provided, otherwise evo.llm_models; every selected model must appear in the llm list.
• Embedding model: if evo.embedding_model is set, it must appear in the embedding list.
• Local OpenAI-compatible models are allowed for LLMs and embeddings via local/<model>@http(s)://host[:port]/v1, and these local models are not expected to appear in shinka_models.

Important runtime rules:

• Do not assume meta recommendations fall back to evo.llm_models. In the current runner, meta recommendations are only enabled when evo.meta_llm_models is explicitly set.
• Prompt evolution does fall back to evo.llm_models when evo.prompt_llm_models is unset.
• Treat local/<model>@http(s)://host[:port]/v1 values as an explicit exception to the shinka_models membership check. Instead, confirm the local endpoint URL and serving status separately before running.
• If any required model is missing from shinka_models, stop and ask the user to either change the config or set the missing credentials first.