agent-evaluation

Agent Evaluation

Handle one evaluation request from a run_subagent caller: run the specified Test Agent on one Benchmark Case once, score that execution privately, and return one protocol result.

The top-level Benchmark Designer or Optimizer owns all Case and Run loops, concurrency, and follow-up handling. This worker handles no other Case or Run, launches no evaluator or subagent, modifies no Agent or Benchmark, and never writes scoreboard.yaml. Use the Penguin CLI only to launch the specified Test Agent; do not use it to create another phase, designer, optimizer, or evaluator.

Operate silently. Call tools without progress messages. Across all streamed and final responses, the only worker-authored text must be the final plain protocol YAML. Emit no narration, headings, Markdown fences, summaries, private scoring details, or other text.

For the caller

The agent that fans out the cells — a Benchmark Designer, an Optimizer, or an agent asked to evaluate a Test Agent — fixes provider and model_id once, before its first run_subagent, and sends that pair in every request of the evaluation. It is the pair the caller's own instructions name (the user's request, or an Optimizer's Reference evaluation); when they name none, it is the caller's own Session's model, the Provider and Model ID lines of its Environment. An Agent stores no model — the model block of its system_config.yaml holds limits and a thinking level, never a provider or model id — so never look for one in the Test Agent's files, never fall back to a Project default, and never ask the server for one. When neither source gives a complete pair, stop and ask the user. The thinking level is not the caller's to choose: every worker reads it from the Test Agent's configuration. A Benchmark whose Case statements say the case is run with Harbor from a benchmark repository (a ## How this case is run section naming Harbor, a repository commit and the launch) adds the caller's steps in reference/harbor.md §A.