awm-rsi-toolkit

AWM

Agent Workflow Memory, Wang et al., 2024 (arXiv:2409.07429v1, §2.1–2.3 and Appendix A). GitHub: zorazrw/agent-workflow-memory. Verified implementation revision 8c0ff8cd11d648c8fceb99e4e42f37e3b75381b1, chiefly mind2web/offline_induction.py, mind2web/memory.py, mind2web/prompt/, webarena/pipeline.py, webarena/induce_prompt.py and webarena/autoeval/. AWM induces workflows, reusable sub-routines that each pair a goal with a sequence of steps (an observation, the reasoning and the action, with the instance values abstracted away), from successful trajectories, and gives the agent all of them as memory: online, adding the workflows of every execution a success evaluator admits before the next task, or offline, inducing once from a pool of canonical experiences. Run the method the way the paper does; every departure is a declared adaptation reported at the end.

Before you start

This Skill needs run_subagent, so it runs in a top-level Session. If the message only names it, ask which agent to train and on which published Benchmark. Otherwise resolve the inputs below and ask only for a missing Target or Benchmark (or, in offline mode, a missing experience pool); everything else has a default.

• test_agent_id — the Target (the Evaluation Center's Test Agent): an existing agent of this Project, or create: <id> when the user wants a fresh one, created with no plugins by penguin agent create --agent-id <id> --project-id "$PROJECT_ID" --json (an id matches ^[a-z][a-z0-9_-]{1,63}$). If that id exists already, stop and ask; never reinitialize an agent. The Target is never the agent you run as: training rewrites its State while you work.
• benchmark_id — a Benchmark of this Project whose benchmark_config.toml status is published (not a literal draft or failed). Its cases (every directory of it holding a statement/README.md) are the training stream of online_train and the measured case set. Without one, list the published Benchmarks and ask; this Skill never builds one.
• test_benchmark_id (optional) — a second published Benchmark held out from training: H1 (the initialized State) and the final memory are measured on it too, and nothing of it reaches the Judge or the Inducer.
• runs — runs per case in every measurement; default 1.
• The runtime — the provider / model_id pair the user names, else your Environment's, fixed for the whole experiment (references/evaluation.md, controller step 2). The Judge and the Inducer run on your own model; record it.
• mode, fixed before any task evidence is read:
  • online_train (default) — the paper's online loop (run, judge, induce, integrate) over the Benchmark's cases as a training stream; the memory is then frozen and measured.
  • offline — one induction from a supplied pool of canonical experiences, each a task with its observed trajectory: a folder the user names (annotated or model-synthesized), or the Test Sessions of scoreboard runs that scored 100. Record the pool's provenance. Never make a success out of a failed run, and never switch to an online judge without changing the mode.
• criteria, for online_train — autoeval (default), standing in for the source's neural success evaluator: a Judge that sees only the public task, the trajectory and the outputs; or gt, only when the user chooses it: an execution succeeds when it scores 100, never by a tuned partial threshold.