aamas-experiments

AAMAS Experiments

Use this before submission when the empirical or simulation story is not yet locked. At AAMAS the experiment exists to test the interaction claim, not to top a benchmark.

Experiment audit

• Map each empirical claim to a game, a self-play run, a population sweep, an ablation, or a deviation test.
• Choose opponents deliberately: self-play alone rarely suffices; include held-out opponents, population sets, or classical strategies as the claim requires.
• Separate simulations that validate a solution concept (where the equilibrium is known) from real or applied studies that show practical multiagent behavior.
• Report uncertainty for stochastic results over both seeds and opponents: standard errors, confidence intervals, or paired tests.
• Report the environment, number of agents, training regime, evaluation protocol, metrics, hyperparameter ranges, chosen settings, seeds, hardware, software versions, and runtime.
• Add ablations for the interaction mechanism (communication, reward sharing, the payment rule), not just cosmetic variants.
• Audit for the mismatch between the strategic claim and the setup: an equilibrium claim tested against only one fixed opponent, or a cooperation claim that hides a reward-shaping constant.

What experiments are for at this venue

• The strongest design shows the interaction under stress: agents that can deviate, opponents the method did not train against, and populations that vary in size or composition.
• One experiment that lets agents try to exploit the mechanism and fails to profit is worth more than five extra environments where nothing strategic is tested.
• Reviewers, often game theorists, check whether the metric matches the claim: convergence to a named solution concept, exploitability, social welfare, or regret - not just episodic return.

Interaction-validation design table