vmas-simulator-guide

VMAS: Vectorized Multi-Agent Simulator Guide

Overview

VMAS is a vectorized simulator for multi-agent reinforcement learning (MARL) that runs thousands of parallel environments on GPU via PyTorch. It provides a diverse set of 2D cooperative, competitive, and mixed scenarios for benchmarking multi-agent algorithms. Orders of magnitude faster than CPU-based simulators, enabling rapid research iteration on multi-agent coordination problems.

Installation

pip install vmas

Quick Start

import vmas

# Create vectorized environment
env = vmas.make_env(
    scenario="simple_spread",
    num_envs=1024,         # Parallel environments
    num_agents=3,
    device="cuda",         # GPU acceleration
    continuous_actions=True,
)

# Environment loop
obs = env.reset()
for step in range(100):
    # Random actions for demonstration
    actions = [env.action_space[i].sample()
               for i in range(env.n_agents)]

    obs, rewards, dones, infos = env.step(actions)
    # obs: list of [num_envs, obs_dim] tensors
    # rewards: list of [num_envs] tensors

Scenarios

Scenario	Type	Agents	Description
simple_spread	Cooperative	3	Cover N landmarks
simple_tag	Competitive	4	Predator-prey
transport	Cooperative	4	Move package to goal
wheel	Cooperative	4	Coordination on wheel
flocking	Cooperative	5+	Reynolds flocking
discovery	Cooperative	3	Explore and discover
navigation	Mixed	N	Multi-agent navigation

Integration with MARL Libraries

# With TorchRL
from torchrl.envs import VmasEnv

env = VmasEnv(
    scenario="simple_spread",
    num_envs=512,
    device="cuda",
)

# With RLlib
from ray.rllib.env import MultiAgentEnv
# VMAS provides RLlib-compatible wrapper

# With CleanRL / custom training
import torch

env = vmas.make_env("transport", num_envs=2048, device="cuda")
obs = env.reset()

# All tensors on GPU — train directly without CPU transfer
policy_output = policy_network(obs[0])  # Agent 0 observations