microduck_rl AGENTS.md

AGENTS.md

RL training environments for Microduck — a ~800 g, ~25 cm tall bipedal robot with 14 Dynamixel XL330 servos — built on mjlab (MuJoCo Warp) with PPO (rsl_rl). Policies are trained here at 50 Hz, exported to ONNX, and deployed by the runtime in the pollen-robotics/microduck repo on the real robot. Sim2real transfer is the whole point: every convention below exists because breaking it produced a policy that worked in the viewer and failed on hardware.

Commands

uv run list-envs                                    # live task registry
uv run train <TASK_ID> --env.scene.num-envs 4096    # train (add --hf-jobs for Hugging Face Jobs)
uv run train <TASK_ID> --env.scene.num-envs 64 --agent.max_iterations 5   # SMOKE TEST — always run first
uv run play <TASK_ID> --wandb-run-path <entity/project/run_id>
uv run scripts/export.py <TASK_ID> --wandb-run-path <...>   # → ONNX (bakes obs normalizer — mandatory path)
uv run scripts/infer_policy.py --walking out.onnx   # CPU MuJoCo deployment rehearsal
uv run --with pytest pytest tests/

A 5-iteration smoke test at 64 envs catches ~95% of config errors for cents. Never launch a long run without one.

Repo map

• src/mjlab_microduck/tasks/mdp.py — ALL custom MDP functions (rewards, events, observations, commands, curricula). Add new functions here, grouped by task.
• src/mjlab_microduck/tasks/microduck_*_env_cfg.py — one cfg module per task family. microduck_velocity_env_cfg.py is the main walking recipe AND the shared base (robot, DR, obs, commands) other envs build on or mirror.
• src/mjlab_microduck/tasks/__init__.py — task registration (base + -Backlash- variants).
• src/mjlab_microduck/tasks/backlash.py — wraps any env cfg into its backlash twin.
• src/mjlab_microduck/robot/microduck_constants.py — robot cfgs, HOME frame, BAM actuator cfg.
• src/mjlab_microduck/robot/microduck/ — MJCF exports from Onshape (onshape-to-robot, one config_mjcf_*.json per model) + scenes + add_backlash.py.
• src/mjlab_microduck/actuator/friction_dr_bam.py — BAM actuator + friction DR + backlash encoder.
• scripts/ — export, infer, sim2real comparison, wandb helpers.
• tests/ — cfg-invariant and mdp-function regression tests (CPU, no GPU needed).