SimFoundry AGENTS.md

SimFoundry — agent guide

Orientation for coding agents. This covers what the code does not say out loud: which environment to run in, which trees are safe to edit, and which contracts break silently. For what the project is, read README.md.

What this repo does

SimFoundry turns a short real-world video into a physics-ready OmniGibson scene: segment the objects, reconstruct geometry, generate textured meshes, compile a scene with physical parameters and task proposals. The pipeline runs in three stages under scripts/pipeline/:

Stage	Directory	Produces
A — reconstruction	A_reconstruction/stages/	Depth, segmentation, meshes, s11_sim/scene_objects_info.json, s12_physics/pb_scene_poses.json, an OmniGibson scene
B — augmentation	B_augmentation/stages/	Digital-cousin variants, scene sampling, task proposals
C — application	C_application/stages/	Teleop, demo generation, policy evaluation

Scene editing sits between B and C: a human corrects what the pipeline got wrong — object poses, scales, cameras — before the scene is used for learning or evaluation. Everything downstream inherits those corrections.

Environments

There are many conda envs on a dev box (3dgrut, da3, hunyuan, …); most belong to a single pipeline stage. For general work:

• simfoundry — the main env. OmniGibson, Isaac Sim, torch. Used by the pipeline, the OmniGibson editor, and settle.py. find_settle_python in light_editor/server.py checks that the env's omnigibson resolves inside this repo.
• simfoundry-editor — the light editor's env, no GPU. Created by scripts/installation/install_light_editor.sh; re-run that after a pull. See the trap below.

Trap: omnigibson can resolve outside this checkout. An env may carry this repo's simfoundry package but a different clone's OmniGibson. The editor then runs against the wrong deps/ tree and stamps the wrong OmniGibson version into saved scenes. run_editor.sh warns about this at startup; do not ignore it. Check with python -c 'import omnigibson; print(omnigibson.__file__)'.