worldforge AGENTS.md

Agent Guide

Project Identity

WorldForge is a harness framework for building world-model-based workflows for physical AI. It is the application builder's counterpart to model-training stacks like Stable World Model: it helps roboticists and physical-AI builders compose, evaluate, and benchmark workflows built on top of world models, not train them. It is a library and CLI, not a hosted service, on-chain contract, or end-user application.

The whole framework is organized around one backbone loop: planning and scoring action candidates with an action-conditioned predictive world model, in latent space. A policy provider proposes candidate actions, a predict provider rolls them out as forward dynamics, a score provider ranks them as a cost oracle, and LatentMPCController owns the CEM/receding-horizon optimizer that ties them together. Evaluation and benchmarking sit on top so a builder can measure and select the best provider/configuration. World generation, symbolic-scenario simulation, and media generation are out of scope.

Intended users are Python developers and roboticists composing provider-backed planning loops, evaluation harnesses, and benchmarks to choose configurations for physical-AI tasks.

Architecture Map

• src/worldforge/models.py: public compatibility facade that re-exports model contracts, validation errors, shared helpers, provider contracts, scene models, and capability results.
• src/worldforge/_model_utils.py: shared JSON-native validation helpers, framework errors, deterministic IDs, and numeric/probability checks re-exported through models.py.
• src/worldforge/scene_models.py: geometry primitives, actions, scene objects, and scene patches used to build provider world_state dicts (no symbolic world runtime).
• src/worldforge/capability_results.py: embedding, action-score, and embodied-policy result payload contracts.
• src/worldforge/provider_models.py: public compatibility facade for provider-facing contracts.
• src/worldforge/provider_profiles.py: provider capabilities, provider info, and profile metadata.
• src/worldforge/provider_request_policy.py: retry/backoff and operation timeout policies for HTTP-backed providers.
• src/worldforge/provider_events.py: structured provider events and event-field validation.
• src/worldforge/provider_diagnostics.py: provider health, lifecycle readiness, and doctor report models.
• src/worldforge/provider_redaction.py: observable-field sanitization shared by provider events, diagnostics, manifests, routing, and reports.
• src/worldforge/capabilities/__init__.py: runtime-checkable capability protocols for narrow Cost, Policy, Predictor, Embedder, and RunnableModel integrations.
• src/worldforge/control/: score-driven controller primitives. LatentMPCController samples action horizons in pure Python, scores them through score_actions, and returns an optimized chunk without importing optional ML runtimes or stepping host environments.
• src/worldforge/framework_capabilities.py: internal capability-protocol registry, structural dispatch, observable-wrapper ownership, and direct/named capability target resolution.
• src/worldforge/framework.py: WorldForge, provider registration, capability dispatch (predict/score/policy/embed), diagnostics, provider operations, and top-level evaluation helpers. The backbone planner is control.LatentMPCController, which calls score_actions.
• src/worldforge/providers/base.py: provider interfaces, ProviderError, remote-provider base behavior, and PredictionPayload.
• src/worldforge/providers/observable.py: internal wrapper that adds ProviderEvent, health, profile, info, and timing behavior around pure capability protocol implementations.
• src/worldforge/providers/catalog.py: provider factories and auto-registration policy for the in-repo provider catalog.
• src/worldforge/providers/mock.py: deterministic local provider used by tests, examples, and contract checks.
• src/worldforge/providers/cosmos_policy.py: host-owned NVIDIA Cosmos-Policy ALOHA /act server adapter for selecting embodied action chunks through the policy capability.
• src/worldforge/providers/leworldmodel.py: real optional LeWorldModel JEPA cost-model adapter for scoring action candidates through stable_worldmodel.policy.AutoCostModel.
• src/worldforge/providers/gr00t.py: experimental host-owned NVIDIA Isaac GR00T PolicyClient adapter for selecting embodied action chunks through the policy capability.
• src/worldforge/providers/lerobot.py: host-owned Hugging Face LeRobot PreTrainedPolicy adapter for selecting embodied action chunks through the policy capability.
• src/worldforge/providers/jepa_wms.py: candidate contract scaffold for facebookresearch/jepa-wms score-provider work; it supports injected test/runtime scoring and a host-owned torch-hub runtime but is intentionally not exported or registered.
• src/worldforge/providers/remote.py: credential-gated scaffold providers for jepa and genie; these intentionally use deterministic mock behavior after credential checks.
• src/worldforge/evaluation/: built-in physics and planning suites plus report renderers.
• src/worldforge/benchmark.py: capability-aware provider latency, retry, and throughput harness.
• src/worldforge/observability.py: composable ProviderEvent sinks for JSON logging, in-memory recording, and metrics aggregation.
• src/worldforge/workflow_trace.py: JSON-native composed workflow trace artifacts for planning, evaluation, provider-event conversion, Markdown export, and optional Rerun logging.
• src/worldforge/rerun.py: optional Rerun SDK bridge for sanitized provider events, world snapshots, plans, workflow traces, benchmark reports, robotics showcase visual layers, and JSON artifacts. Rerun is not a provider capability and stays behind the rerun extra or host-owned optional runtimes that already provide rerun-sdk.
• src/worldforge/testing/: reusable adapter contract helpers, fixture loaders, fixture snapshot manifest helpers, runtime markers, and deterministic controls for artifact/report tests.
• src/worldforge/demos/: packaged demo entry points exposed through uv run console scripts.
• src/worldforge/demos/lerobot_e2e.py: packaged LeRobot policy-plus-score planning demo exposed through uv run worldforge-demo-lerobot.
• src/worldforge/demos/rerun_showcase.py: packaged Rerun observability and artifact showcase exposed through uv run --extra rerun worldforge-demo-rerun.
• scripts/demo_showcases.py: checkout-safe demo evidence runner for the first-run, diagnostics, robotics replay, remote dry-run, adapter authoring, batch eval, service host, Rerun gallery, failure lab, cookbook, external provider package, custom evaluation suite, and policy+score candidate lab, fixture drift review, capability negotiation preflight, and embodied policy replay comparison, non-developer evidence review, and provider failure gallery workflows.
• scripts/release_readiness_drill.py: checkout-safe release readiness drill that renders clean-pass and controlled-failure release-evidence artifacts without publishing, tagging, signing, or running host-owned optional runtimes.
• scripts/generate_release_evidence.py: checkout-safe release evidence generator that records validation gate status, sanitized command output, artifact hashes, live-smoke manifest links, known limitations, and claim boundaries without publishing, tagging, or signing.
• scripts/generate_dependency_audit_evidence.py: checkout-safe dependency-audit evidence wrapper around the documented uv export plus pip-audit flow; writes JSON and Markdown summaries with sanitized raw details and without preserving the temporary requirements file.
• scripts/generate_quality_dashboard.py: local quality dashboard generator that reads release evidence, dependency-audit evidence, and core-performance output and emits JSON/Markdown status summaries with sanitized raw details and deterministic redacted-key collision handling, without running gates.
• scripts/generate_release_notes.py: maintainer-editable release notes draft generator that assembles CHANGELOG.md, optional closed GitHub issue metadata, release evidence JSON, validation summaries, row-level validation gate status, caveats, and host-owned optional runtime evidence through shared text redaction, without publishing.
• src/worldforge/harness/: robotics showcase flow/report package. Keep flow metadata and runners independent from Textual; tui.py is the only Textual-dependent module. Current flows cover LeWorldModel score planning, LeRobot policy-plus-score planning, Cosmos-Policy ALOHA replay, GR00T PolicyClient replay, and robotics policy replay comparison.
• src/worldforge/harness/tui_styles.py: Textual-free CSS constants consumed by tui.py; keep styling declarations here instead of embedding large CSS strings in widget classes.
• src/worldforge/smoke/: packaged optional-runtime smoke entry points exposed through uv run console scripts.
• src/worldforge/smoke/lerobot_leworldmodel.py: optional host-owned real robotics showcase that composes a LeRobot policy checkpoint with a LeWorldModel score checkpoint through forge.select_actions (policy) plus forge.score_actions (score) candidate ranking.
• src/worldforge/smoke/robotics_showcase.py: one-command PushT real robotics showcase that wires the packaged PushT observation, score, translator, and candidate bridge defaults into lewm-lerobot-real.
• src/worldforge/smoke/pusht_showcase_inputs.py: packaged PushT showcase hooks for building the LeRobot observation, LeWorldModel score tensors, and checkpoint-native action candidates.
• src/worldforge/smoke/leworldmodel_checkpoint.py: optional host-owned builder for creating the LeWorldModel *_object.ckpt file expected by AutoCostModel from Hugging Face LeWM assets.
• examples/leworldmodel_e2e_demo.py: checkout-safe end-to-end LeWorldModel provider-surface score-planning compatibility wrapper for uv run worldforge-demo-leworldmodel.
• examples/lerobot_e2e_demo.py: checkout-safe end-to-end LeRobot policy-plus-score planning compatibility wrapper with an injected deterministic policy.
• scripts/generate_provider_docs.py: provider catalog documentation generator and drift check.
• scripts/check_docs_snippets.py: checkout-safe snippet gate for selected Python and JSON docs blocks, with explicit host-owned, credentialed, and illustrative skip markers.
• scripts/check_optional_import_boundaries.py: checkout-safe static and import-time audit that keeps Textual, Rerun, torch, stable-worldmodel, LeRobot, GR00T, and Cosmos-Policy imports behind their allowed optional-runtime modules.
• scripts/scaffold_provider.py: safe scaffold generator for new provider adapter files, fixture placeholders, tests, runtime manifest stubs, docs stubs, and workbench checklists.
• scripts/smoke_leworldmodel.py: compatibility wrapper for uv run --python 3.13 --with "stable-worldmodel @ git+https://github.com/galilai-group/stable-worldmodel.git" --with "datasets>=2.21" worldforge-smoke-leworldmodel.
• scripts/smoke_gr00t_policy.py: optional live GR00T PolicyClient smoke for host environments with Isaac-GR00T or a reachable policy server; startup command logs must redact forwarded secret-shaped server args and host-local paths.
• scripts/smoke_cosmos_policy.py: optional live Cosmos-Policy /act smoke for host environments with a reachable ALOHA policy server.
• scripts/smoke_lerobot_policy.py: optional live LeRobot PreTrainedPolicy smoke for host environments with LeRobot and robot-specific dependencies.