PiLoT: Instructions file for Codex

AGENTS.md — PiLoT agent guide

Instructions for AI coding assistants (Cursor, Claude Code, Codex, Windsurf, …) working in this repository.

What this repo does

PiLoT localizes a UAV camera from a query image + a geo-referenced 3D map. Two processes run in parallel (main.py):

1. Render worker — renders color + depth from the current pose estimate.
2. Localization worker — refines pose via dense feature alignment (PixLoc).

Closed loop: localization output → next render pose → next frame.

Repo layout

main.py                     # entry point, dual-process orchestration
configs/demos/              # official YAML configs (start here)
pixloc/localization/        # refiner, feature extraction
pixloc/pixlib/models/       # learned optimizer (CUDA)
pixloc/utils/citygs/        # CityGaussian renderer
pixloc/utils/gs3d/          # vanilla 3DGS PLY renderer
pixloc/utils/osg/           # legacy OSG / 3D Tiles renderer
scripts/                    # run & download helpers
data_demo/                  # demo data (not in git; download from HF)
DirectAbsoluteCostCuda/     # build pose optimizer CUDA ext

Install (conda env: pilot)

Prerequisites: Linux, NVIDIA GPU, CUDA, conda.

git clone https://github.com/Choyaa/PiLoT.git && cd PiLoT

# CityGaussian (SMBU renderer) — also accepts ../CityGaussian if already cloned
git clone https://github.com/DekuLiuTesla/CityGaussian.git third_party/CityGaussian

cd third_party/CityGaussian
conda env create -f environment.yml -n pilot
conda activate pilot
cd ../..

pip install -r requirements.txt

cd DirectAbsoluteCostCuda && CUDA_HOME=/usr/local/cuda python setup_build.py install && cd ..

# Feicuiwan PLY renderer only:
pip install git+https://github.com/graphdeco-inria/diff-gaussian-rasterization.git
pip install git+https://github.com/graphdeco-inria/gaussian-splatting.git#subdirectory=submodules/simple-knn

Verify:

python -c "import direct_abs_cost_cuda, torch; print('ok', torch.__version__)"