running-python

Running Python

Rule: use a project environment, control threads, log clearly, and make outputs resumable.

What you get: with uv you control the interpreter and every package yourself, so nothing depends on what Python happens to be preinstalled on the cluster — no bootstrap step to worry about.

Tooling defaults

Use uv. It is the package manager for cluster Python work; everything below assumes it. Don't reach for conda, mamba, poetry, pipenv, pip-tools, or pip install --user — uv supersedes all of them. Why specifically uv on this cluster:

• Single tool for dependencies, lockfile, virtualenv, and Python interpreter — no module/conda/pyenv stack to coordinate. The cluster's system python3 is old, and the python modules are a fixed set that changes between maintenance windows (check python3 --version and module spider python); uv downloads and pins whatever recent Python your project needs (pyproject.toml's requires-python).
• Lockfile (uv.lock) is built-in and resolves identically on the login node, compute node, and your laptop — no conda env export games, no "works on my machine."
• 10–100× faster than conda on GPFS. A uv sync --frozen is a few seconds; a conda env create is a multi-minute metadata storm because conda writes thousands of small files into one directory.
• Single .venv directory in your project — easy to inspect, easy to nuke, easy to atomically swap.

These pair with uv:

• ruff for lint + format. Catches mistakes locally before burning a Slurm allocation.
• pyrefly for type checking. Same reason as ruff.
• pytest for tests. Smoke tests on small inputs save many cluster reruns.
• argparse for batch scripts (one-file entry points like run_task.py --task-id). click only when you grow into a reusable project CLI; the extra dependency is not worth it for a single sbatch script.
• pathlib over os.path. Joining paths and checking parents is what you do most on the cluster.
• logging as the baseline (configured below). loguru is fine when its structured output materially helps incident debugging.
• pyproject.toml + uv.lock committed; .venv/ gitignored. The lockfile is what makes runs reproducible across login and compute nodes.