bioinformatics-project

Bioinformatics Project Organization

Set up a computational-biology project so a stranger can understand what was done and why, and so every analysis can be rerun end-to-end from a script. Apply this when creating, restructuring, or reproducibility-hardening any genomics, metagenomics, sequencing, or computational-biology project — and when running any single analysis command that should be reproducible.

Two assumptions drive every rule below (Noble 2009): a stranger must be able to reconstruct the analysis from the files alone, and everything will be rerun — with new data, new parameters, or a fixed bug — so build each experiment to re-execute from a script, never from memory.

Instructions

New project startup workflow

Before creating directories or running the first command, turn the project into a small written contract:

1. Name the project with a durable, specific directory name.
2. Write the objective, analysis tracks, expected deliverables, input data types, compute environment, and known reference databases in README.md.
3. Create tasks/METHODS.md immediately and record every setup command, software version, database path, option, seed, and SLURM job ID as work proceeds.
4. Create SUMMARY.md for current status and high-level counts.
5. Create tasks/todo.md for active work and tasks/lessons.md for corrections and prevention rules. When memd is available, initialize its project scope and let memd memory-md maintain the root memory.md; do not hand-edit that file.
6. Initialize dependency management before analysis: use pixi.toml for bioinformatics stacks, uv only for Python-only projects, and never rely on system Python or ad hoc conda environments.
7. Check existing shared database locations before downloading references; if a new reference is needed, put it in a named database directory and document source URL, version, date, checksum, and command.
8. Put raw inputs and metadata in read-only data directories; never write cleaned, mapped, filtered, or derived outputs back into raw data.
9. Before exploratory analysis, create a hypothesis register with at least five biological, technical, null, sampling, and database explanations. Keep ruled-out hypotheses visible with the evidence that changed their status.
10. For each analysis stage, create a restartable driver script plus a README before launching compute, then update methods and summaries after the run.
11. Keep root-level directories intuitive from day one; do not let raw data, generated batch outputs, publication figures, logs, and scratch files become peers at the project root.