scientific-coding

Scientific Coding Principles

Industry code serves users. Scientific code serves truth. A wrong result that looks right is the worst possible outcome. Every rule below follows from this.

These are not general coding tips. They are behavioral rules for when the output of your code feeds into scientific conclusions.

1. Understand the Experiment Before Writing Code

The experimental design determines the code architecture. Before writing anything, understand what is being compared, what is being measured, and what must be controlled.

Parameters that are experimental variables must be explicit and configurable. Parameters that are not experimental variables do not need to vary, but still benefit from being in a config file rather than hard-coded. A config file is a record of what was used. Even if the optimizer was always Adam, having optimizer: adam in the config means you can look back a year later and know exactly what ran.

Example: if the experiment compares PyTorch 2.0 vs 1.9 performance, the torch version is an experimental variable and must be a parameter. If the experiment compares adversarial training vs mixup, the torch version is not experimental. It does not need to vary, but recording it in config is still useful.

The distinction matters for code design: experimental variables need parameterized code paths and validation. Non-experimental settings just need to be recorded. Do not confuse the two.

Code structure follows experimental structure.

Share code between experimental conditions. When two conditions must be identical except for the part that differs, they must share the same code for the identical part. This is not only about avoiding duplication for maintainability. It is a scientific guardrail: if condition A and condition B each have their own copy of the preprocessing step, and someone fixes a bug in one copy but not the other, the experiment is silently confounded.

# WRONG: separate preprocessing per condition in different files
# preprocess_adversarial.py
def preprocess_data_adversarial(config):
    data = load_dataset(config["dataset"])
    data = normalize(data, config["norm_mean"], config["norm_std"])
    data = augment(data, config["flip"], config["crop_size"])

    data = build_batch_adversarial(data)
    return data

# preprocess_mixup.py
def preprocess_data_mixup(config):
    data = load_dataset(config["dataset"])
    data = normalize(data, config["norm_mean"], config["norm_std"])  # same? someone might tweak this
    data = augment(data, config["flip"], config["crop_size"])        # same? nothing enforces it

    data = build_batch_mixup(data)
    return data
# If someone fixes a bug in one file but not the other,
# the comparison is silently confounded.

# RIGHT: one shared preprocessing function, condition-specific logic separate
def preprocess_data(config):
    data = load_dataset(config["dataset"])
    data = normalize(data, config["norm_mean"], config["norm_std"])  # shared, guaranteed identical
    data = augment(data, config["flip"], config["crop_size"])

    if config["method"] == "mixup": # <- only fork when it really is required
        data = build_batch_mixup(data)
    elif config["method"] == "adversarial":
        data = build_batch_adversarial(data)
    else:
        raise ValueError(...) # < - failing loudly
    return data

def build_batch_adversarial(data): ...  # only the part that actually differs
def build_batch_mixup(data): ...