using-gpus

Using GPUs

Rule: hold a GPU only while GPU code is actively running. Do CPU preprocessing, downloads, tokenization, and web/API calls elsewhere.

GPUs are the scarcest resource on the cluster. An idle interactive GPU session — srun --pty bash left open while you go to lunch — is blocking another user's job right now. Cancel it. The H100 partition now has two 4-GPU nodes when healthy, but it remains the scarcest partition; treat it accordingly.

Account for every GPU-hour

Treat each requested GPU-hour as compute somebody else cannot use. The cluster has finite GPUs and H100s are scarce; there are no per-user GPU caps, so it is on you to cancel idle GPU jobs the instant you notice them — scancel JOBID — and never request more GPUs than your code uses.

Do you need a GPU?

Use a GPU for:

• deep learning training
• transformer/LLM inference
• CUDA/PyTorch/JAX/TensorFlow code
• RAPIDS/CuPy code that is explicitly GPU-backed

Do not use a GPU for:

• Stata regressions
• fixest, pyfixest, statsmodels, or ordinary CPU dataframe work
• data cleaning, merging, reshaping, tokenization
• downloading data or making network/API requests
• bootstrap/Monte Carlo unless code is actually CUDA-backed

Request one GPU first

#!/bin/bash
#SBATCH --job-name=gpu-test
#SBATCH --partition=gpunormal
#SBATCH --gres=gpu:1
#SBATCH --time=01:00:00
#SBATCH --cpus-per-task=4
#SBATCH --mem=32G
#SBATCH --output=logs/%x_%j.out

set -euo pipefail

export OMP_NUM_THREADS=${SLURM_CPUS_PER_TASK:-1}
export MKL_NUM_THREADS=${SLURM_CPUS_PER_TASK:-1}
export OPENBLAS_NUM_THREADS=${SLURM_CPUS_PER_TASK:-1}

nvidia-smi
# Environment was created during setup with: uv sync --frozen
srun .venv/bin/python train.py

Only request multiple GPUs if the code explicitly uses multiple GPUs.

Use gpunormal by default. (Note: gpunormal is the cluster's general production queue on the GPU nodes, not a GPU-only partition — you get a GPU only because you asked with --gres=gpu:1. A job there without --gres runs CPU-only and consumes no GPU. See overview for the partition map.) Use h100 only when H100 performance or 80 GB VRAM specifically matters: