deep-learning

deep-learning — train a neural net in PyTorch without the silent bugs

You own training and understanding neural networks in PyTorch: the loop, autograd, mixed precision, optimizers/schedulers, multi-GPU, and the reproducibility/checkpoint hygiene that separates a real result from a lucky one. Nets from scratch, vision, custom architectures — all here. This is PyTorch-first by design: JAX and TensorFlow are real and fine, but the patterns, APIs, and gotchas below are Torch's.

Version reality (verify at author time). Current stable is PyTorch 2.x — ~2.13 as of mid-2026 (pytorch.org/get-started, releases move fast; don't hard-pin a minor). Everything below is stable 2.x API. The one namespace shift to know: AMP now lives under torch.amp (torch.amp.GradScaler("cuda")), not the old torch.cuda.amp.*.

Am I in the right skill?

You are doing…	Skill
Training/debugging a net in PyTorch (from scratch, vision, custom loop, AMP, multi-GPU)	deep-learning (here)
Adapting a pretrained LLM — LoRA/QLoRA, SFT, trl/peft	finetuning
Classic/tabular — sklearn, XGBoost/LightGBM, feature engineering	machine-learning
Tokenization, NLP task modeling, task metrics (F1/BLEU/ROUGE)	nlp
Envs, packaging, tests and hygiene around the model code	python

1. PyTorch essentials

Three objects carry everything.

• Tensor — an n-d array on a device (cpu/cuda/mps) with a dtype. requires_grad=True makes autograd track ops on it. .to(device) / .detach() / .item() are the moves you use constantly; .item() pulls a Python scalar and drops the graph (see the loop bugs below).
• autograd — a tape. Every op on a requires_grad tensor records a node; loss.backward() walks it and accumulates into each leaf's .grad. "Accumulates" is the word that bites people (§2). Wrap read-only regions in torch.no_grad() to skip taping.
• nn.Module — the model container. __init__ registers submodules/params; forward defines compute. model.parameters() feeds the optimizer; model.train() / model.eval() flip train-vs-eval behavior for Dropout and BatchNorm.