pytorch

PyTorch - Deep Learning & Tensors

PyTorch is a Python-based scientific computing package that uses the power of Graphics Processing Units (GPUs) and provides maximum flexibility and speed through its dynamic computational graph system.

When to Use

• Building and training Deep Neural Networks (CNN, RNN, Transformers).
• Researching new AI architectures with dynamic graph needs.
• Accelerating tensor math on NVIDIA (CUDA) or Mac (MPS) hardware.
• Solving Physics-Informed Neural Networks (PINNs).
• Implementing Generative models (GANs, Diffusion).
• Large-scale optimization using Autograd (automatic differentiation).
• Production-grade AI deployment (via TorchScript/ONNX).

Reference Documentation

Official docs: https://pytorch.org/docs/
Tutorials: https://pytorch.org/tutorials/
Search patterns: torch.nn, torch.optim, torch.utils.data, Autograd, Tensor.to(device)

Core Principles

The Tensor

The central data structure, similar to NumPy's ndarray, but with two key additions: it can live on a GPU and it supports automatic differentiation.

Dynamic Computational Graph (Autograd)

PyTorch builds the graph "on the fly" as code executes. This allows for standard Python control flow (if/for) inside your models.

Modules and Parameters

nn.Module is the base class for all neural network components. It automatically tracks nn.Parameter objects (weights/biases) for optimization.

Quick Reference

Installation

# CPU
pip install torch torchvision
# GPU (Check pytorch.org for specific CUDA versions)
pip install torch --index-url https://download.pytorch.org/whl/cu121

Standard Imports

import torch
import torch.nn as nn
import torch.optim as optim
import torch.nn.functional as F
from torch.utils.data import DataLoader, Dataset

Basic Pattern - Simple Linear Regression (The "PyTorch Way")

import torch

# 1. Data (Tensors)
X = torch.tensor([[1.0], [2.0], [3.0]], requires_grad=True)
y = torch.tensor([[2.0], [4.0], [6.0]])

# 2. Simple Model
model = torch.nn.Linear(1, 1) # y = w*x + b

# 3. Loss and Optimizer
criterion = torch.nn.MSELoss()
optimizer = torch.optim.SGD(model.parameters(), lr=0.01)

# 4. Training Loop
for epoch in range(100):
    prediction = model(X)
    loss = criterion(prediction, y)
    
    optimizer.zero_grad() # Clear previous gradients
    loss.backward()       # Compute gradients (Autograd)
    optimizer.step()      # Update weights