build-optimization

Skill: Build Optimization

Expertise: Docker BuildKit cache mounts, GitHub Actions/GitLab CI caching, parallelization, selective test runs.

When to load

When a pipeline takes > 5 minutes, Docker builds are slow, or you need to reduce CI costs.

Docker Layer Cache Strategy

# ✅ Order layers by change frequency (least → most)
FROM python:3.12-slim AS builder

# 1. System deps (changes rarely → cache long)
RUN apt-get update && apt-get install -y --no-install-recommends gcc libpq-dev

# 2. Python deps (changes on lockfile update → medium cache)
WORKDIR /app
COPY requirements.txt .
RUN pip install --user --no-cache-dir -r requirements.txt

# 3. App code (changes every commit → no cache benefit here)
COPY src/ ./src/

BuildKit Cache Mounts (Docker 18.09+)

# Cache pip downloads across builds (never invalidated by lockfile change)
RUN --mount=type=cache,target=/root/.cache/pip \
    pip install --user -r requirements.txt

# Cache npm across builds
RUN --mount=type=cache,target=/root/.npm \
    npm ci --cache /root/.npm

# Cache Go modules
RUN --mount=type=cache,target=/go/pkg/mod \
    go build ./...

GitHub Actions Cache

# Python: built-in setup-python cache
- uses: actions/setup-python@v5
  with:
    python-version: "3.12"
    cache: pip                      # hashes requirements*.txt automatically

# Node: built-in setup-node cache
- uses: actions/setup-node@v4
  with:
    node-version: "20"
    cache: npm

# Go: manual cache
- uses: actions/cache@v4
  with:
    path: |
      ~/.cache/go-build
      ~/go/pkg/mod
    key: go-${{ runner.os }}-${{ hashFiles('**/go.sum') }}
    restore-keys: go-${{ runner.os }}-

# Docker layer cache via GHA cache backend (BuildKit)
- uses: docker/build-push-action@v6
  with:
    cache-from: type=gha
    cache-to: type=gha,mode=max

GitLab CI Cache

# Global cache shared across jobs (same branch)
cache:
  key: ${CI_COMMIT_REF_SLUG}
  paths: [.cache/pip, node_modules/]

# Per-job cache with fallback
test:
  cache:
    key:
      files: [requirements.txt]     # invalidate only when lockfile changes
    paths: [.cache/pip]
    fallback_keys:
      - ${CI_COMMIT_REF_SLUG}-pip
      - pip