skypilot-multi-cloud-orchestration

SkyPilot Multi-Cloud Orchestration

Comprehensive guide to running ML workloads across clouds with automatic cost optimization using SkyPilot.

When to use SkyPilot

Use SkyPilot when:

• Running ML workloads across multiple clouds (AWS, GCP, Azure, etc.)
• Need cost optimization with automatic cloud/region selection
• Running long jobs on spot instances with auto-recovery
• Managing distributed multi-node training
• Want unified interface for 20+ cloud providers
• Need to avoid vendor lock-in

Key features:

• Multi-cloud: AWS, GCP, Azure, Kubernetes, Lambda, RunPod, 20+ providers
• Cost optimization: Automatic cheapest cloud/region selection
• Spot instances: 3-6x cost savings with automatic recovery
• Distributed training: Multi-node jobs with gang scheduling
• Managed jobs: Auto-recovery, checkpointing, fault tolerance
• Sky Serve: Model serving with autoscaling

Use alternatives instead:

• Modal: For simpler serverless GPU with Python-native API
• RunPod: For single-cloud persistent pods
• Kubernetes: For existing K8s infrastructure
• Ray: For pure Ray-based orchestration

Quick start

Installation

pip install "skypilot[aws,gcp,azure,kubernetes]"

# Verify cloud credentials
sky check

Hello World

Create hello.yaml:

resources:
  accelerators: T4:1

run: |
  nvidia-smi
  echo "Hello from SkyPilot!"

Launch:

sky launch -c hello hello.yaml

# SSH to cluster
ssh hello

# Terminate
sky down hello

Core concepts

Task YAML structure

# Task name (optional)
name: my-task

# Resource requirements
resources:
  cloud: aws              # Optional: auto-select if omitted
  region: us-west-2       # Optional: auto-select if omitted
  accelerators: A100:4    # GPU type and count
  cpus: 8+                # Minimum CPUs
  memory: 32+             # Minimum memory (GB)
  use_spot: true          # Use spot instances
  disk_size: 256          # Disk size (GB)

# Number of nodes for distributed training
num_nodes: 2

# Working directory (synced to ~/sky_workdir)
workdir: .

# Setup commands (run once)
setup: |
  pip install -r requirements.txt

# Run commands
run: |
  python train.py