loom-karpenter

Karpenter

Overview

Karpenter provisions right-sized nodes directly from cloud-provider APIs based on aggregate pending-pod resource requests — no node groups, sub-minute scale-up, active consolidation to cheaper nodes. vs Cluster Autoscaler: no predefined ASGs, picks from all instance types, bin-packs, first-class spot (on-demand fallback is opt-in, NOT automatic).

Core dependency: accurate pod resources.requests — Karpenter bin-packs against requests (limits are ignored for scheduling). Instance-type flexibility is the engine of bin-packing and consolidation; over-constraining families defeats both and narrows the spot pool.

This skill targets the v1 API (Karpenter 1.0+): NodePool = karpenter.sh/v1, EC2NodeClass = karpenter.k8s.aws/v1. The pre-v1 Provisioner/AWSNodeTemplate and v1beta1 are gone (see Currency).

Examples

1. Basic NodePool (broad, flexible)

apiVersion: karpenter.sh/v1
kind: NodePool
metadata: {name: default}
spec:
  template:
    spec:
      # v1 requires group AND kind alongside name (no default fallback → NotReady).
      nodeClassRef:
        group: karpenter.k8s.aws
        kind: EC2NodeClass
        name: default
      requirements:
        - {key: kubernetes.io/arch, operator: In, values: ["amd64", "arm64"]}
        # Prefer instance-category + generation over a fixed family list (broader
        # spot pool, auto-adopts new generations — the official default).
        - {key: karpenter.k8s.aws/instance-category, operator: In, values: ["c", "m", "r"]}
        - {key: karpenter.k8s.aws/instance-generation, operator: Gt, values: ["2"]}
        # Both types in ONE NodePool = on-demand fallback when spot is exhausted.
        - {key: karpenter.sh/capacity-type, operator: In, values: ["spot", "on-demand"]}
      # expireAfter is a v1 TEMPLATE field and is drift-able (changing it rolls nodes).
      expireAfter: 720h
      taints: [{key: workload-type, value: general, effect: NoSchedule}]
  limits: {cpu: 1000, memory: 1000Gi}   # soft, eventually-consistent, per-NodePool (see Gotchas)
  disruption:
    consolidationPolicy: WhenEmptyOrUnderutilized   # v1 name; WhenUnderutilized is rejected
    consolidateAfter: 30s
    budgets: [{nodes: "10%", duration: 5m}]
  weight: 10