data-workflows: Skill for Claude Code

Hex Tuning

How to size a vector hex job, and how to choose resolutions that stay joinable to the rest of the catalog.

Memory and Chunking Mental Model

RAM is driven by the H3 cell count of the single largest feature in a chunk — not dataset size or bounding box.

⛔ MEASURE EVERY dimension you request — never inherit one from a neighbouring job

kubectl top pod columns are NAME CPU MEMORY — cpu is $2, memory is $3. Reading $1 gives you the pod name and every number comes out zero.

# memory: handle both Mi and Gi — kubectl top mixes them, and a naive gsub(/Mi/,"") reports
# a peak BELOW the mean
kubectl -n geo-workflows top pod --no-headers | grep '^<prefix>' | awk '{
  v=$3; if(v~/Gi$/){gsub(/Gi/,"",v); v=v*1024} else gsub(/Mi/,"",v);
  if(v+0>m)m=v+0; s+=v; n++} END {printf "n=%d peak=%.2fGi mean=%.2fGi\n", n, m/1024, s/n/1024}'

# cpu
kubectl -n geo-workflows top pod --no-headers | grep '^<prefix>' | awk '{
  c=$2; gsub(/m/,"",c); if(c+0>m)m=c+0; s+=c; n++} END {
  printf "n=%d peak=%.2f mean=%.2f cores\n", n, m/1000, s/n/1000}'

Measured on the CHELSA build, against what was requested — three dimensions, all inherited rather than measured, each throttling throughput a different way:

The ephemeral case is the sharpest lesson. A join pod asked for 40 Gi of ephemeral storage to write a 2 KB partition, having inherited the number from a different job. It did not run slowly — it cycled Leased -> Pending -> LeaseReturned -> Queued indefinitely, because no node had that much free ephemeral, while 121 identical jobs placed fine. Nothing in the logs says "your request is too large"; the job simply never starts.