boettiger-lab/data-workflows

Dataset processing workflows for cloud-native geospatial data on NRP Kubernetes

5Stars on the repository
13Mods indexed here, across every type
2d agoLast push, which is what freshness is scored on
BSD-3-ClauseLicence, which decides whether bodies are shown

armada-pipeline

01

boettiger-lab/data-workflows

Skill Claude CodeCodex

Set up and use the Armada batch queue on NRP: headless device-code authentication (the documented PKCE flow cannot work from a shell), armadactl install and config, queue-to-namespace mapping, Armada priority classes, and when to microslice work into thousands of small jobs instead of hundreds of large ones. Use when…

5 2d ago A 107 tokens original BSD-3-Clause

dataset-recipes

02

boettiger-lab/data-workflows

Skill Claude CodeCodex

Worked end-to-end examples of dataset ingests to copy from: PAD-US multi-layer GDB, Census TIGER per-state zips with a preprocess job, a single-COG carbon raster, reading one small table out of a huge remote zip via GDAL range reads, and preprocessing multi-file zipped sources. Use as a starting template when a new…

5 2d ago A 82 tokens original BSD-3-Clause

hex-tuning

03

boettiger-lab/data-workflows

Skill Claude CodeCodex

Tune H3 hex generation for vector datasets: the memory and chunking mental model (peak RAM is driven by the largest single feature, not dataset size), H3 native and parent resolution conventions for join compatibility across the catalog, vector workflow parameters, and reprocessing failed chunks. Use when a hex job…

5 2d ago A 80 tokens original BSD-3-Clause

job-troubleshooting

04

boettiger-lab/data-workflows

Skill Claude CodeCodex

Diagnose failing or stuck Kubernetes jobs and DuckDB parquet errors: OOMKilled, evictions, ContainerStatusUnknown, flaky-node hangs, ephemeral-storage limits and PVC scratch, pod quota errors, 404s on convert, blank PMTiles in MapLibre, and the DuckDB httpfs stoi crash on oversized parquet column chunks. Use when a…

5 2d ago A 88 tokens original BSD-3-Clause

pod-preemption

05

boettiger-lab/data-workflows

Skill Claude CodeCodex

Diagnose and prevent preemption of long-running Kubernetes jobs: reading the succeeded-vs-failed gap that reveals pod mortality, why priorityClassName opportunistic (-2000000000) is unsafe for multi-hour pods, choosing a priority class and parallelism, and keeping failed pods inspectable when backoffLimitPerIndex…

5 2d ago A 105 tokens original BSD-3-Clause

raster-hexing

06

boettiger-lab/data-workflows

Skill Claude CodeCodex

Build and hex raster datasets: choosing the native H3 resolution from the source pixel, selecting the --hex-resampling reducer (sum, mean, mode, max, min), mosaicking a many-tile source into one COG, raster-workflow parameters, and the campaign checklist for re-hexing an existing raster. Use for any GeoTIFF or COG…

5 2d ago A 92 tokens original BSD-3-Clause

stac-authoring

07

boettiger-lab/data-workflows

Skill Claude CodeCodex

Author and verify STAC collection metadata for a published dataset: the verify-stac.py gate, the user-facing description register, table:columns placement, SPDX license rules, categorical values arrays, hex per-feature duplication notes, h3: resolution declarations, and registering a collection in its parent…

5 2d ago A 82 tokens original BSD-3-Clause