data-workflows AGENTS.md

Agent Instructions: Dataset Processing

You work in data-workflows, which uses cng-datasets to convert geospatial data into cloud-native formats on Kubernetes. This file carries the boundaries you must never cross and the shape of the pipeline; the procedural detail lives in skills that load on demand.

Skills — load the one that matches what you are doing

These are .claude/skills/<name>/SKILL.md in this repo. They work in Claude Code and opencode alike (both read .claude/skills/), and they are plain markdown, so any agent can read the file directly if it has no skill support.

skill	load it when
stac-authoring	writing or editing any stac-collection.json or dataset README (Steps 5–6)
raster-hexing	ingesting or re-hexing a GeoTIFF/COG — resolution, reducer, mosaicking
hex-tuning	a hex job OOMs, or choosing native/parent H3 resolutions and chunk sizes
job-troubleshooting	a job fails, hangs, or a published parquet will not read
dataset-recipes	starting an ingest that resembles a worked example

Everything below is always in force, skill or no skill.

⛔ HARD BOUNDARY: Scoping decisions live in the GitHub issue, NEVER in memory

Every decision that defines what a dataset task delivers — spatial extent (full upstream coverage vs. a regional clip), H3 resolution, reducer, source product/version, target bucket and naming, acceptance criteria — MUST be written into the GitHub issue before you act on it. Sessions are disposable: the laptop dies, agent memory is wiped, a different agent picks up the task. The issue is the single source of truth that survives all of that. Treat yourself like any professional developer — nobody is expected to carry scope "in their head," and you must not rely on agent memory (~/.claude/.../memory) for scope. Memory is for how you work, never what a task delivers.

• Before building: if the issue does not state the extent, resolution, and acceptance criteria explicitly, the issue is underspecified. Stop, propose the scope, get agreement, and edit the issue to record it (gh issue edit / a scoping comment) before running cluster jobs. A vague label like "the wyoming group" is not a scope.
• When scope changes mid-task (e.g. "extend to the full upstream extent, not the regional clip"): update the issue body in the same turn you learn it — don't just remember it.
• When the user tells you a fact only they hold — a dataset's origin, why a licence is what it is, who contributed something, what an upstream field means — write it into the issue in the same turn, before using it. These facts are unrecoverable: nobody can re-derive them from the data, so a fact left in session context is destroyed when the session ends, and the next agent asks the same question again. If you are about to ask the user something, first search the repo, git history, and the issues for the answer: being asked twice is the symptom that the first answer was never recorded (#579).
• Never infer scope from the bucket name, an existing clipped COG, or a prior build's resolution. Those are artifacts of how an earlier (possibly wrong) pass happened to run, not statements of intent.
• A reviewer (human or agent) must be able to read the issue alone and know exactly what to build. If you found yourself reconstructing scope from code, S3 layout, or memory, that is the signal the issue needs updating.