EnsembleKalmanProcesses.jl: Skill for Claude Code

slurm-pipeline-manager

This skill takes an EKP Julia example (decoupled or monolithic) and produces a complete SLURM job-dependency tree inside a slurm-variant/ subdirectory. slurm-variant/ is fully self-contained — it is both the HPC submission directory and the Julia project root. The original example is never modified.

The generated tree uses a one-shot precompile job followed by a dependency-chained setup → forward_map[array] → update_ensemble loop across iterations, ending in postprocessing — so ensemble members run in parallel and every stage pays precompile cost only once.

It also supports maintenance mode: if slurm-variant/hpc_config.sh already exists, re-invoking the skill updates files in place without clobbering the user's manifest settings.

---

Core principle — slurm-variant/ is fully self-contained

Everything new lives in slurm-variant/. The original example directory is never modified.

This includes:

• All SLURM / shell scripts (.sbatch, .sh)
• All decoupled Julia scripts (initialize_EKP.jl, run_forward_model.jl, etc.)
• Any shared model file (model.jl or similar)
• Parameter TOML files (priors.toml)
• A Project.toml copy with any additional dependencies

slurm-variant/ is the working directory for every HPC job — users cd into it before running anything. This means JULIA_PROJECT="." in hpc_config.sh and script names have no path prefix (e.g. SETUP_SCRIPT="initialize_EKP.jl").

---

How the generated pipeline is structured

precompile ─(manual, run first)

setup ──afterok──► fwd_iter_0[array 1..N] ──afterok──► update_iter_0
      (afterok chain continues for each iteration)
      ──afterok──► fwd_iter_K-1[array 1..N] ──afterok──► update_iter_K-1
                                                         ──afterok──► postprocess

Key design choices:

• One precompile job only. Every run-stage job sets JULIA_PKG_PRECOMPILE_AUTO=0. Without this, 60 concurrent array tasks each attempt to precompile — thrashing the shared Julia depot and wasting time.
• Forward map as a SLURM array. --array=1-$N_ENSEMBLE gives one task per ensemble member; the iteration index travels via --export=ALL,ITERATION=$i.
• afterok + --kill-on-invalid-dep=yes everywhere. If one stage fails the whole tree is killed, not left stuck in the queue. EKP errors on < 2 successful members (commit 8f2d3fa), so partial-run continuation is not useful.
• Date-stamped output root. RUN_DATE in hpc_config.sh pins all jobs in a run to the same output directory.