nisar-snow-diagnose

Diagnosing a product

This is the only part of the system with agency. Training and prediction had every decision deliberately removed so results stay reproducible; diagnostics is where the loop, the hypotheses, and the judgment live.

It runs on the analysis side, never in the path that generates a product. It reads artifacts that already exist and writes findings. It does not write rasters, does not refit, does not touch models/<id>/. If a diagnosis implies a change, that change belongs to nisar-snow-train and produces a new pinned model_id — never an edit to the current one.

There is no diagnose.py. This is an exploratory loop over saved artifacts, which is correct: freezing it into a module would be freezing the part that should stay flexible.

What you have to work with

import json, joblib, pandas as pd, numpy as np, rasterio
d    = "models/mcs-dsd-20260828-0f28ba57"
card = json.load(open(f"{d}/model_card.json"))
rows = pd.read_parquet(f"{d}/training_rows.parquet")   # features + dsd_mean + x/y
est  = joblib.load(f"{d}/model.joblib")
with rasterio.open("products/dsd_077_2026-02-07_<model_id>.tif") as ds:
    dsd, coh, gate = ds.read(1), ds.read(2), ds.read(3)
    tags = ds.tags()

rows carries the bookkeeping columns as well as the features — cell_id, row, col, x_center, y_center, cc_label, mask_water, mask_sub_ref, mask_sub_sec, pair_id, track — which is what makes spatial and mask-conditioned residual analysis possible without re-streaming.

Residuals must be out-of-fold. cross_validate returns (results, oof); in-sample residuals on a ridge with 12 features will look flat and tell you nothing.

The axes worth looking at