analysis-manifest

Analysis Manifest

Maintain a persistent JSON record of the current analysis session. Every skill stage reads the manifest to understand what has been completed and writes its outputs back. This solves context drift in long analyses where agents lose track of earlier decisions.

Why a manifest: In a multi-hour analysis, agents forget what hypotheses were agreed in brainstorming, what metric was declared primary, and what artifacts already exist. The manifest is the answer to "where are we?" at any point in the session.

Iron Law

NO STAGE MAY CONTRADICT A DECISION RECORDED IN THE MANIFEST WITHOUT EXPLICIT USER APPROVAL.

If model-selection wants to change the primary metric that was declared in brainstorming, that change requires a human decision — not a silent override.

Manifest Structure

artifacts/analysis_manifest.json:

{
  "project": "<analysis name>",
  "created_at": "<ISO timestamp>",
  "last_updated": "<ISO timestamp>",
  "git_commit": "<hash at session start>",

  "brainstorming": {
    "completed": false,
    "design_doc": null,
    "hypotheses": [],
    "primary_metric": null,
    "baseline_expectation": null,
    "validation_strategy": null
  },

  "data_profiling": {
    "completed": false,
    "profile_path": null,
    "target_col": null,
    "n_rows": null,
    "n_cols": null,
    "hidden_nulls_found": []
  },

  "data_exploration": {
    "completed": false,
    "eda_report_path": null,
    "quality_score": null,
    "leakage_candidates": []
  },

  "data_validation": {
    "completed": false,
    "tdds_report_path": null,
    "decision": null
  },

  "leakage_guard": {
    "completed": false,
    "report_path": null,
    "decision": null
  },

  "feature_engineering": {
    "completed": false,
    "registry_path": null,
    "n_features": null,
    "transformer_paths": []
  },

  "model_selection": {
    "completed": false,
    "chosen_model": null,
    "baseline_score": null,
    "best_cv_score": null,
    "hpo_trials": null
  },

  "model_evaluation": {
    "completed": false,
    "test_evaluated": false,
    "final_score": null,
    "ci_lower": null,
    "ci_upper": null,
    "shap_path": null
  },

  "report": {
    "completed": false,
    "report_path": null
  },

  "warnings": [],
  "human_approvals": []
}