monitor-setup

/monitor setup

Set up a new metric monitor or register a metric definition.

Instructions

You are helping the user set up structured metric monitoring. This command does two things:

1. Register a metric definition in .analysis/metrics/definitions/
2. Create a monitor for that metric in .analysis/metrics/monitors/

Follow these steps in order.

Step 1: Check prerequisites

• Check if .analysis/metrics/ folder exists. If not, create the structure:

  .analysis/metrics/
  ├── definitions/
  │   ├── north-star/
  │   ├── leading/
  │   ├── guardrail/
  │   └── diagnostic/
  ├── monitors/
  └── alerts/
  

• Read .analysis/config.md for existing metric framework (North Star, Leading, Guardrail, Diagnostic).

Step 2: Identify the metric

Q1: Which metric do you want to monitor?

Three paths:

• From config.md: "Pick from your existing metrics in config.md" → show list
• From EVOLVE proposal: "Register a metric proposed during analysis" → ask for analysis ID, then:
  1. Resolve file path: Full analysis → analyses/active/{ID}_{slug}/05_evolve.md, Quick → analyses/active/quick_{ID}_{slug}.md (EVOLVE section)
  2. Read the "Proposed New Metrics" section and extract: Background, Purpose (including proposed tier), Definition & Logic, Interpretation Guide, STEDII Validation
  3. Use the extracted content to pre-fill the metric definition in Step 3
  4. STEDII carry-forward: If the EVOLVE proposal has completed STEDII checkboxes ([x]), carry them into the definition. If any are unchecked ([ ]), guide the user through the remaining criteria before proceeding.
• New metric: "Define a new metric from scratch" → guide through definition

Step 3: Register metric definition (if not already registered)

If the metric doesn't have a definition file yet, create one.

Create file: .analysis/metrics/definitions/{tier}/{metric-slug}.md

# Metric: {name}
> Tier: {🌟 North Star / 📊 Leading / 🛡️ Guardrail / 🔬 Diagnostic}
> Registered: {YYYY-MM-DD}
> Owner: {team/person}
> Source: {config.md / analysis {ID} / experiment {ID} / new}

## Definition
- **What it measures**: (one sentence)
- **Formula**: {calculation}
- **Data source**: {table, API, dashboard}
- **Granularity**: daily / weekly / monthly / per-cohort
- **Refresh cadence**: {how often the data updates}

## Interpretation
- **Direction**: ↑ higher is better / ↓ lower is better
- **Healthy range**: {min} — {max}
- **Warning threshold**: {value}
- **Critical threshold**: {value}
- **Counter-metric**: {what to watch to prevent gaming}

## Context
- **Plain-language**: "When this goes up, it means... When it drops, it means..."
- **Segments that behave differently**: {list}
- **Seasonal patterns**: {if any}
- **Known data caveats**: {if any}

## STEDII Validation
- [x] **Sensitive** — {evidence}
- [x] **Trustworthy** — {evidence}
- [x] **Efficient** — {evidence}
- [x] **Debuggable** — {evidence}
- [x] **Interpretable** — {evidence}
- [x] **Inclusive** — {evidence}

## History
| Date | Change | Reason |
|------|--------|--------|
| {YYYY-MM-DD} | Created | {source} |