observability-gap

Observability gap

Making the next incident cheaper than this one.

Why this exists

After an incident everyone discusses the cause. Almost nobody discusses why it took forty minutes to find — and that second question is usually the more valuable one, because the cause was specific to this incident while the diagnosis difficulty will recur across every future one.

If the answer was hard to find, that is a fixable defect. A missing log line, an absent metric, a dashboard that shows the symptom but not the cause, an alert that fires on the wrong thing. Each one costs time in every incident that touches that area, forever, and each is usually a small change.

This is the compounding skill in the portfolio. It's also the one most easily deferred, because the incident is over and the pressure is gone. That's precisely why it needs to be a deliberate step rather than an intention.

The other half is prospective: before shipping, asking whether this change's failure would be silent. A failure nobody detects is far more dangerous than one that pages someone at 3am.

When this applies

• After any incident where diagnosis was hard
• Before shipping anything whose failure would be silent
• A change has no way to be detected going wrong
• Diagnosis required guessing rather than looking
• nfr-baseline flagged a silent-failure risk

When it doesn't

• During an active incident — fix it first, come back after
• The system is already well instrumented in the relevant area
• Adding telemetry nobody will look at, which is cost without benefit

Prerequisites

• Locate the workspace: FDE_WORKSPACE, else the charter Location, else .fde/, else ../<repo>-fde/
• .fde/incidents/*.md — specifically the "what made this hard" section
• .fde/03b-nfrs.md — the silent-failure analysis
• .fde/08-release-readiness.md — the observability dimension

Procedure

1. Start from a real diagnosis, not from a checklist

The best input is a specific incident and the specific moments where you were stuck. incident-triage records these under "what made this hard" for exactly this reason.