audit-observability

Audit Observability

Scans the codebase and running services for missing or insufficient observability: logging, metrics, tracing, and alerting. Produces a gaps report ranked by risk so teams can prioritize adding instrumentation before issues arise in production.

Prerequisites

• Working directory is the root of the repository
• Optional: $1 — path or service name to scope the audit (defaults to .)
• Access to the codebase to search for instrumentation patterns
• Read .sdlc/context/architecture.md to understand the services and infrastructure

Observability Pillars

Pillar	Purpose	Examples
Logging	Record discrete events for debugging	Structured logs, error logs, audit logs
Metrics	Quantitative measurements over time	Request count, latency histogram, error rate
Tracing	Follow a request across service boundaries	Distributed trace IDs, span propagation
Alerting	Notify on-call when conditions degrade	PagerDuty, OpsGenie, Grafana alerts

Steps

1. Read .sdlc/context/architecture.md to identify services, endpoints, databases, and external dependencies.

2. Detect which observability libraries and frameworks are already in use:

   rg -n -i "(structlog|logging|logger|prometheus|datadog|opentelemetry|otel|jaeger|zipkin|sentry)" \
     -g '*.{py,ts,js,go}' ${1:-.} | head -40
   

3. Audit logging coverage:

   • Are errors logged with context (request ID, user ID, stack trace)?
   • Are external service calls logged (request/response, latency, status)?
   • Are business-critical operations logged (auth events, data mutations, payments)?
   • Is structured logging used consistently?
   • Are log levels appropriate (not everything at INFO/DEBUG)?

   rg -n -i "(except|catch|error|raise|throw)" \
     -g '*.{py,ts,js,go}' ${1:-.} | rg -v "log|logger|sentry|capture" | head -30
   

   Flag error handlers that swallow exceptions without logging.

4. Audit metrics coverage:

   • Are HTTP endpoints instrumented (request count, latency, error rate)?
   • Are database queries instrumented (query latency, connection pool usage)?
   • Are external service calls instrumented (call count, latency, error rate)?
   • Are business metrics tracked (orders placed, emails sent, jobs completed)?
   • Are resource metrics available (CPU, memory, disk, connections)?

   rg -n -i "(counter|histogram|gauge|summary|metric|observe|inc\(|time\()" \
     -g '*.{py,ts,js,go}' ${1:-.} | head -30
   

   Identify endpoints and services with no metrics instrumentation.