monitoring-observability

Skill: monitoring-observability

Cursor Agent-Requested Rule — applied when AI determines relevance.

---

Monitoring & Observability - See What Your System Is Doing

You can't fix what you can't see. Observability is debugging production.

---

The Three Pillars

┌─────────────────────────────────────────┐
│           OBSERVABILITY                 │
├─────────────┬─────────────┬─────────────┤
│   METRICS   │    LOGS     │   TRACES    │
├─────────────┼─────────────┼─────────────┤
│  Numerical  │   Events    │  Request    │
│   data      │   over      │   flows     │
│  over time  │   time      │             │
├─────────────┼─────────────┼─────────────┤
│  "What is   │  "What      │  "Where is  │
│ happening?" │ happened?"  │ the issue?" │
└─────────────┴─────────────┴─────────────┘

---

Logging

Log Levels

Level	Use Case	Example
ERROR	Failures requiring intervention	Database connection failed
WARN	Unexpected but handled	Retry attempt 3/5
INFO	Normal operations	User login successful
DEBUG	Development troubleshooting	Variable values
TRACE	Detailed execution flow	Function entry/exit

Structured Logging

# ✅ GOOD — Structured, queryable logs
import logging
import json

logger = logging.getLogger(__name__)

logger.info(
    "User action completed",
    extra={
        "user_id": user.id,
        "action": "purchase",
        "item_id": item.id,
        "amount": 99.99,
        "duration_ms": 245
    }
)
# Output: {"message": "User action completed", "user_id": 123, "action": "purchase", ...}

# ❌ BAD — Unstructured, hard to query
logger.info(f"User {user.id} purchased item {item.id} for {amount}")

Log Best Practices

• Correlation IDs: Track requests across services
• Context: Include relevant identifiers
• No Sensitive Data: Never log passwords, tokens
• Rate Limiting: Prevent log flooding

---

Metrics