log-strategy

Log Strategy

Most teams either log too little to investigate anything or too much to find anything (and end up storing PII they didn't mean to). This skill is the middle path: enough to support incident response, not so much that the log becomes a liability of its own.

Pairs with incident-response (where logs get used) and gdpr-technical-controls (where log privacy lives).

When to invoke

• Starting a new service or workflow
• Investigation revealed missing fields ("we don't know what request triggered this")
• Logs are leaking PII or secrets
• Log volume is becoming expensive / unwieldy
• Consolidating to a central logging stack (Loki, ELK, OpenObserve, hosted)
• After an incident where the log was insufficient

Three classes of log, three retention tiers

Treat your logs as three distinct streams. Conflating them is where the trouble starts.

Class	Purpose	Typical retention	Storage tier
Operational	Debugging, performance, errors	7–30 days	Hot index
Access	Who hit what, when (webserver + app request log)	30–90 days	Hot + warm
Audit	Security-relevant events (auth, permission changes, sensitive actions)	12 months+	Append-only, immutable where possible

Costs and tools differ per class. Mixing them gives the audit log's slow-and-expensive retention to the operational stream and is the most common reason logging gets killed by finance.

Operational logs — what to capture

Structured logging beats freeform every time. Choose JSON (or logfmt) and a single library for the whole app.

// pino example — Node
import pino from 'pino';

const log = pino({
  level: process.env.LOG_LEVEL ?? 'info',
  redact: {
    paths: ['password', 'token', 'authorization', '*.password', '*.token', 'req.headers.authorization', 'req.headers.cookie'],
    censor: '[REDACTED]',
  },
  serializers: {
    req: (req) => ({
      method: req.method,
      url: req.url,
      requestId: req.id,
      // explicitly NOT: headers (except a few safe ones), body
    }),
    err: pino.stdSerializers.err,
  },
});

log.info({ orderId, userId: anonymize(userId) }, 'order created');
log.warn({ requestId, latencyMs }, 'slow query');
log.error({ requestId, err }, 'unhandled error in handler');