secrets-management

Secrets Management

Purpose

Keep credentials out of code, out of logs, and out of git history — and be able to rotate them quickly when, inevitably, one leaks.

When to Use

• Setting up secret storage for an application.
• A secret has been committed or exposed.
• Implementing rotation.
• Auditing a repository or a running system for exposed credentials.

Capabilities

• Secret storage: cloud secret managers, Vault, sealed secrets.
• Injection: environment, mounted files, SDK-based retrieval.
• Rotation, including zero-downtime rotation of database credentials.
• Leak detection in code, history, logs, and error reports.
• Incident response for an exposed secret.

Inputs

• The secrets an application needs, and their blast radius if leaked.
• The runtime and how it can receive them.
• The rotation capability of each upstream provider.

Outputs

• No secrets in the repository, the image, or the logs.
• Secrets injected at runtime from a managed store.
• A rotation procedure that has been tested.

Workflow

1. Eliminate static credentials first — Prefer workload identity: IAM roles, OIDC federation, managed identities. A secret that does not exist cannot leak. This is the single most effective change available.
2. Store what remains in a secret manager — Never in code, never in a config file in the repository, never in a container image layer.
3. Inject at runtime — Mounted file or environment variable, fetched from the manager at start. The application never contains the value.
4. Redact in logs — Structured logging with a redaction filter on known secret field names, and never logging the full request body of an auth endpoint.
5. Scan continuously — A pre-commit hook and a CI scan on the full history. Detection after the fact is far better than not detecting it.
6. Rotate on a schedule and on exposure — And test the rotation before you need it in an emergency.

Best Practices

• A secret committed to git is compromised the moment it is pushed, regardless of whether the repository is private, whether it was force-pushed away, or whether anyone noticed. Rotate it. Deleting the commit is not a remediation.
• Environment variables are visible in /proc, in crash dumps, in some error reporters, and to any process running as the same user. A mounted file with restricted permissions is stronger.
• Rotation must be zero-downtime: support two valid credentials simultaneously (issue the new one, deploy, revoke the old one). A rotation that requires an outage will not be done.
• Never pass a secret as a command-line argument. It is visible in ps to every user on the host.
• Do not build a secret into a container image, even in a "private" registry. Image layers are extractable with docker history and are cached in many places.
• Short-lived, dynamically generated credentials (Vault's database secrets engine, IAM database authentication) make rotation continuous and leak impact minimal.