Secrets Management Lifecycle
Secrets: birth to death. The lifecycle.
Creation
Secrets have a lifecycle: created, used, rotated, retired. Each stage has its own discipline. Most teams handle creation and use reasonably well; the rotation and retirement stages are where most teams fall short. Treating the lifecycle as a unified discipline produces secrets management that holds up over years.
What good creation discipline looks like:
- Auto-generated, not human-chosen.: Secrets are produced by cryptographically secure random generators. Humans choosing passwords or keys produce predictable patterns; entropy from human-chosen secrets is inadequate. The generator produces secrets at appropriate length and randomness for their use.
- Stored in vault, not anywhere else.: Created secrets go directly into a secret store (Vault, AWS Secrets Manager, GCP Secret Manager). They are not stored in tickets, in chat messages, in email, in shared documents. The vault is the single authoritative location.
- Never plaintext in transit.: The secret is never sent over a channel that does not encrypt. Email is out (most email is plaintext somewhere along the path). SMS is out. Internal tools without TLS are out. The transmission uses encrypted channels exclusively.
- Bound to scope at creation.: The secret's scope (what it can access, what permissions it has) is determined at creation. The scope is recorded in the vault metadata. Future operations on the secret reference the scope.
- Audit logged from creation.: The creation event is logged with timestamp, creator, recipient, scope. The audit trail begins at creation and continues throughout the lifecycle. Future investigation can trace the secret back to its origin.
Creation is the foundation. Bad creation produces secrets that are hard to manage; good creation produces secrets that flow cleanly through the rest of the lifecycle.
Use
The use phase is where most secrets spend their lives. Applications fetch them, use them to authenticate, return them to the cache. The discipline at this stage is about minimizing the surface area where the secret exists in plaintext.
- Apps fetch at startup.: The application requests secrets from the vault at startup, holds them in memory, uses them. The fetch is over an authenticated, encrypted channel. The plaintext exists in application memory only for as long as the application needs it.
- Refresh on schedule.: Long-running applications refresh secrets periodically. If the secret rotates, the refresh picks up the new value. The application stays current without restart.
- Workload identity preferred.: Where possible, replace static secrets with workload identity. The application authenticates with its identity (IRSA, GCP Workload Identity, Vault auth methods); the vault returns short-lived credentials. The static secret is replaced by the dynamic one.
- No plaintext in environment variables long-term.: Environment variables are convenient but they are visible to other processes on the same host, dumped in crash reports, logged by misconfigured loggers. Secrets in env vars are a deliberate trade-off; the trade-off is acceptable for ephemeral runtime injection but not for long-term storage.
- Audit access to vault.: Every fetch of a secret is logged. The audit trail shows which application accessed which secret when. Anomalies (a service fetching a secret it has not touched in months, a fetch from an unusual source) trigger investigation.
The use phase is mostly invisible when done well; it surfaces only in the audit logs that confirm the use is appropriate.
Rotation
Rotation is the discipline most teams skip. Secrets that never rotate accumulate risk: every additional minute the same secret is in use is a minute of additional compromise risk. Auto-rotation removes the human from the rotation loop and makes the discipline sustainable.
- Auto-rotate on schedule.: The vault rotates the secret on a configured schedule (30 to 90 days for static secrets, hours for dynamic secrets). The rotation is automatic; the application's refresh picks up the new value; the team's involvement is configuration, not execution.
- Versioned for rollback.: When a secret rotates, the old version is retained for a transition period. If the rotation breaks something, rollback is fast: revert to the previous version, investigate the break, fix and re-rotate. Without versioning, a bad rotation requires generating yet another new value.
- Downtime-free rotation.: The rotation is designed to not require downtime. The transition period during which both old and new values are valid lets consumers update without coordinated cutover. The seamless rotation is what makes auto-rotation viable in production.
- Cascade through dependents.: Some secrets are used in many places. The rotation propagates to all consumers; the dependency tree is exercised. Tools like External Secrets Operator handle this propagation automatically.
- Retirement after rotation.: Once the rotation is verified, the old secret value is invalidated. The retirement is part of the lifecycle; secrets that "rotated but the old value still works" produce false security.
Secrets management lifecycle done right is mostly invisible. The team configures the policies; the platform handles the operations. Nova AI Ops integrates with secret stores across the lifecycle, surfaces secrets that are overdue for rotation, and produces the audit trail that makes compliance discussions short.
Destruction
After rotation: old version destroyed.
No long tail.