Retroactive Instrumentation: When You Need More Detail
Sometimes you need detail you did not capture. The pattern of retroactive instrumentation: add it now, replay later.
The limit
Cannot capture data that was not emitted. The current incident's missing detail is gone forever.
But: similar future incidents will have the data.
What to do
Add the instrumentation now. The incident postmortem includes the instrumentation as an action item.
Verify in pre-prod that the new data lands correctly.
Compound
Each incident teaches what to add. Over years, instrumentation matches the actual debug needs.
Year 1: incident-driven additions. Year 3: most useful data is already captured before incidents.