Trace Ratio by Percentile: A Useful Dashboard
The ratio of fast to slow traces by percentile reveals workload health. The panel and the trends to watch.
The panel
X-axis: time. Y-axis: percentage of traces in each percentile bucket.
Stacked area. Shape reveals workload changes.
Normal shape
Stable percentile distribution. The shape is consistent week over week.
Anomaly: sudden change in distribution shape. The mean might be stable but the shape shifted.
What to watch
Slow-trace percentage growing: workload getting harder. Maybe new feature, maybe regression.
Slow-trace percentage shrinking: optimisation paid off. Celebrate.