Connection Warmup

Warm pools; cold-start.

Overview

Connection warmup pre-establishes connections so the first real request does not pay the connection cost. Cold-start latency in modern stacks comes from TCP three-way handshake, TLS handshake, TCP slow-start congestion-window ramp, and HTTP/2 stream initialization. Each of these is a real round-trip cost that the user pays unless the connection is already warm. The discipline is to pre-warm on service startup, keep-alive during idle to preserve the warm state, and monitor tail latency to catch when warmup breaks.

• Warm connection pool. Connections pre-established at startup; the first real request does not pay handshake cost.
• TCP slow-start mitigation. TCP starts with a small congestion window; pre-grown windows on warm connections avoid the slow-start ramp.
• TLS session resumption. Cached TLS state via session tickets; resumption skips the full handshake.
• HTTP/2 connection priming plus cold-start elimination. First HTTP/2 connection ramps up settings; pre-warmup removes first-request cost across the stack.

The approach

The practical approach is to pre-warm connection pools on service startup (synthetic requests through the pool establish the connections), enable keep-alive during idle to prevent connection close (idle timeout longer than the typical inter-request gap), enable TLS session tickets across instances so horizontal scale preserves resumption, monitor p99 and p99.9 latency to catch warmup regressions, and document the per-service warmup sequence in the service repo.

• Pre-warm on startup. Service startup pre-establishes connections via synthetic warmup requests; cold-request cost lands at startup, not during real traffic.
• Keep-alive during idle. Idle keep-alive longer than typical inter-request gap; preserves the warm state through quiet periods.
• TLS session tickets. Cached TLS state across instances; horizontal scale benefits from shared resumption state.
• Monitor tail latency plus documented warmup. p99 and p99.9 reveal warmup issues; per-service startup sequence committed to the repo.

Why this compounds

Connection warmup compounds across services. Each warmed connection produces consistent latency where cold connections would spike; the team builds intuition for connection-cost patterns that pays off on every new service. Without the discipline, tail latency stays unpredictable and the team chases spikes that warmup would have prevented.

• Lower tail latency. Warm connections remove handshake spikes; the user experiences consistent responsiveness.
• Cold-start handling. Pre-warmup removes first-request cost; auto-scaling new instances do not produce a latency wave.
• Lower CPU on TLS. Session resumption reduces TLS handshake CPU; the same hardware serves more requests.
• Institutional knowledge. Each tail-latency investigation teaches connection patterns; the team learns where warmup matters.

Connection warmup is an operational discipline that pays off across years. Nova AI Ops integrates with latency telemetry, surfaces warmup patterns, and supports the team’s performance discipline.