HTTP Keep-Alive

Reuse connections.

Overview

HTTP keep-alive reuses TCP connections across multiple requests rather than handshaking on every call. Without keep-alive, every HTTP request pays the TCP three-way handshake plus TLS negotiation; with it, those costs amortise across many requests on the same connection. Modern HTTP enables keep-alive by default; the discipline is tuning the timeouts and monitoring the connection pool so the savings actually land.

• Connection reuse. One TCP connection serves many sequential requests. Handshake cost amortises away.
• Lower latency. No handshake per request after the first. Response times drop measurably.
• Lower CPU. Fewer TLS negotiations. Compute capacity stretches further.
• Tunable timeout plus HTTP/1.1 default. KeepAliveTimeout controls idle-connection lifetime; HTTP/1.1 enables keep-alive by default and HTTP/2 makes it implicit.

The approach

Three habits make keep-alive produce real wins: enable connection pooling on clients, tune server-side timeouts to match request intervals, and monitor connection-pool health as a standing signal.

• HTTP client connection pool. Reusable client objects (urllib3, OkHttp, requests Session). Per-request clients defeat the point of keep-alive.
• Server KeepAliveTimeout. Match expected request interval. Too short defeats reuse; too long ties up server slots.
• Load balancer alignment. ALB, nginx, Envoy each have their own keep-alive settings. The chain only works when every hop agrees.
• Connection pool monitoring plus documented timeouts. Pool exhaustion surfaces as queuing; per-tier the timeout settings documented.

Why this compounds

Each reused connection saves a handshake. Across hundreds of millions of requests per day, the savings show up in latency, CPU, and capacity headroom. The team’s HTTP performance fluency deepens; new services inherit the conventions on day one.

• Lower latency. Reused connections skip the handshake. p50 and p99 both improve.
• Lower cost. Less TLS compute per request. CPU headroom translates to capacity savings.
• Better scale. Fewer connection establishments support higher throughput on the same capacity.
• Year-one investment, year-two habit. First tuning is investment. By year two, every new service ships with keep-alive on day one.