Memory Allocator Choice

jemalloc vs system.

Overview

Memory allocator choice matches allocator to workload. Long-running processes fragment differently than short-lived ones; latency-sensitive services prefer different allocators than throughput-bound ones; blanket adoption is the wrong approach.

• jemalloc. Lower fragmentation; favoured by long-running processes; the default for Redis, Rust, and Firefox for a reason.
• tcmalloc. Lower latency; thread-cached; favoured by latency-sensitive services; Google’s default for high-fanout systems.
• System malloc. Simple; modern glibc malloc is competitive; the right choice when nothing else is needed.
• Per-language defaults. Go has its own; Rust uses system by default; Java has its GC; the runtime constrains the choice.

The approach

The practical approach: profile first, switch deliberately, document the choice. Allocator change is high-leverage when the workload matches; without profiling, the change is theatre.

• Profile first. Measure RSS growth, fragmentation ratio, allocation rate; the data tells you whether allocator is the bottleneck.
• jemalloc for long-running. Services that run for days; fragmentation accumulates without it; the swap is one library preload.
• tcmalloc for latency-sensitive. High-fanout services; thread cache cuts allocation latency; verify with p99 measurement.
• Per-language awareness. Go runtime owns its own allocator; Rust prefers system; Java has GC concerns; respect the runtime.
• Document the choice. Per-process rationale committed to the repo; supports operational reviews and future tuning.

Why this compounds

Allocator discipline compounds across services. Each correctly-matched allocator produces ongoing performance; the team’s runtime expertise grows; new services inherit the muscle.

• Better stability. Right allocator avoids fragmentation; long-running processes do not need scheduled restarts to reclaim memory.
• Better latency. Right allocator reduces tail latency; thread caches and arena strategies cut p99 allocation cost.
• Better resource utilisation. Right allocator reduces memory waste; the cloud bill thanks you for the allocator pick.
• Institutional knowledge. Each profile teaches allocator patterns; the team’s performance engineering muscle grows.

Allocator discipline is an engineering discipline that pays off across years. Nova AI Ops integrates with runtime telemetry, surfaces patterns, and supports the team’s performance discipline.