CDC: Change Data Capture

Stream DB changes.

Overview

Change Data Capture (CDC) streams database changes to downstream consumers. The dual-write alternative looks simpler on paper and breaks under partial failure; CDC produces real consistency by reading the database’s own write-ahead log.

• Stream DB changes. Per-row change event; the database is the source; downstream stays in sync without dual-write.
• Debezium for Postgres/MySQL. Industry-standard CDC; reads the WAL/binlog; the modern default.
• Replication slot for Postgres. Persistent change cursor; durability across consumer restarts; resumable from any point.
• Per-table schema plus consumer lag. Per-table change format supports schema evolution; per-consumer lag tolerance defines SLOs.

The approach

The practical approach: Debezium for the source connector, monitor slot lag as a first-class metric, document the schema in a registry. The team’s discipline produces CDC that survives the first schema change.

• Debezium default. Reads Postgres WAL or MySQL binlog; the industry-standard CDC stack; battle-tested.
• Monitor slot lag. Per-slot consumer lag; an unbounded slot fills the disk and breaks the database.
• Per-table schema. Change format defined per table; supports schema evolution without breaking consumers.
• Schema registry. Per-stream schema registry (Confluent, Apicurio); consumers validate against the registered schema.
• Document the integration. Per-stream consumer named; supports operational reviews and the "who breaks if I change this?" lookup.

Why this compounds

CDC discipline compounds across streams. Each integrated stream supports modern event-driven architecture; the team’s CDC expertise accrues; new integrations get cheaper.

• Better integration. CDC produces real-time event flow; downstream consumers react in seconds, not the next batch window.
• Better consistency. CDC avoids dual-write; the database is the source; partial-failure inconsistency disappears.
• Better operational fit. Right consumer per workload; analytics, search, cache invalidation all consume the same stream.
• Institutional knowledge. Each slot teaches CDC patterns; the team’s event-driven engineering muscle grows.

CDC discipline is a database investment that pays off across years. Nova AI Ops integrates with CDC telemetry, surfaces patterns, and supports the team’s database engineering discipline.