Each test is 5 questions with varying difficulty.
AI Prep covers AI Agents, Generative AI, ML Fundamentals, NLP & LLMs and a lot more, with adaptive tests and daily challenges. Fully offline on Android. Free to try, one-time unlock for lifetime access.
Change Data Capture (CDC) is a design pattern for determining and tracking data changes in a database so that action can be taken using the extracted data. In 2026, CDC is the backbone of modern real-time data pipelines, enabling seamless synchronization between transactional databases and downstream systems like data warehouses, search indexes, and cache layers. For Data Engineers, ML Engineers, and Backend Architects, mastering CDC is essential for building decoupled, event-driven architectures. Interviewers focus on CDC to test a candidate's understanding of database internals, transaction integrity, and distributed systems reliability. Junior candidates are typically expected to understand the difference between polling and log-based approaches, while senior candidates must demonstrate expertise in handling schema evolution, managing transaction log offsets, ensuring exactly-once delivery, and mitigating the performance impact on production databases.
CDC is critical because it solves the 'dual write' problem in distributed systems. Without CDC, developers often attempt to update a database and emit an event to a message broker separately, which leads to data inconsistency if one operation fails. CDC provides a source-of-truth mechanism by reading the database's own transaction logs, ensuring that every committed change is captured and propagated downstream. In 2026, as organizations move toward real-time AI and analytics, the latency between a database update and the availability of that data in a vector store or analytical engine has become a competitive differentiator. A strong interview answer on CDC reveals a candidate's grasp of how databases manage durability and atomicity. A weak answer often relies on 'polling'—a naive approach that adds unnecessary load to the primary database and introduces significant latency. Understanding CDC demonstrates that a candidate prioritizes system stability, low-latency data movement, and the decoupling of microservices.
CDC architectures operate by intercepting the database's commit stream. A connector (like Debezium) acts as a client to the database, reading the WAL or binlog entries, converting them into a structured event format (JSON/Avro), and pushing them to a message bus. Downstream consumers then process these events to update secondary data stores.
The database writes to the log, the connector reads the log, the connector publishes to the broker, and the sink consumes from the broker.
[Source Database]
↓
[Transaction Log]
↓
[CDC Connector]
↓
[Schema Registry]
↓
[Message Broker]
↓
[Downstream Sink]Write business data and event data to the same DB in a single transaction, then use a relay to push events.
Trade-offs: Ensures atomicity but adds latency and requires extra table management.
Directly tailing database binary logs to capture changes with minimal overhead.
Trade-offs: High performance and low latency but requires specific DB configuration and permissions.
Designing downstream consumers to handle the same event multiple times without side effects.
Trade-offs: Increases complexity in consumer logic but crucial for exactly-once semantics.
| Reliability | Use replication slots to prevent log truncation; implement dead-letter queues for failed events. |
| Scalability | Partition Kafka topics by primary key to ensure ordered processing across multiple consumers. |
| Performance | Minimize log parsing overhead by using dedicated CDC connectors and tuning WAL/binlog retention. |
| Cost | CDC increases storage and network costs; optimize by filtering unnecessary columns or tables. |
| Security | Restrict CDC connector database access to read-only; encrypt data in transit and at rest. |
| Monitoring | Monitor 'offset lag' to detect pipeline delays and 'connector status' for uptime. |
Polling involves periodically querying the database for changes (e.g., SELECT * WHERE updated_at > last_check), which is resource-intensive and misses deletes. CDC uses log-based extraction to capture every change in real-time with minimal database impact.
Yes, log-based CDC captures all operations recorded in the transaction log, including DELETE statements, which is impossible with standard polling techniques that rely on timestamps or version columns.
The Outbox pattern is used when native log-based CDC is unavailable. It involves writing business data and an event record to an 'outbox' table in the same transaction, ensuring that the event is only emitted if the business data is saved.
Modern CDC tools use a Schema Registry to track versions of data structures. When a schema changes, the connector updates the registry, allowing downstream consumers to adapt their deserialization logic accordingly.
CDC is near real-time. The latency depends on the database transaction commit speed, the connector's log reading frequency, and the message broker's throughput, usually resulting in sub-second to few-second delays.
A replication slot is a server-side object that ensures the database retains WAL (Write Ahead Log) files until they are successfully consumed by the CDC connector, preventing data loss during network outages.
Log-based CDC has a negligible impact because it reads binary logs directly from disk or memory, bypassing the SQL execution engine. However, improper configuration or excessive log retention can impact storage and I/O.
Exactly-once delivery is a guarantee that each change event is processed exactly once by the downstream system. It is achieved through transactional producers, idempotent consumers, and careful offset management.
Avoid CDC if the database does not support logical decoding, if the operational overhead of managing a Kafka cluster is too high for your scale, or if your data requirements are strictly batch-oriented and infrequent.
Consistency is maintained by ensuring the CDC pipeline preserves the order of operations from the transaction log and by using idempotent downstream consumers to handle potential retries or duplicates.
AI Prep covers AI Agents, Generative AI, ML Fundamentals, NLP & LLMs and a lot more, with adaptive tests and daily challenges. Fully offline on Android. Free to try, one-time unlock for lifetime access.