Each test is 5 questions with varying difficulty.
AI Prep covers AI Agents, Generative AI, ML Fundamentals, NLP & LLMs and a lot more, with adaptive tests and daily challenges. Fully offline on Android. Free to try, one-time unlock for lifetime access.
Exactly-once semantics (EOS) represents the gold standard in distributed messaging, ensuring that an event is processed exactly one time, even in the presence of network partitions, process crashes, or infrastructure failures. In 2026, as microservices and event-driven architectures become the default for high-scale systems, the ability to reason about state consistency across distributed boundaries is a critical differentiator for senior engineering roles. Interviewers focus on EOS to test a candidate's understanding of the fundamental trade-offs between performance, availability, and consistency. Junior candidates are expected to understand the basic definitions of at-least-once and at-most-once delivery. Senior candidates, however, must demonstrate deep knowledge of how to achieve EOS through idempotent operations, distributed transactions, and atomic state transitions. Mastery of this topic is essential for roles in backend engineering, data engineering, and distributed systems architecture, where data integrity is paramount.
Exactly-once semantics are the bedrock of financial systems, inventory management, and billing services where duplicate processing results in direct monetary loss or data corruption. In a distributed system, network failures are inevitable; therefore, achieving EOS is not about preventing failures, but about ensuring the system state remains consistent despite them. In 2026, the industry has shifted away from expensive two-phase commit (2PC) protocols toward more resilient patterns like the Transactional Outbox and idempotent message processing. This shift is driven by the need for high throughput in cloud-native environments. Interviewers use this topic to gauge if a candidate understands the 'Fallacies of Distributed Computing.' A weak candidate will suggest 'making the network reliable,' while a strong candidate will discuss how to handle retries, deduplication keys, and atomic database updates. This topic reveals whether a candidate can design systems that are 'fault-tolerant by design' rather than 'fault-avoidant,' which is a key trait of senior-level engineering talent.
The architecture for achieving exactly-once semantics typically involves a source-of-truth database and a message broker, linked by a change data capture (CDC) mechanism or a transactional outbox pattern. The flow ensures that state changes and event emissions are coupled atomically.
The application writes to the local DB and the Outbox table in one transaction. A relay process polls the Outbox and publishes to the broker. The consumer processes the message and records the processed ID in its own DB to ensure idempotency.
[Application]
↓
[Local Database]
↓ ↓
[Business] [Outbox Table]
↓
[CDC Relay]
↓
[Message Broker]
↓
[Consumer]
↓
[Idempotency Check]Write business state and events to the same database transaction. A separate process polls the outbox table to publish messages.
Trade-offs: Ensures atomicity but introduces slight latency in event delivery and requires polling or CDC.
Include a unique message ID in the payload. The consumer checks a persistent store before processing to ensure the ID hasn't been handled.
Trade-offs: Guarantees correctness but requires an additional storage lookup for every message.
Maintain processing state (like offsets) within the same storage as the output, allowing atomic commits.
Trade-offs: Highly efficient but binds the consumer to a specific storage technology.
| Reliability | Achieved through persistent outbox tables and idempotent consumer logic. Failure modes include broker downtime and DB partition issues. |
| Scalability | Scales by partitioning the outbox and using distributed idempotency stores like Redis or Cassandra. |
| Performance | Bottlenecks include DB write contention and network round-trips for idempotency checks. Throughput is optimized via batching. |
| Cost | Driven by storage requirements for idempotency keys and additional infrastructure for CDC relays. |
| Security | Requires securing the outbox table and ensuring only authorized services can consume/produce events. |
| Monitoring | Track consumer lag, duplicate event rates, and outbox processing latency. |
Technically, 'exactly-once' is an abstraction. It is achieved by combining at-least-once delivery with idempotent processing. The system ensures that the end-to-end effect is as if the message was processed exactly once, even if the underlying network delivers it multiple times.
They are often used interchangeably. 'Effectively-once' is a more accurate term because it highlights that while the system might perform multiple internal operations, the final state reflects exactly one successful execution.
2PC is a blocking protocol. It requires all participants to be available simultaneously, which significantly reduces system availability and throughput. In modern distributed systems, we prefer asynchronous patterns like the Transactional Outbox.
At-least-once guarantees that no message is lost, but duplicates are possible. Exactly-once adds a layer of deduplication or idempotency to ensure that those duplicates do not result in incorrect state changes.
It is extremely difficult. You need a persistent store to track the state of processed messages. Without a reliable store to record what has been processed, you cannot distinguish between a new message and a duplicate.
Kafka provides transactional APIs that allow for atomic writes across producer and consumer offsets. However, the application must still be designed to handle the idempotency of the business logic itself.
The message ID acts as a unique identifier that the consumer uses to check if the message has already been processed. It is the key to implementing the Idempotent Consumer pattern.
Dual-writes (writing to DB and broker separately) are prone to failure if the process crashes between the two writes. The transactional outbox ensures both are committed in a single atomic database transaction.
Use a TTL (Time-To-Live) on the keys in your idempotency store (like Redis). Set the TTL to be longer than the maximum possible retry window for your system.
If your idempotency store is unavailable, you cannot safely process messages without risking duplicates. In a high-consistency system, you should fail the transaction rather than risk data corruption.
No. It is only necessary when the business logic is not naturally idempotent. If your operation is 'set value to X', it is naturally idempotent. If it is 'increment value by X', it is not.
AI Prep covers AI Agents, Generative AI, ML Fundamentals, NLP & LLMs and a lot more, with adaptive tests and daily challenges. Fully offline on Android. Free to try, one-time unlock for lifetime access.