Exactly-once ends at the edge of the cluster

"We enabled exactly-once, so we cannot get duplicates." I have heard this in design reviews, and it is the most expensive misunderstanding in event-driven Java systems. Kafka's exactly-once semantics are real. They are also narrower than the name suggests. ## What EOS actually gives you Two things. The idempotent producer. The broker tracks a producer ID and a sequence number per partition, so a retried send writes the record only once. Transactions. The offsets you commit and the records you produce land atomically across partitions. With isolation.level=read_committed on the consumer, you get a correct read-process-write loop.

properties
enable.idempotence=true
transactional.id=payments-processor-1
isolation.level=read_committed

Both guarantees stop at the edge of the cluster. ## Where it breaks The moment your consumer does something Kafka does not know about — a database insert, a call to a payment gateway, an S3 write — that side effect is outside the transaction. It has no producer ID. It has no sequence number. Kafka cannot roll it back, because Kafka cannot see it. So: your consumer processes the message, writes to Postgres, and crashes before the offset commit. The offset is unchanged. The message is redelivered. Kafka has not broken a single promise it made. You have a duplicate row. ## The guarantee you actually need It is idempotency at the destination, and Kafka cannot give it to you. In practice that means one of: - A unique constraint on a natural idempotency key. - A processed-messages table, written inside the same database transaction as the business change. - The transactional outbox, when the write is also the event source.

java
@Transactional
public void handle(PaymentEvent event) {
    // Same transaction as the business write. If this insert violates the
    // unique constraint, the whole thing rolls back and redelivery costs a
    // wasted cycle instead of a second payment.
    processed.insert(event.id());
    ledger.credit(event.accountId(), event.amount());
}

The pattern I keep coming back to: put the dedup key and the business write in one database transaction, and treat the offset commit as best-effort. Redelivery then costs a wasted cycle rather than a duplicate payment, and you stop needing the offset commit to be atomic with anything. ## The habit Kafka gives you exactly-once inside Kafka. Everything past that boundary is your design problem. So the question worth asking in the review is not "is EOS enabled?" It is: where is the boundary in this system, and does the consumer know it is there?