Kafka19 interview questions

Kafka Exactly-Once Semantics in Financial Systems

Idempotent producers, transactional APIs, and how to achieve exactly-once processing in payment pipelines. Includes failure modes and recovery patterns.

KafkaTransactionsIdempotencyEOS

Kafka Exactly-Once Semantics in Financial Systems

The Delivery Guarantees

At-most-once: Messages may be lost, never duplicated
At-least-once: No message loss, duplicates possible
Exactly-once: No loss, no duplicates (Kafka 0.11+)

Idempotent Producer

java

Properties props = new Properties();
props.put(ProducerConfig.ENABLE_IDEMPOTENCE_CONFIG, "true");
props.put(ProducerConfig.ACKS_CONFIG, "all");
props.put(ProducerConfig.MAX_IN_FLIGHT_REQUESTS_PER_CONNECTION, "5");

The broker assigns a Producer ID (PID) and sequence numbers. Duplicate sends are detected and deduplicated.

Transactional Producer

java

props.put(ProducerConfig.TRANSACTIONAL_ID_CONFIG, "payment-processor-1");
KafkaProducer<K, V> producer = new KafkaProducer<>(props);
producer.initTransactions();

try {
    producer.beginTransaction();

    producer.send(new ProducerRecord<>("payments.processed", payment));
    producer.send(new ProducerRecord<>("audit.log", auditEvent));

    // Commit offsets atomically with the transaction
    producer.sendOffsetsToTransaction(offsets, consumerGroupMetadata);
    producer.commitTransaction();
} catch (Exception e) {
    producer.abortTransaction();
}

Consumer Configuration

java

props.put(ConsumerConfig.ISOLATION_LEVEL_CONFIG, "read_committed");

Interview Questions

1.What is the difference between idempotent and transactional producers?
2.How does Kafka prevent zombie transactions?
3.What is the performance cost of exactly-once semantics?
4.How do you handle exactly-once when writing to an external database?