We need to intercept all requests in Kafka and send them to an audit or telemetry system. How would you do it?

To intercept all requests in Kafka and send them to an audit or telemetry system, several approaches can be used:

1. Using Kafka Interceptor API — Kafka provides the ProducerInterceptor and ConsumerInterceptor interfaces, which allow intercepting messages during sending and receiving. These interceptors can implement logic to send data to the audit system.

public class AuditProducerInterceptor implements ProducerInterceptor<String, String> {
    @Override
    public ProducerRecord<String, String> onSend(ProducerRecord<String, String> record) {
        // Send data to the audit system
        auditService.send(record);
        return record;
    }

    @Override
    public void onAcknowledgement(RecordMetadata metadata, Exception exception) {}
    @Override
    public void close() {}
    @Override
    public void configure(Map<String, ?> configs) {}
}

2. Proxy service — create an intermediate service that receives messages from producers, sends them to Kafka, and simultaneously duplicates them to the audit system.

3. Using Kafka Connect with Sink Connector — configure a connector that reads from a topic and sends data to the audit system.

4. Processing on the consumer side — if auditing is important for consumption, you can add audit sending logic in the consumer code.

The most transparent and scalable option for Java applications is to use ProducerInterceptor and ConsumerInterceptor, as they are built into the Kafka client and do not require architecture changes.