How to make all instances read the same messages?

To ensure that all instances of an application read the same messages, a centralized messaging system or message broker is usually used, such as Apache Kafka, RabbitMQ, or JMS. In this case, all instances subscribe to the same queue or topic and receive identical messages.

If we're talking about Kafka, to have all instances read the same messages, they can be configured into a single consumer group, but then messages will be distributed among them (each message will be read by only one instance). To have all instances read all messages, each must have a unique consumer group id, so each instance will receive all messages from the topic.

Here's an example in Java with Kafka Consumer, where each instance reads all messages:

Properties props = new Properties();
props.put("bootstrap.servers", "localhost:9092");
props.put("group.id", UUID.randomUUID().toString()); // unique group id for each instance
props.put("key.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
props.put("value.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");

KafkaConsumer<String, String> consumer = new KafkaConsumer<>(props);
consumer.subscribe(Collections.singletonList("my-topic"));

while (true) {
    ConsumerRecords<String, String> records = consumer.poll(Duration.ofMillis(100));
    for (ConsumerRecord<String, String> record : records) {
        System.out.printf("offset = %d, key = %s, value = %s%n", record.offset(), record.key(), record.value());
    }
}

Thus, each instance will receive all messages from the topic.