Sobes.tech
Middle

How to change Kafka cluster parameters (for example, the number of replicas for a topic) and what happens next with the clients?

sobes.tech AI

Answer from AI

To change Kafka cluster parameters, such as the number of replicas for a topic, you need to use the kafka-topics.sh command with the --alter option. For example, to change the number of replicas, perform:

kafka-topics.sh --zookeeper <zookeeper_host>:<port> --alter --topic <topic_name> --replication-factor <new_replication_factor>

However, it is important to note that changing the replication factor is not always trivial and requires that there are enough brokers in the cluster to host the new replicas. When increasing the number of replicas, Kafka will start creating new copies of partitions on other brokers.

What happens to clients:

  • Clients that read from the topic do not require a restart — they automatically receive updated partition and replica metadata during the next metadata refresh.
  • Producers also automatically update their information about partitions and replicas.

Thus, changing topic parameters occurs at the cluster level, and clients adapt dynamically without manual intervention.

Important: In newer Kafka versions, it is recommended to use the Kafka Admin API or kafka-topics with bootstrap-server instead of Zookeeper, as Zookeeper is gradually being deprecated.

How to change Kafka cluster parameters (for example… - sobes.tech