Who writes messages in Kafka? How does message format change occur?

In Kafka, messages are usually written by producers — applications or services that send data to Kafka topics.

Changing the message format occurs through versioning the data schema. This is often done using schema management systems, such as Confluent Schema Registry, where the schema version is stored (e.g., Avro, Protobuf, or JSON Schema).

The process of changing the message format:

1. A new schema version with changes is created.
2. Producers are updated to use the new schema.
3. Consumers are updated to correctly handle both versions or migrate to the new one.

This ensures backward compatibility and prevents failures when the message structure changes.

Example: if initially a message contains fields id and name, and then an email field is added, the new schema version will include email as an optional field so that old consumers can continue to work.