Is it possible to send messages of different formats and data types in Kafka topics?
sobes.tech AI
Answer from AI
In Apache Kafka, topics are message logs where each message is a sequence of bytes. Kafka itself does not impose restrictions on the format or data type of messages, so different formats and types can be sent in the same topic.
However, it is recommended to adhere to a single message format within a topic for easier processing and data consistency. If different message types are needed, usually separate topics are created or serialization schemes with type indicators are used.
Example: using Avro or Protobuf with a schema that includes a type field to distinguish messages, or simply serializing different objects into JSON and sending them to one topic, with consumers capable of handling different formats.
Thus, technically possible, but architecturally and for maintenance, it is better to avoid mixing formats in one topic.