Can you describe your experience with Kafka system and its application in projects?

Kafka is a distributed data streaming platform that is often used for processing large volumes of messages in real-time. In my projects, I used Kafka to facilitate message exchange between microservices and for collecting logs and metrics.

Example of usage with Python using the kafka-python library:

from kafka import KafkaProducer, KafkaConsumer

# Sending a message
producer = KafkaProducer(bootstrap_servers='localhost:9092')
producer.send('my_topic', b'Hello Kafka')
producer.flush()

# Receiving messages
consumer = KafkaConsumer('my_topic', bootstrap_servers='localhost:9092')
for message in consumer:
    print(f"Received message: {message.value.decode('utf-8')}")

This approach allows building scalable and fault-tolerant systems with asynchronous data processing.