Sobes.tech
Senior

If one of the services is unavailable during enrichment (for example, the service with the user's full name/data), what happens? Does the entire Kafka stop?

sobes.tech AI

Answer from AI

If during data enrichment one of the services, such as a service with full name or user data, is unavailable, it does not cause the entire Kafka to stop. Kafka as a messaging system continues to operate because it is separated from the data processing and enrichment logic.

What happens in practice:

  • A consumer that reads messages from Kafka and calls the enrichment service may encounter an error or timeout when accessing the unavailable service.
  • Depending on the implementation, the consumer may:
    • Log the error and skip enrichment for this message,
    • Retry with exponential backoff,
    • Send the message to a DLQ (Dead Letter Queue) for later analysis,
    • Pause message processing if it is critical.

Thus, the failure of one service does not stop Kafka but may affect message processing in a specific consumer. To ensure system resilience, it is recommended to implement error and timeout handling, as well as use resilience patterns such as Circuit Breaker.

If one of the services is unavailable during… - sobes.tech