How was the desynchronization caught — by Kafka partitions or between layers?

Desynchronization in Kafka can be detected mainly in two ways:

1. By Kafka partitions:

   • The offset in each partition is checked. If processing lags behind or messages are not arriving in a partition, it indicates desynchronization within Kafka.
   • Monitoring lag for each partition helps identify where exactly the desynchronization has accumulated.

2. Between layers (for example, between producer, broker, and consumer):

   • Logging and tracing messages at each layer help identify where delays or message losses occur.
   • Using correlation IDs or unique message identifiers allows matching events between layers and detecting desynchronization.

Thus, desynchronization is usually identified either by analyzing lag across Kafka partitions or through monitoring and tracing messages between system layers.