What to do when the system degrades (latency increased from 50ms to a minute)? How to diagnose and what to share during retro?

In case of a sudden increase in latency from 50ms to a minute, it is urgent to conduct diagnostics to identify the cause of degradation.

Diagnostics:

• Check metrics and logs: CPU, memory, GC, request count, errors.
• Analyze request tracing to understand at which stage the delay occurs.
• Check the status of dependencies: databases, external services, queues.
• Evaluate changes in code or infrastructure that could have caused the problem.
• Use profiling tools (e.g., Java Flight Recorder) to identify bottlenecks.

What to share in the retrospective:

• Description of the problem and its impact on users.
• Steps taken for diagnostics and resolution.
• Identified causes (e.g., memory leak, locks, slow queries).
• Measures taken to prevent recurrence (monitoring, alerts, optimizations).
• Plan to improve processes and tools for faster response in the future.