Sobes.tech
Middle+

Imagine you have a critical service supported by your team, which depends on another team's API. At some point, the service starts returning a 500 error, and clients complain. What would you do?

sobes.tech AI

Answer from AI

  1. Service log check. Studying logs will help identify the nature of the error, its occurrence time, and possible stack traces.
  2. Service metrics monitoring. Check load graphs, resource consumption (CPU, RAM), request counts, and errors. This will help understand the scale of the problem and its dynamics.
  3. Check the availability and metrics of the dependent API. Use monitoring tools to check the API status of another team. Look at error counts, response delay metrics, and overall availability.
  4. Network accessibility check. Ensure that the service can establish a connection with the API.
  5. API contract check. Make sure that the request and response formats match expectations. Perhaps another team has made changes.
  6. Contact the team responsible for the API. Provide information about the problem, logs, and metrics. Coordinate diagnostic efforts.
  7. Analysis of recent changes. Check if there have been any deployments or infrastructure changes in our service or the dependent API.
  8. Find a workaround. While the API team works on fixing the issue, look for ways to minimize impact on clients, such as temporary data caching or fallback logic.
  9. Notify stakeholders about the problem. Inform management and other teams/clients who may be affected.

Example of log diagnostics:

// Search for 500 errors in service logs from the last hour
grep "HTTP/1.1\" 500" /var/log/my_service/access*.log | tail

// Search for specific error in application logs
grep "upstream connect error or disconnect/reset before headers" /var/log/my_service/app.log

Example of API availability check:

// Check API availability using cURL
curl -v -I https://api.other-team.com/some_endpoint