Sobes.tech
Senior

What to do if the test only fails on CI?

sobes.tech AI

Answer from AI

If a test fails only on CI but passes locally, you should:

  1. Check the CI environment: there might be different software versions, environment variables, settings, or access rights.
  2. Review logs and artifacts: understand at which step the test fails, and whether there are errors or timeouts.
  3. Verify dependencies: databases, external services that might be unavailable or behave differently on CI.
  4. Add additional logging or state snapshots: to better understand the problem.
  5. Check parallelism: tests might conflict with each other on CI.
  6. Stabilize the test: add waits, remove dependencies on time or state.
  7. Run tests locally in an environment as close as possible to CI: for example, in a Docker container with the same environment.

This approach helps identify and eliminate causes of test instability on CI.

What to do if the test only fails on CI? — QA / QA… - sobes.tech