Senior
What to do if the test only fails on CI?
sobes.tech AI
Answer from AI
If a test fails only on CI but passes locally, you should:
- Check the CI environment: there might be different software versions, environment variables, settings, or access rights.
- Review logs and artifacts: understand at which step the test fails, and whether there are errors or timeouts.
- Verify dependencies: databases, external services that might be unavailable or behave differently on CI.
- Add additional logging or state snapshots: to better understand the problem.
- Check parallelism: tests might conflict with each other on CI.
- Stabilize the test: add waits, remove dependencies on time or state.
- Run tests locally in an environment as close as possible to CI: for example, in a Docker container with the same environment.
This approach helps identify and eliminate causes of test instability on CI.