Sobes.tech
Middle

How to speed up the build of a large project in CI? Where is the cache stored?

sobes.tech AI

Answer from AI

To speed up the build of a large project in CI, dependency caching and intermediate artifacts are used. This prevents rebuilding everything from scratch with each run.

Main approaches:

  • Caching dependency directories (e.g., node_modules, .m2 for Maven, vendor for Go).
  • Caching build results (e.g., object files, compiled libraries).
  • Using incremental builds if the build tool supports it.

Where the cache is stored:

  • Inside the CI system: most CI systems (GitLab CI, GitHub Actions, Jenkins) have built-in caching mechanisms that store the cache on their servers or in dedicated storage.
  • External storage: sometimes caches are stored in cloud storage (S3, GCS) or on local cache servers.

Example for GitLab CI:

cache:
  paths:
    - node_modules/
    - build/

This will save and restore the cache between builds, speeding up the process.