How do you invalidate in-memory caches on different pods? How do you synchronize caches between different service instances?

To invalidate in-memory cache on different pods and synchronize caches between service instances, the following approaches are usually used:

1. Using a centralized message broker (e.g., Redis Pub/Sub, Kafka)

   • When data changes, the service publishes an invalidation event to a channel.
   • All pods subscribe to this channel and, upon receiving a message, reset or update their local cache.

2. Using a distributed cache (e.g., Redis, Memcached)

   • Instead of a local cache, store data in a distributed cache so that all instances access a single source.

3. Using TTL (time-to-live) for cache

   • Cache automatically expires after a set period, reducing the need for manual invalidation.

4. Coordinated update via a coordination service (e.g., ZooKeeper, etcd)

   • Store cache state and update signals so that all instances can synchronize.

Example with Redis Pub/Sub:

// When updating data
func updateData() {
    // update data in DB
    // ...
    // publish invalidation event
    redisClient.Publish(ctx, "cache_invalidate_channel", "key_to_invalidate")
}

// In each pod
func subscribeInvalidation() {
    sub := redisClient.Subscribe(ctx, "cache_invalidate_channel")
    ch := sub.Channel()
    for msg := range ch {
        key := msg.Payload
        localCache.Delete(key) // reset local cache
    }
}

Thus, when data changes, all pods learn about the need to update or reset the cache and maintain consistency.