How will Kubernetes update secrets if I have six pod replicas with the same secret?
sobes.tech AI
Answer from AI
Kubernetes does not update secrets directly while the pod is running. Pods use a snapshot of the secret that existed at the time of their launch.
To have pods "see" the updated secret, they need to be restarted or recreated. Common deployment strategies for this include:
-
Rolling Update: Old pods are gradually replaced with new ones with the updated secret. This is the standard strategy for Deployments.
-
Recreate: All old pods are deleted simultaneously, and new ones are created. This results in a short service downtime.
-
Blue/Green Deployment: A completely new version of the application (with the updated secret) is deployed in parallel with the old one. After verification, all traffic is switched to the new version, and the old one is removed.
-
Canary Release: A new version of the application (with the updated secret) is deployed to a small subset of users, gradually increasing traffic to it.
When using ConfigMaps and Secrets mounted as files, changes in the secret can be automatically updated inside an already running pod, but this occurs with some delay (kubernetes.io/docs/concepts/configuration/secret/#using-secrets-as-files-from-a-pod). However, to use a secret, for example, in environment variables, the pod needs to be restarted.
Example of updating a Deployment using Rolling Update:
apiVersion: apps/v1
kind: Deployment
metadata:
name: my-app
spec:
replicas: 6
selector:
matchLabels:
app: my-app
strategy:
type: RollingUpdate # Standard strategy
rollingUpdate:
maxUnavailable: 1 # Maximum number of unavailable pods during update
maxSurge: 1 # Maximum number of new pods created above the desired replicas
template:
metadata:
labels:
app: my-app
spec:
containers:
- name: my-container
image: my-image:latest
env:
- name: MY_SECRET_VALUE
valueFrom:
secretKeyRef:
name: my-secret # Secret name
key: my-key # Secret key
After changing the my-secret secret, you need to update the Deployment (for example, by changing the image version or adding an annotation) to trigger a Rolling Update:
kubectl apply -f deployment.yaml
Kubernetes will detect changes in the Deployment spec and start creating new pods that will access the updated secret. Old pods will be terminated.