Sobes.tech
Middle+

What factors contribute to the correct termination of a node depending on the pod's priority?

sobes.tech AI

Answer from AI

Correct termination of a node is achieved through the following factors:

  1. Timeouts:
    • terminationGracePeriodSeconds: Defines the time during which the node will attempt to gracefully terminate the pods before forcefully destroying them. Default is 30 seconds.
    • pod.spec.terminationGracePeriodSeconds: You can override the timeout for a specific pod.
    # Example of a pod manifest with an increased termination timeout
    apiVersion: v1
    kind: Pod
    metadata:
      name: my-pod-with-grace-period
    spec:
      containers:
      - name: my-container
        image: my-image
      terminationGracePeriodSeconds: 60 # Increased timeout
    
  2. Signals:
    • Kubernetes sends the SIGTERM signal to the pod's containers.
    • The application inside the container should be able to catch SIGTERM and handle it properly (e.g., close current connections, save state).
    • If the application does not terminate upon SIGTERM within terminationGracePeriodSeconds, a SIGKILL is sent.
  3. Pod Priority (priorityClassName):
    • During eviction (e.g., resource shortage or node eviction), pods with lower priority are terminated first.
    • This ensures that critical (high-priority) pods run longer and have more time for proper termination if the node becomes unavailable.
    # Example of a pod manifest with a priority class
    apiVersion: v1
    kind: Pod
    metadata:
      name: my-priority-pod
    spec:
      containers:
      - name: my-container
        image: my-image
      priorityClassName: high-priority # Using a priority class
    
  4. PreStop Hooks:
    • Additional scripts or calls that are executed inside the container just before sending SIGTERM.
    • Can be used for synchronization, unregistering from services, or other tasks necessary for proper termination.
    # Example of a PreStop hook
    apiVersion: v1
    kind: Pod
    metadata:
      name: my-pod-with-prestop
    spec:
      containers:
      - name: my-container
        image: my-image
        lifecycle:
          preStop:
            exec:
              command: ["/bin/sh", "-c", "echo 'Stopping container...' && sleep 5 && exit 0"] # Example command
    
  5. Endpoint Management:
    • Kubernetes removes the pod's IP from the Endpoints object of the Service at the start of termination, so new traffic is not directed to the terminating pod.
    • This allows existing connections to finish within the grace period.

Together, these mechanisms allow Kubernetes to manage the pod termination process on a node, considering their priority, giving them a chance for proper shutdown, and avoiding forced destruction where possible.