What is the practical application of DaemonSet in Kubernetes?

A DaemonSet ensures that a copy of a pod is running on all (or a specified subset of) nodes in the cluster. This is useful for deploying system daemons that must run on each node.

Practical applications:

• Log collection: Running a log collection agent (e.g., Fluentd, Logstash) on each node to forward logs to a centralized storage.
• Node monitoring: Deploying a monitoring agent (e.g., Prometheus Node Exporter, Datadog Agent) on each node to collect node status metrics.
• Cluster storage: Running a storage daemon (e.g., Ceph, Glusterfs) on each node to provide distributed storage.
• Security agents: Deploying security or audit agents on each node.

Difference from Deployment: A DaemonSet creates a pod on each node, whereas a Deployment aims to maintain a specified number of replicas across the entire cluster, distributing them across available nodes.

Example DaemonSet manifest for a log collection agent:

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: fluentd-elasticsearch
  labels:
    k8s-app: fluentd-logging
spec:
  selector:
    matchLabels:
      name: fluentd-elasticsearch
  template:
    metadata:
      labels:
        name: fluentd-elasticsearch
    spec:
      containers:
      - name: fluentd-elasticsearch
        image: fluentd/fluentd-kubernetes-daemonset:v1.14-debian-elasticsearch7-1
        env:
        - name:  FLUENT_ELASTICSEARCH_HOST
          value: "elasticsearch.default.svc.cluster.local" # Example Elasticsearch address
        - name:  FLUENT_ELASTICSEARCH_PORT
          value: "9200"
        - name: FLUENT_ELASTICSEARCH_SCHEME
          value: "http"
        volumeMounts:
        - name: varlog
          mountPath: /var/log
        - name: varlibdockercontainers
          mountPath: /var/lib/docker/containers
          readOnly: true
      terminationGracePeriodSeconds: 30
      volumes:
      - name: varlog
        hostPath:
          path: /var/log
      - name: varlibdockercontainers
        hostPath:
          path: /var/lib/docker/containers

DaemonSet automatically adds a pod to new nodes and removes it when a node is removed from the cluster.