How did you set up a Kubernetes cluster from scratch on-premise? What tools did you use?

Setting up an on-premise Kubernetes cluster from scratch typically involves several stages and choosing tools based on requirements and infrastructure.

1. Infrastructure Preparation:

   • Setting up physical or virtual servers with the necessary OS (usually Linux).
   • Configuring network, DNS, load balancers.

2. Installing Kubernetes:

   • Using tools to simplify installation, such as:
     • kubeadm — the official tool for cluster initialization.
     • kops or Rancher — for more comprehensive management.
     • Kubespray — Ansible playbooks for automation.

3. Configuring components:

   • Initializing the master (kubeadm init).
   • Adding workers (kubeadm join).
   • Installing a network plugin (Calico, Flannel, Weave) for network communication.
   • Configuring storage (e.g., Ceph, NFS) for Persistent Volumes.

4. Installing and configuring management tools:

   • Helm — package manager for Kubernetes.
   • Prometheus and Grafana — for monitoring.
   • Fluentd or ELK — for logging.

5. Security and access:

   • Setting up RBAC.
   • Configuring TLS certificates.
   • Integrating with LDAP/Active Directory if necessary.

Example of a basic cluster initialization command:

kubeadm init --pod-network-cidr=192.168.0.0/16

After that, a network plugin like Calico is installed:

kubectl apply -f https://docs.projectcalico.org/manifests/calico.yaml

In conclusion, the choice of tools depends on scale and requirements, but kubeadm with a network plugin and additional monitoring and security tools is a standard approach for on-premise clusters.