What complex tasks have you had to solve in the field of network interaction between containers?

Optimization of network interaction performance between services in a mesh network based on Istio. It was necessary to identify and eliminate bottlenecks related to Service Proxy (Envoy) overhead, mTLS configuration, and policy engine. This included analyzing Istio metrics (request duration, error count, delays), tracing distributed requests, and tuning Envoy configuration.

Troubleshooting traffic routing issues in dynamically scalable Kubernetes clusters. When scaling services, delays occurred in propagating EndpointSlices and load balancing was incorrect. This was resolved by configuring more aggressive caching timeouts in kube-proxy and using more advanced load balancing algorithms on the ingress controller (e.g., least_request).

Implementing secure interaction between containers in different subnets with strict network policy rules. It was necessary to configure policies to allow only the required traffic between specific pod or namespace pairs, while blocking all others. This required a deep understanding of Network Policy selectors and their application mechanisms by third-party CNI plugins (e.g., Calico).

Debugging name resolution (DNS) issues inside containers, especially in environments with multiple DNS servers or complex redirection rules. It involved analyzing traffic at the pod level using tcpdump and checking the /etc/resolv.conf configuration in containers, as well as the CoreDNS configuration in the Kubernetes cluster.

# Example of DNS traffic analysis inside a pod
kubectl exec <pod-name> -- tcpdump -i any udp port 53 -v

# Example NetworkPolicy to allow traffic between two namespaces
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-frontend-to-backend
  namespace: frontend-namespace
spec:
  podSelector:
    matchLabels:
      app: frontend
  policyTypes:
    - Egress
  egress:
    - to:
      - podSelector:
          matchLabels:
            app: backend
        namespaceSelector:
          matchLabels:
            name: backend-namespace

Addressing complexities related to using non-standard protocols that are not supported "out of the box" at the Service Mesh or CNI level, requiring custom proxy configurations or specialized solutions. For example, organizing interaction over a protocol different from HTTP/1.1 or gRPC within Istio, which might have required writing custom Envoy filters.