Kubernetes networking basics
Enabling inter-pod and pod-to-pod communication across nodes (CNI)
Kubernetes does not provide a way for containers across nodes to communicate with each other, it assumes that each container (pod) has a unique, routable IP inside the cluster.
To facilitate inter-container connectivity across nodes, any networking solution based on Pure Layer-3 or VxLAN or UDP model, can be used. A CNI (Containier Network Interface) plugin need to be installed to enable that communication. Flannel is one such solution which provides an overlay network using UDP as well as VxLAN based model.
A CNI plugin is responsible for inserting a network interface into the container network namespace (e.g., one end of a virtual ethernet (veth) pair) and making any necessary changes on the host (e.g., attaching the other end of the veth into a bridge). It then assigns an IP address to the interface and sets up the routes consistent with the IP Address Management section by invoking the appropriate IP Address Management (IPAM) plugin.
Enabling Kuberentes Services load balancing
Kubernetes Service is an abstract way to expose an application running on a set of Pods as a network service. Kubernetes gives Pods their own IP addresses and a single DNS name (and single Virtual IP address) for a set of Pods, and can load-balance across them. Two components are key for this mechanism: kube-proxy in charge of implementing the Virtual IP address and load balance mechanism and kube-dns responsible for mapping DNS service name to virtual IP address.
See details in Kubernetes documentation - Service Concept.
kube-proxy: Kuberentes Services internal Load-balancing
kube-proxy is a key component of any Kubernetes deployment. Its role is to load-balance traffic that is destined for services (via Cluster IPs and Node Ports) to the correct backend pods.
Kube-proxy can run in one of three modes, each implemented with different data plane technologies available (user-space, iptables, or IPVS).
User-space mode is legacy implementation not in use . Iptables proxy mode is the defatult mode since Kubernetes v1.2 and IPVS mode has been introduced in Kubernetes v1.8 (GA in v1.11). iptables and IPVS both use the operating system packet filtering layer (netfilter).
In iptables mode, kube-proxy watches the Kubernetes control plane for the addition and removal of Service and Endpoint objects. For each Service, it installs iptables rules, which capture traffic to the Service’s clusterIP and port, and redirect that traffic to one of the Service’s backend sets. For each Endpoint object, it installs iptables rules which select a backend Pod.
By default, kube-proxy in iptables mode chooses a backend at random. See more details in IPVS documentation
As a summary:
- A service is a collection of pods, which each have their own IP address (like 10.1.0.3, 10.2.3.5, 10.3.5.6).
- Every Kubernetes service gets an IP address (like 10.23.1.2)
- CNI plugin is in charge to allocate the POD addresess and ensure that that the pods are routable
- kube-dns resolves Kubernetes service DNS names to IP addresses (so my-svc.my-namespace.svc.cluster.local might map to 10.23.1.2)
- kube-proxy sets up iptables rules in order to do random load balancing between them.
So when request to my-svc.my-namespace.svc.cluster.local is made, it resolves to 10.23.1.2, and then iptables rules on the node (generated by kube-proxy) redirect it to one of 10.1.0.3 or 10.2.3.5 or 10.3.5.6 at random.
Routing incoming HTTP/HTTPS traffic
Ingress exposes HTTP and HTTPS routes from outside the cluster to services within the cluster. Traffic routing is controlled by rules defined on the Ingress resource.
An Ingress may be configured to give Services externally-reachable URLs, load balance traffic, terminate SSL / TLS, and offer name-based virtual hosting. An Ingress Controller is the component responsible for fulfilling the Ingress rules. Ingress Controller need to be deployed within the cluster usually with an external load balancer to handle the incoming traffic.
Every Ingress resource belong to IngressClass resource that contains information about the Ingress Controller implementing the class. This way Ingresses can be implemented by different controllers.
See detailed information in Kubernetes documentation - Ingress Contoller Concept.