Kubernetes Basics

Kubernetes, also known as K8s, is an open-source cluster orchestration system for automating deployment, scaling, and management of containerized applications

Local Development

The Kubernetes command-line tool, kubectl (opens in a new tab), allows you to run commands against Kubernetes clusters. You can use kubectl to deploy applications, inspect and manage cluster resources, and view logs. To run Kubernetes on your local computer, you have some options:

• MiniKube (opens in a new tab) is local Kubernetes, focusing on making it easy to learn and develop for Kubernetes. All you need is Docker (or similarly compatible) container or a Virtual Machine environment, and Kubernetes is a single command away: minikube start.
• kind (opens in a new tab) is a tool for running local Kubernetes clusters using Docker container "nodes". kind was primarily designed for testing Kubernetes itself, but may be used for local development or CI. It lets you run Kubernetes on your local computer. This tool requires that you have either Docker or Podman installed. Command is kind create cluster to create a cluster, and kind delete cluster to delete it.
• kubeadm (opens in a new tab) You can use the kubeadm tool to create and manage Kubernetes clusters. It performs the actions necessary to get a minimum viable, secure cluster up and running in a user friendly way.

Kubernetes Concepts

Kubernetes coordinates a highly available cluster of computers that are connected to work as a single unit. Kubernetes automates the distribution and scheduling of application containers across a cluster in a more efficient way. A Kubernetes cluster consists of two types of resources:

• The Control Plane coordinates & manages the cluster
  • This includes scheduling applications, maintaining applications' desired state, scaling applications, and rolling out new updates.
• Nodes are the workers that run applications
  • A node is a VM or a physical computer that serves as a worker machine in a Kubernetes cluster.
  • Each node has a Kubelet, which is an agent for managing the node and communicating with the Kubernetes control plane. The node should also have tools for handling container operations, such as containerd or CRI-O.
  • Node-level components, such as the kubelet, communicate with the control plane using the Kubernetes API

Once you have a running Kubernetes cluster, you can deploy your containerized applications on top of it. To do so, you create a Kubernetes Deployment. The Deployment instructs Kubernetes how to create and update instances of your application. Once you've created a Deployment, the Kubernetes control plane schedules the application instances included in that Deployment to run on individual Nodes in the cluster.

Once the application instances are created, a Kubernetes Deployment controller continuously monitors those instances. If the Node hosting an instance goes down or is deleted, the Deployment controller replaces the instance with an instance on another Node in the cluster. This provides a self-healing mechanism to address machine failure or maintenance.

Kubernetes Pods

When you create a Deployment with kubectl create deployment, Kubernetes creates a "Pod" to host your application instance. A Pod is a Kubernetes abstraction that represents a group of one or more application containers (such as Docker), and some shared resources for those containers. Those resources include:

• Shared storage, as Volumes
• Networking, as a unique cluster IP address
• Information about how to run each container, such as the container image version or specific ports to use

A Pod models an application-specific "logical host" and can contain different application containers which are relatively tightly coupled. For example, a Pod might include both the container with your Node.js app as well as a different container that feeds the data to be published by the Node.js webserver. The containers in a Pod share an IP Address and port space, are always co-located and co-scheduled, and run in a shared context on the same Node.

Pods are the atomic unit on the Kubernetes platform. When we create a Deployment on Kubernetes, that Deployment creates Pods with containers inside them (as opposed to creating containers directly).

Nodes

A Pod always runs on a Node. A Node is a worker machine in Kubernetes and may be either a virtual or a physical machine, depending on the cluster. Each Node is managed by the control plane. A Node can have multiple pods, and the Kubernetes control plane automatically handles scheduling the pods across the Nodes in the cluster. The control plane's automatic scheduling takes into account the available resources on each Node.

Every Kubernetes Node runs at least:

• Kubelet, a process responsible for communication between the Kubernetes control plane and the Node; it manages the Pods and the containers running on a machine.
• A container runtime (like Docker) responsible for pulling the container image from a registry, unpacking the container, and running the application.

kubectl Basics

The common format of a kubectl command is: kubectl action resource. For example, to view the nodes in the cluster, run the kubectl get nodes command. Let’s deploy our first app on Kubernetes with the kubectl create deployment command. We need to provide the deployment name and app image location (include the full repository url for images hosted outside Docker Hub).

kubectl create deployment kubernetes-bootcamp --image=gcr.io/google-samples/kubernetes-bootcamp:v1

This performed a few things:

1. Searched for a suitable node where an instance of the application could be run (we have only 1 available node)
2. Scheduled the application to run on that Node
3. Configured the cluster to reschedule the instance on a new Node when needed

To list your deployments use the kubectl get deployments command.

$ kubectl get deployments
NAME                  READY   UP-TO-DATE   AVAILABLE   AGE
kubernetes-bootcamp   1/1     1            1           67s

We see that there is 1 deployment running a single instance of the app. The instance is running inside a container on the node. Pods that are running inside Kubernetes are running on a private, isolated network. By default they are visible from other pods and services within the same Kubernetes cluster, but not outside that network. When we use kubectl, we're interacting through an API endpoint to communicate with our application. We will cover other options on how to expose your application outside the Kubernetes cluster in the Exposing Your App section.

The kubectl proxy command can create a proxy that will forward communications into the cluster-wide, private network. The proxy can be terminated by pressing control-C and won't show any output while its running.

$ kubectl proxy
Starting to serve on 127.0.0.1:8001

We now have a connection between our host (the terminal) and the Kubernetes cluster. The proxy enables direct access to the API from these terminals. The API server will automatically create an endpoint for each pod, based on the pod name, that is also accessible through the proxy.

# First we need to get the Pod name,
POD_NAME=$(kubectl get pods -o go-template --template '{{range .items}}{{.metadata.name}}{{"\n"}}{{end}}')
echo Name of the Pod: $POD_NAME
curl http://localhost:8001/api/v1/namespaces/default/pods/$POD_NAME/

Overall, the most common operations can be done with the following kubectl subcommands:

• kubectl get - list resources
• kubectl describe - show detailed information about a resource
• kubectl logs - print the logs from a container in a pod
• kubectl exec - execute a command on a container in a pod

For example, we can use kubectl get pods to see which pods we have running, and kubectl describe pods to view what containers are inside that Pod and what images are used to build those containers

$ kubectl get pods
NAME                                   READY   STATUS    RESTARTS   AGE
kubernetes-bootcamp-855d5cc575-pcqcf   1/1     Running   0          27m

$ kubectl describe pods
Name:             kubernetes-bootcamp-855d5cc575-pcqcf
Namespace:        default
Priority:         0
Service Account:  default
Node:             minikube/172.26.166.10
Start Time:       Tue, 17 Oct 2023 12:12:28 +0100
Labels:           app=kubernetes-bootcamp
                  pod-template-hash=855d5cc575
...

Anything that the application would normally send to standard output becomes logs for the container within the Pod. We can retrieve these logs using the kubectl logs command:

$ kubectl logs $POD_NAME

apply manages applications through files defining Kubernetes resources. It creates and updates resources in a cluster through running kubectl apply. This is the recommended way of managing Kubernetes applications on production.

Executing Commands on the Container

We can execute commands directly on the container once the Pod is up and running. For this, we use the exec subcommand and use the name of the Pod as a parameter. Let’s list the environment variables:

$ kubectl exec "$POD_NAME" -- env

Next let’s start a bash session in the Pod’s container:

$ kubectl exec -ti $POD_NAME -- bash

We have now an open console on the container where we run our NodeJS application. The source code of the app is in the server.js file, which we can view using cat server.js for example. To close your container connection, type exit.

Services: Exposing Your App

Kubernetes Pods are mortal. Pods have a lifecycle. When a worker node dies, the Pods running on the Node are also lost. A ReplicaSet might then dynamically drive the cluster back to the desired state via the creation of new Pods to keep your application running. As another example, consider an image-processing backend with 3 replicas. Those replicas are exchangeable; the front-end system should not care about backend replicas or even if a Pod is lost and recreated. That said, each Pod in a Kubernetes cluster has a unique IP address, even Pods on the same Node, so there needs to be a way of automatically reconciling changes among Pods so that your applications continue to function.

A Service in Kubernetes is an abstraction which defines a logical set of Pods and a policy by which to access them. Services enable a loose coupling between dependent Pods. A Service is defined using YAML or JSON, like all Kubernetes object manifests. The set of Pods targeted by a Service is usually determined by a label selector.

Although each Pod has a unique IP address, those IPs are not exposed outside the cluster without a Service. Services allow your applications to receive traffic. Services can be exposed in different ways by specifying a type in the spec of the Service:

• ClusterIP (default) - Exposes the Service on an internal IP in the cluster. This type makes the Service only reachable from within the cluster.
• NodePort - Exposes the Service on the same port of each selected Node in the cluster using NAT. Makes a Service accessible from outside the cluster using <NodeIP>:<NodePort>. Superset of ClusterIP.
• LoadBalancer - Creates an external load balancer in the current cloud (if supported) and assigns a fixed, external IP to the Service. Superset of NodePort.
• ExternalName - Maps the Service to the contents of the externalName field (e.g. foo.bar.example.com), by returning a CNAME record with its value. No proxying of any kind is set up. This type requires v1.7 or higher of kube-dns, or CoreDNS version 0.0.8 or higher.

More information about the different types of Services can be found in the Using Source IP (opens in a new tab) tutorial. Also see Connecting Applications with Services (opens in a new tab).

Services match a set of Pods using labels and selectors, a grouping primitive that allows logical operation on objects in Kubernetes. Labels are key/value pairs attached to objects and can be used in any number of ways:

• Designate objects for development, test, and production
• Embed version tags
• Classify an object using tags

Labels can be attached to objects at creation time or later on. They can be modified at any time.

Creating a Service

Let's expose our application now using a Service and apply some labels. After verifying our application is still running with kubectl get pods, we can run kubectl get services to see what Services are currently running in our cluster.

$ kubectl get services
NAME         TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
kubernetes   ClusterIP   10.96.0.1    <none>        443/TCP   5m4s

We have a Service called kubernetes that is created by default when minikube/kind starts the cluster. To create a new service and expose it to external traffic we'll use the expose command with NodePort as parameter:

kubectl expose deployment/kubernetes-bootcamp --type="NodePort" --port 8080

Running kubectl get services again now returns to us:

$ kubectl get services
NAME                  TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)          AGE
kubernetes            ClusterIP   10.96.0.1      <none>        443/TCP          8m24s
kubernetes-bootcamp   NodePort    10.96.169.31   <none>        8080:30856/TCP   25s

We have now a running Service called kubernetes-bootcamp. Here we see that the Service received a unique cluster-IP, an internal port and an external-IP (the IP of the Node). To find out what port was opened externally (for the type: NodePort Service) we’ll run the describe service subcommand:

$ kubectl describe services/kubernetes-bootcamp
Name:                     kubernetes-bootcamp
Namespace:                default
Labels:                   app=kubernetes-bootcamp
Annotations:              <none>
Selector:                 app=kubernetes-bootcamp
Type:                     NodePort
IP Family Policy:         SingleStack
IP Families:              IPv4
IP:                       10.102.110.86
IPs:                      10.102.110.86
Port:                     <unset>  8080/TCP
TargetPort:               8080/TCP
NodePort:                 <unset>  30452/TCP
Endpoints:                10.244.0.3:8080
Session Affinity:         None
External Traffic Policy:  Cluster
Events:                   <none>

Using Labels

The Deployment created automatically a label for our Pod. With the kubectl describe deployment subcommand you can see the name (the key) of that label:

$ kubectl describe deployment
Name:                   kubernetes-bootcamp
Namespace:              default
CreationTimestamp:      Tue, 17 Oct 2023 16:58:15 +0100
Labels:                 app=kubernetes-bootcamp

We can use labels to query the pods we have. For example, we can use the kubectl get pods -l command followed by the label values

kubectl get pods -l app=kubernetes-bootcamp

Or with the list of services:

kubectl get services -l app=kubernetes-bootcamp

We can also apply new labels using the kubectl label command followed by the object type, object name and the new label:

POD_NAME="$(kubectl get pods -o go-template --template '{{range .items}}{{.metadata.name}}{{"\n"}}{{end}}')"
echo "Name of the Pod: $POD_NAME
kubectl label pods "$POD_NAME" version=v1

This will apply a new label to our Pod (we pinned the application version to the Pod), and we can check it with the describe pod command kubectl describe pods "$POD_NAME" or we can now query the list of pods using the new label:

kubectl get pods -l version=v1

Deleting a Service

To delete Services you can use the delete service subcommand. Labels can also be used here:

kubectl delete service -l app=kubernetes-bootcamp

Scaling

The Deployment created only one Pod for running our application. When traffic increases, we will need to scale the application to keep up with user demand. Scaling is accomplished by changing the number of replicas in a Deployment

Scaling out a Deployment will ensure new Pods are created and scheduled to Nodes with available resources. Scaling will increase the number of Pods to the new desired state. Kubernetes also supports autoscaling of Pods. Scaling to zero is also possible, and it will terminate all Pods of the specified Deployment.

Running multiple instances of an application will require a way to distribute the traffic to all of them. Services have an integrated load-balancer that will distribute network traffic to all Pods of an exposed Deployment. Services will monitor continuously the running Pods using endpoints, to ensure the traffic is sent only to available Pods.

Scaling is accomplished by changing the number of replicas in a Deployment.

Once you have multiple instances of an application running, you would be able to do Rolling updates without downtime. Starting again from the get deployments command:

$ kubectl get deployments
NAME                  READY   UP-TO-DATE   AVAILABLE   AGE
kubernetes-bootcamp   1/1     1            1           17h

Here:

• NAME: lists the names of the Deployments in the cluster.
• READY: shows the ratio of CURRENT/DESIRED replicas
• UP-TO-DATE: displays the number of replicas that have been updated to achieve the desired state.
• AVAILABLE: displays how many replicas of the application are available to your users.
• AGE: displays the amount of time that the application has been running.

To see the ReplicaSet created by the Deployment, run kubectl get rs. Notice that the name of the ReplicaSet is always formatted as [DEPLOYMENT-NAME]-[RANDOM-STRING]. The random string is randomly generated and uses the pod-template-hash as a seed.

$ kubectl get rs
NAME                             DESIRED   CURRENT   READY   AGE
kubernetes-bootcamp-855d5cc575   1         1         1       17h

Two important columns of this output are:

• DESIRED: displays the desired number of replicas of the application, which you define when you create the Deployment. This is the desired state.
• CURRENT: displays how many replicas are currently running.

Next, let’s scale the Deployment to 4 replicas. We’ll use the kubectl scale command, followed by the Deployment type, name and desired number of instances:

$ kubectl scale deployments/kubernetes-bootcamp --replicas=4
deployment.apps/kubernetes-bootcamp scaled

Now if we run kubectl get deployments, we see:

$ kubectl get deployments
NAME                  READY   UP-TO-DATE   AVAILABLE   AGE
kubernetes-bootcamp   4/4     4            4           17h

We have 4 instances of the application available. Similarly, if we view the number of pods:

$ kubectl get pods -o wide
NAME                                   READY   STATUS    RESTARTS      AGE   IP           NODE       NOMINATED NODE   READINESS GATES
kubernetes-bootcamp-855d5cc575-d5xds   1/1     Running   0             62s   10.244.0.7   minikube   <none>           <none>
kubernetes-bootcamp-855d5cc575-gsdjx   1/1     Running   0             62s   10.244.0.8   minikube   <none>           <none>
kubernetes-bootcamp-855d5cc575-pv4ft   1/1     Running   0             63s   10.244.0.6   minikube   <none>           <none>
kubernetes-bootcamp-855d5cc575-zsj2p   1/1     Running   1 (59m ago)   17h   10.244.0.5   minikube   <none>           <none>

There are 4 Pods now, with different IP addresses. The change was registered in the Deployment events log. To check that, use the describe subcommand, kubectl describe deployments/kubernetes-bootcamp

To scale back down to 2 replicas for example, we can run the scale command again:

$ kubectl scale deployments/kubernetes-bootcamp --replicas=2
deployment.apps/kubernetes-bootcamp scaled

Now:

$ kubectl get deployments
NAME                  READY   UP-TO-DATE   AVAILABLE   AGE
kubernetes-bootcamp   2/2     2            2           18h

This confirms that 2 Pods were terminated.

Rolling Updates

Rolling updates allow Deployments' update to take place with zero downtime by incrementally updating Pods instances with new ones. The new Pods will be scheduled on Nodes with available resources.

Scaling is a requirement for performing updates without affecting application availability. By default, the maximum number of Pods that can be unavailable during the update and the maximum number of new Pods that can be created, is one. Both options can be configured to either numbers or percentages (of Pods). In Kubernetes, updates are versioned and any Deployment update can be reverted to a previous (stable) version.

Similar to application Scaling, if a Deployment is exposed publicly, the Service will load-balance the traffic only to available Pods during the update. An available Pod is an instance that is available to the users of the application.

Rolling updates allow the following actions:

1. Promote an application from one environment to another (via container image updates)
2. Rollback to previous versions
3. Continuous Integration and Continuous Delivery of applications with zero downtime

For example, lets update our deployment. kubectl describe pods will describe our pods that we have running, and will include a Image field. To update the image of the application to version 2, use the set image subcommand, followed by the deployment name and the new image version:

$ kubectl set image deployments/kubernetes-bootcamp kubernetes-bootcamp=jocatalin/kubernetes-bootcamp:v2
deployment.apps/kubernetes-bootcamp image updated

The command notified the Deployment to use a different image for your app and initiated a rolling update. We can view the old pods terminating:

$ kubectl get pods
NAME                                   READY   STATUS        RESTARTS      AGE
kubernetes-bootcamp-69b6f9fbb9-2h6pl   1/1     Running       0             30s
kubernetes-bootcamp-69b6f9fbb9-cv7wk   1/1     Running       0             33s
kubernetes-bootcamp-855d5cc575-pv4ft   1/1     Terminating   0             17m
kubernetes-bootcamp-855d5cc575-zsj2p   1/1     Terminating   1 (75m ago)   18h

Verify an Update

You can confirm the update by running the rollout status subcommand:

$ kubectl rollout status deployments/kubernetes-bootcamp
deployment "kubernetes-bootcamp" successfully rolled out

And again, we can view the running image version with the kubectl describe pods command.

Rollback an Update

Say we update again our app to a version that doesn't actually exist:

$ kubectl set image deployments/kubernetes-bootcamp kubernetes-bootcamp=gcr.io/google-samples/kubernetes-bootcamp:v10
deployment.apps/kubernetes-bootcamp image updated

Now if we view the Pods:

$ kubectl get pods
NAME                                   READY   STATUS         RESTARTS   AGE
kubernetes-bootcamp-66566cb7f-qvqcd    0/1     ErrImagePull   0          32s
kubernetes-bootcamp-69b6f9fbb9-2h6pl   1/1     Running        0          29m
kubernetes-bootcamp-69b6f9fbb9-cv7wk   1/1     Running        0          29m

Some of the Pods have a status of ErrImagePull. To get more insight into the problem, run the describe pods subcommand:

$ kubectl describe pods
Events:
  Type     Reason     Age                From               Message
  ----     ------     ----               ----               -------
  Normal   Scheduled  63s                default-scheduler  Successfully assigned default/kubernetes-bootcamp-66566cb7f-qvqcd to minikube
  Normal   Pulling    25s (x3 over 63s)  kubelet            Pulling image "gcr.io/google-samples/kubernetes-bootcamp:v10"
  Warning  Failed     24s (x3 over 62s)  kubelet            Failed to pull image "gcr.io/google-samples/kubernetes-bootcamp:v10": rpc error: code = Unknown desc = Error response from daemon: manifest for gcr.io/google-samples/kubernetes-bootcamp:v10 not found: manifest unknown: Failed to fetch "v10" from request "/v2/google-samples/kubernetes-bootcamp/manifests/v10".
  Warning  Failed     24s (x3 over 62s)  kubelet            Error: ErrImagePull
  Normal   BackOff    10s (x3 over 62s)  kubelet            Back-off pulling image "gcr.io/google-samples/kubernetes-bootcamp:v10"
  Warning  Failed     10s (x3 over 62s)  kubelet            Error: ImagePullBackOff

In the Events section of the output for the affected Pods, notice that the v10 image version did not exist in the repository. To roll back the deployment to your last working version, use the rollout undo subcommand:

$ kubectl rollout undo deployments/kubernetes-bootcamp
deployment.apps/kubernetes-bootcamp rolled back

The rollout undo command reverts the deployment to the previous known state (v2 of the image). Updates are versioned and you can revert to any previously known state of a Deployment. The Deployment is once again using a stable version of the app (v2). The rollback was successful. Remember to clean up your local cluster aswell:

kubectl delete deployments/kubernetes-bootcamp services/kubernetes-bootcamp

Custom Resources

Custom resources (opens in a new tab) are extensions of the Kubernetes API. Imagine you have a set of building blocks for making different types of toys. Now, you want to create some unique toys that aren't part of the standard set. So, you decide to make your own custom building blocks that fit with the standard ones. This is a bit like what Custom Resources are in Kubernetes.

Custom Resources in Kubernetes are like special building blocks that you create to extend the capabilities of Kubernetes. They allow you to define and manage new types of objects in your Kubernetes cluster, beyond the built-in resources like pods, services, and deployments.

Controllers: To make your custom resource do something useful, you often need to create controllers. Controllers are like the toy-making machines in our analogy. They watch for changes to your custom resource instances and take action accordingly. For example, if you have a custom resource for databases, a controller could ensure that a database server is created whenever you define a new instance of your custom resource.

Operator pattern

Operators are software extensions to Kubernetes that make use of custom resources (opens in a new tab) to manage applications and their components. Operators follow Kubernetes principles, notably the control loop (opens in a new tab)

Imagine you have a robot that can do certain tasks automatically without your direct involvement, like making your morning coffee or watering plants. This robot is programmed to understand your preferences and perform these tasks efficiently. This is somewhat similar to the concept of an Operator in Kubernetes.

Operator Pattern in Kubernetes is a way to make your cluster smarter and more self-sufficient. It's like having specialized robots, called "Operators," that understand how to manage and maintain complex applications or services running on Kubernetes without constant human intervention.

Just like you teach your robot about your coffee preferences, in Kubernetes, you define Custom Resources (CRs). These CRs represent your application's requirements and configuration. For example, if you have a database application, you create a custom resource that specifies how your database should look and behave.

Kind: Kubernetes in Docker

kind is a tool for running local Kubernetes clusters using Docker container "nodes". kind was primarily designed for testing Kubernetes itself, but is really useful for local development or CI. If you have go 1.16+ and docker or podman installed go install sigs.k8s.io/kind@v0.20.0 && kind create cluster is all you need!

Helm: The Package Manager for Kubernetes

Helm is a tool for managing Charts. Charts are packages of pre-configured Kubernetes resources. Use Helm to:

• Find and use popular software packaged as Helm Charts (opens in a new tab) to run in Kubernetes
• Share your own applications as Helm Charts
• Create reproducible builds of your Kubernetes applications
• Intelligently manage your Kubernetes manifest files
• Manage releases of Helm packages

Helm is a tool that streamlines installing and managing Kubernetes applications. Think of it like apt/yum/homebrew for Kubernetes.

• Helm renders your templates and communicates with the Kubernetes API
• Helm runs on your laptop, CI/CD, or wherever you want it to run.
• Charts are Helm packages that contain at least two things:
  • A description of the package (Chart.yaml)
  • One or more templates, which contain Kubernetes manifest files
• Charts can be stored on disk, or fetched from remote chart repositories (like Debian or RedHat packages)

To rapidly get Helm up and running, start with the Quick Start Guide (opens in a new tab).

Resources:

• Kubernetes Tutorials (opens in a new tab)
• MiniKube (opens in a new tab)
• kind (opens in a new tab)
• Helm: The Package Manager for Kubernetes (opens in a new tab)