Tuesday, August 29, 2023

Rolling Upgrade and Rollback



Rollout newer version of nginx

Lets create a deployment with the name nginx

kubectl create deployment nginx --image=nginx:1.14.0
deployment.apps/nginx created

kubectl get pods
NAME                    READY   STATUS    RESTARTS   AGE
nginx-b6464dbd4-dt6m8   1/1     Running   0          8s

kubectl describe pod nginx
Name:         nginx-b6464dbd4-dt6m8
Namespace:    default
Priority:     0
Node:         node01/172.23.132.6
Start Time:   Tue, 05 Jul 2022 06:50:04 +0000
Labels:       app=nginx
              pod-template-hash=b6464dbd4
Annotations:  <none>
Status:       Running
IP:           10.244.1.10
IPs:
  IP:           10.244.1.10
Controlled By:  ReplicaSet/nginx-b6464dbd4
Containers:
  nginx:
    Container ID:   docker://a0bf6b89c06ea4f966a63560cf87a100add32d80b2e32a3888673cc97a3969c9
    Image:          nginx:1.14.0
    Image ID:       docker-pullable://nginx@sha256:8b600a4d029481cc5b459f1380b30ff6cb98e27544fc02370de836e397e34030
    Port:           <none>
    Host Port:      <none>
    State:          Running
      Started:      Tue, 05 Jul 2022 06:50:09 +0000
    Ready:          True
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-46q9z (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             True
  ContainersReady   True
  PodScheduled      True
Volumes:
  kube-api-access-46q9z:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type    Reason     Age   From               Message
  ----    ------     ----  ----               -------
  Normal  Scheduled  41s   default-scheduler  Successfully assigned default/nginx-b6464dbd4-dt6m8 to node01
  Normal  Pulling    40s   kubelet            Pulling image "nginx:1.14.0"
  Normal  Pulled     37s   kubelet            Successfully pulled image "nginx:1.14.0" in 3.381384908s
  Normal  Created    36s   kubelet            Created container nginx
  Normal  Started    36s   kubelet            Started container nginx

Now lets upgrade the nginx to nginx:1.15.0

kubectl set image deployment/nginx nginx=nginx:1.15.0
deployment.apps/nginx image updated

kubectl get pods
NAME                     READY   STATUS        RESTARTS   AGE
nginx-6489c547d7-5s4wh   1/1     Running       0          5s
nginx-b6464dbd4-dt6m8    1/1     Terminating   0          3m12s

kubectl get pods
NAME                     READY   STATUS    RESTARTS   AGE
nginx-6489c547d7-5s4wh   1/1     Running   0          13s

kubectl describe pod nginx
Name:         nginx-6489c547d7-5s4wh
Namespace:    default
Priority:     0
Node:         node01/172.23.132.6
Start Time:   Tue, 05 Jul 2022 06:53:11 +0000
Labels:       app=nginx
              pod-template-hash=6489c547d7
Annotations:  <none>
Status:       Running
IP:           10.244.1.11
IPs:
  IP:           10.244.1.11
Controlled By:  ReplicaSet/nginx-6489c547d7
Containers:
  nginx:
    Container ID:   docker://fa443b1e8986b01d678351ad21a2146de606e53d00e404b3bf1179cc16a4445f
    Image:          nginx:1.15.0
    Image ID:       docker-pullable://nginx@sha256:62a095e5da5f977b9f830adaf64d604c614024bf239d21068e4ca826d0d629a4
    Port:           <none>
    Host Port:      <none>
    State:          Running
      Started:      Tue, 05 Jul 2022 06:53:15 +0000
    Ready:          True
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-vrbjn (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             True
  ContainersReady   True
  PodScheduled      True
Volumes:
  kube-api-access-vrbjn:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type    Reason     Age   From               Message
  ----    ------     ----  ----               -------
  Normal  Scheduled  58s   default-scheduler  Successfully assigned default/nginx-6489c547d7-5s4wh to node01
  Normal  Pulling    57s   kubelet            Pulling image "nginx:1.15.0"
  Normal  Pulled     54s   kubelet            Successfully pulled image "nginx:1.15.0" in 3.315037526s
  Normal  Created    54s   kubelet            Created container nginx
  Normal  Started    54s   kubelet            Started container nginx

Now lets check the status

$ kubectl rollout status deployment/nginx
deployment "nginx" successfully rolled out

You can also check the rollout history

$ kubectl rollout history deployment/nginx
deployment.apps/nginx
REVISION  CHANGE-CAUSE
1         <none>
2         <none>

In-case if there are any issues while upgrading, you can rollback to previous version using

$ kubectl rollout undo deployment/nginx
deployment.apps/nginx rolled back

You can check that the revisions are also changing and increasing with the rollback

$ kubectl rollout history deployment/nginx
deployment.apps/nginx
REVISION  CHANGE-CAUSE
2         <none>
3         <none>

Check if the rollback is successful

$ kubectl describe pod nginx
Name:         nginx-b6464dbd4-7fnw5
Namespace:    default
Priority:     0
Node:         node01/172.23.132.6
Start Time:   Tue, 05 Jul 2022 06:57:08 +0000
Labels:       app=nginx
              pod-template-hash=b6464dbd4
Annotations:  <none>
Status:       Running
IP:           10.244.1.12
IPs:
  IP:           10.244.1.12
Controlled By:  ReplicaSet/nginx-b6464dbd4
Containers:
  nginx:
    Container ID:   docker://4c84def08651ccacd65c72fa6af00ab559d5833ccc94da215d3a319dcecc07cc
    Image:          nginx:1.14.0
    Image ID:       docker-pullable://nginx@sha256:8b600a4d029481cc5b459f1380b30ff6cb98e27544fc02370de836e397e34030
    Port:           <none>
    Host Port:      <none>
    State:          Running
      Started:      Tue, 05 Jul 2022 06:57:09 +0000
    Ready:          True
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-skt6j (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             True
  ContainersReady   True
  PodScheduled      True
Volumes:
  kube-api-access-skt6j:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type    Reason     Age   From               Message
  ----    ------     ----  ----               -------
  Normal  Scheduled  63s   default-scheduler  Successfully assigned default/nginx-b6464dbd4-7fnw5 to node01
  Normal  Pulled     62s   kubelet            Container image "nginx:1.14.0" already present on machine
  Normal  Created    62s   kubelet            Created container nginx
  Normal  Started    62s   kubelet            Started container nginx

You can rollback using revision 

$ kubectl rollout undo deployment/nginx --to-revision=2
deployment.apps/nginx rolled back

$ kubectl rollout status -w deployment/nginx
deployment "nginx" successfully rolled out

kubectl describe pod nginx
.

.

.
Containers:
  nginx:
    Container ID:   docker://5c067cef12b3275cf2ce6e641c60a9b5089f440f55c9bd58d53cc4a088cb9d31
    Image:          nginx:1.15.0

Events:
  Type    Reason     Age   From               Message
  ----    ------     ----  ----               -------
  Normal  Scheduled  11s   default-scheduler  Successfully assigned default/nginx-6489c547d7-vzfr2 to node01
  Normal  Pulling    10s   kubelet            Pulling image "nginx:1.15.0"
  Normal  Pulled     6s    kubelet            Successfully pulled image "nginx:1.15.0" in 4.481852846s
  Normal  Created    6s    kubelet            Created container nginx
  Normal  Started    6s    kubelet            Started container nginx