domino logo
Latest (5.6)
  • About Domino
  • Architecture
  • Security and Compliance
  • Domino Infrastructure
  • Kubernetes Infrastructure
  • Installation
  • Data Planes
  • Execution Resources
  • Users
  • Keycloak Authentication Service
  • Data
  • Projects
  • Environments
  • Model Monitoring
  • Control Center
  • Logs and Monitoring
  • Backup and Restore
  • Disaster Recovery
  • Configuration Reference
  • Run the Admin Toolkit
  • MongoDB Console
  • Get Help
  • Send Feedback
domino logo
About Domino
Domino Data LabKnowledge BaseData Science BlogTraining
Admin Guide
>
Installation
>
Azure Deployments
>
Kubernetes Upgrade Guide

Kubernetes Upgrade Guide

This topic describes how to upgrade Kubernetes in your Azure Kubernetes Service (AKS) Domino deployment. AKS is hosted on Azure.

Important
Prerequisites

To upgrade Kubernetes on Azure, you must have a Unix or Linux terminal with the following:

  • Azure CLI installed.

  • Kubectl installed.

Use environment variables to set the values of IDs, names, and labels. This simplifies the commands you’ll run when you upgrade:

export SUB_ID=<Id of the subscription where AKS was deployed>
export RG_NAME=<Name of the resource group where AKS was deployed>
export CLUSTER_NAME=<The name of the cluster where domino is deployed>
Get cluster credentials
  1. If you aren’t already signed in to Azure CLI, run:

    az login
  2. Protect your current kubeconfig from being overwritten:

    export KUBECONFIG=$(pwd)/kubeconfig
  3. Get the remote kubeconfig:

    az aks get-credentials --subscription $SUB_ID --resource-group $RG_NAME --name $CLUSTER_NAME -f $KUBECONFIG
Disable the Disruption Budgets

You must temporarily disable the PodDisruptionBudget (PDB) for certain services. To do this, back up the PDBs, delete the original PDBs, then update Kubernetes and restore the PDBs from the backup after the update is complete.

  1. Back up the PDBs:

    kubectl get pdb -n domino-platform -o yaml mongodb-arbiter > mongodb-arbiter.yml
    kubectl get pdb -n domino-platform -o yaml mongodb-primary  > mongodb-primary.yml
    kubectl get pdb -n domino-platform -o yaml mongodb-secondary > mongodb-secondary.yml
  2. Delete the PDBs:

    kubectl delete pdb -n domino-platform mongodb-arbiter
    kubectl delete pdb -n domino-platform mongodb-primary
    kubectl delete pdb -n domino-platform mongodb-secondary
Update the cluster
  1. Check for available AKS cluster upgrades:

    az aks get-upgrades --subscription $SUB_ID --resource-group $RG_NAME --name $CLUSTER_NAME --output table
  2. Run the cluster update:

    az aks upgrade --subscription $SUB_ID --resource-group $RG_NAME --name $CLUSTER_NAME --kubernetes-version <version-number>
    Note
Kubernetes v1.24
  1. If you upgrade to Kubernetes 1.24, add a path annotation for the nginx-ingress-controller probe:

    kubectl annotate svc -n domino-platform nginx-ingress-controller "service.beta.kubernetes.io/azure-load-balancer-health-probe-request-path=/healthz"
    kubectl annotate svc -n domino-platform nginx-ingress-controller "service.beta.kubernetes.io/azure-load-balancer-health-probe-request-path=/healthz"
    Important
    You must annotate this path to avoid 504 Gateway Time-out errors. This is because, in AKS version 1.24, the path no longer defaults to /healthz.
Restore the disruption budgets

Run these commands from the same folder as the backup:

kubectl apply -f mongodb-arbiter.yml
kubectl apply -f mongodb-primary.yml
kubectl apply -f mongodb-secondary.yml
Domino Data LabKnowledge BaseData Science BlogTraining
Copyright © 2022 Domino Data Lab. All rights reserved.