domino logo
5.0
  • About Domino
  • Architecture
  • Security and Compliance
  • Domino Infrastructure
  • Kubernetes Infrastructure
  • Installation
  • Execution Resources
  • Users
  • Keycloak Authentication Service
  • Data
  • Projects
  • Environments
  • Model Monitoring Notifications
  • Control Center
  • Logs and Monitoring
  • Backup and Restore
  • Disaster Recovery
  • Configuration Reference
  • Run a System Scan
  • MongoDB Console
  • Get Help
domino logo
About Domino
Domino Data LabKnowledge BaseData Science BlogTraining
Admin Guide
>
Installation
>
Google Cloud Platform (GCP) Deployments
>
Kubernetes Upgrade Guide

Kubernetes Upgrade Guide

This topic describes how to upgrade Kubernetes in your Google Kubernetes Engine (GKE) Domino deployment. GKE is hosted on Google Cloud Platform (GCP).

Important
Prerequisites

To upgrade Kubernetes in GKE, you must have a Unix or Linux terminal with the following:

  • gcloud CLI installed.

  • Kubectl installed.

Use environment variables to set the values of IDs, names, and labels. This simplifies the commands you’ll run while upgrading:

export CLUSTER_PROJECT=<The project name of the cluster>
export CLUSTER_NAME=<The name of the cluster>
export CLUSTER_REGION=<The region where the cluster is located>
Get cluster credentials
  1. If you aren’t already signed in with gcloud, run, gcloud init.

  2. To protect your current config file from being overwritten, run export KUBECONFIG=$(pwd)/kubeconfig.

  3. Get the remote kubeconfig:

    gcloud container clusters get-credentials $CLUSTER_NAME --region $CLUSTER_REGION --project $CLUSTER_PROJECT
Disable the disruption budgets

You must temporarily disable the PodDisruptionBudget (PDB) for certain services. To do this, back up the PDBs, delete the original PDBs, then update Kubernetes and restore the PDBs from the backup after the update is complete.

  1. Backup the PDBs:

    kubectl get pdb -n domino-platform -o yaml mongodb-arbiter > mongodb-arbiter.yml
    kubectl get pdb -n domino-platform -o yaml mongodb-primary > mongodb-primary.yml
    kubectl get pdb -n domino-platform -o yaml mongodb-secondary > mongodb-secondary.yml
  2. Delete the PDBs:

    kubectl delete pdb -n domino-platform mongodb-arbiter
    kubectl delete pdb -n domino-platform mongodb-primary
    kubectl delete pdb -n domino-platform mongodb-secondary
Determine upgrade version
  1. Get available GKE versions

    gcloud container get-server-config \
      --region=${CLUSTER_REGION} \
      --flatten="channels" \
      --format="yaml(channels.channel,channels.validVersions)"
  2. Get current master version

gcloud container clusters describe ${CLUSTER_NAME} --region=${CLUSTER_REGION} --format="value(currentMasterVersion)"
Note
Update the cluster
  1. Upgrade the control plane:

    gcloud container clusters upgrade $CLUSTER_NAME --project $CLUSTER_PROJECT --master --region $CLUSTER_REGION --cluster-version <version-number>
    Note
  2. Upgrade the node pools:

    gcloud container clusters upgrade $CLUSTER_NAME --project $CLUSTER_PROJECT --region $CLUSTER_REGION --node-pool=platform
    gcloud container clusters upgrade $CLUSTER_NAME --project $CLUSTER_PROJECT --region $CLUSTER_REGION --node-pool=compute
    gcloud container clusters upgrade $CLUSTER_NAME --project $CLUSTER_PROJECT --region $CLUSTER_REGION --node-pool=gpu
Restore the Disruption Budgets

Run these commands from the same folder as the backup:

kubectl apply -f mongodb-arbiter.yml
kubectl apply -f mongodb-primary.yml
kubectl apply -f mongodb-secondary.yml
Domino Data LabKnowledge BaseData Science BlogTraining
Copyright © 2022 Domino Data Lab. All rights reserved.