Domino Cluster Requirements Checker

The Domino Cluster Requirements Checker is a command-line utility that checks if a Kubernetes cluster conforms to Domino requirements. The Cluster Requirements Checker is a plugin for Sonobuoy, a Kubernetes diagnostic tool. The instructions on this page are used to run only the Domino plugin, and not the full Kubernetes conformance suite.

The Cloud Native Compute Foundation has certified many Kubernetes offerings. Kubernetes certification steps include conformance tests run by Sonobuoy. Domino uses the Sonobuoy Plugin Framework to perform customized Domino conformance checks on a cluster prior to installing Domino.




Instructions

You should perform the following steps from a workstation with kubectl admin access to the target cluster.

  1. Install Sonobuoy binaries

    If the cluster is running Kubernetes 1.13, install sonobuoy v0.15.4.
    If the cluster is running Kubernetes 1.14 or above, install the latest sonobuoy release (v0.16.2)

    Run the following command to determine the Kubernetes version for your cluster:

    kubectl version
  2. Set a KUBECONFIG environment variable to a path to a kubeconfig file with admin access to the target cluster.

    export KUBECONFIG=~/.kube/config
  3. Create a domino-checker.yaml configuration file with the following contents. You can download this file from GitHub here

    sonobuoy-config:
     driver: DaemonSet
     plugin-name: domino
     result-format: junit
     skip-cleanup: true
    
    spec:
     env:
     - name: DOCKER_API_VERSION
       value: '1.38'
     - name: NODE_NAME
       valueFrom:
         fieldRef:
           fieldPath: spec.nodeName
     - name: POD_NAME
       valueFrom:
         fieldRef:
           fieldPath: metadata.name
     - name: POD_NAMESPACE
       valueFrom:
         fieldRef:
           fieldPath: metadata.namespace
     - name: RESULTS_DIR
       value: /tmp/results
     image: quay.io/domino/k8s-validator:latest
     imagePullPolicy: Always
     name: domino
     securityContext:
       privileged: false
     volumeMounts:
     - mountPath: /tmp/results
       name: results
       readOnly: false
     - mountPath: /var/run/docker.sock
       name: docker-mnt
       readOnly: false
    
    extra-volumes:
    - name: docker-mnt
      hostPath:
       path: /var/run/docker.sock
    
  4. Run the plugin.

    1. sonobuoy run -p domino-checker.yaml --wait
    2. resultsfile=$(sonobuoy retrieve)
    3. sonobuoy results $resultsfile --plugin domino
    4. sonobuoy delete --wait

    The last instruction is necessary to remove sonobuoy.
    You must do this step if you want to run Sonobuoy on the cluster again.



Output example

validator> sonobuoy run -p domino-checker.yaml --wait
WARN[0001] Version v1.14.7-gke.14 is not a stable version, conformance image may not exist upstream
INFO[0002] created object                                name=sonobuoy namespace= resource=namespaces
INFO[0002] created object                                name=sonobuoy-serviceaccount namespace=sonobuoy resource=serviceaccounts
INFO[0002] created object                                name=sonobuoy-serviceaccount-sonobuoy namespace= resource=clusterrolebindings
INFO[0002] created object                                name=sonobuoy-serviceaccount namespace= resource=clusterroles
INFO[0002] created object                                name=sonobuoy-config-cm namespace=sonobuoy resource=configmaps
INFO[0002] created object                                name=sonobuoy-plugins-cm namespace=sonobuoy resource=configmaps
INFO[0002] created object                                name=sonobuoy namespace=sonobuoy resource=pods
INFO[0002] created object                                name=sonobuoy-master namespace=sonobuoy resource=services
validator> theFile=$(~/bin/sonobuoy retrieve)
validator> sonobuoy results $theFile --plugin domino
Plugin: domino
Status: failed
Total: 8
Passed: 6
Failed: 2
Skipped: 0

Failed tests:
Node CPU
Node Memory
validator> sonobuoy delete --wait
INFO[0000] deleted                                       kind=namespace namespace=sonobuoy
INFO[0000] deleted                                       kind=clusterrolebindings
INFO[0000] deleted                                       kind=clusterroles



Getting more details on failures

Run the following command to get more information about failed checks.

sonobuoy results $resultsfile --plugin domino --mode=dump

The output will look like this.

name: domino
status: failed
Items:
- name: gke-etienne-gke-1-build-13b06f55-8f2l
  status: failed
  Items:
  - name: domino-junit.xml
    status: failed
    Meta:
      file: results/gke-etienne-gke-1-build-13b06f55-8f2l/domino-junit.xml
    Items:
    - name: Domino Sonobuoy K8s Conformance Plugin
      status: failed
      Items:
      - name: RWX Storage Class Available
        status: passed
      - name: Default Storage Class Set
        status: passed
      - name: Helm (Tiller) Service does not exist
        status: passed
      - name: Node Labels
        status: passed
      - name: Node CPU
        status: failed
        Details:
          failure: Insufficient 24 required but only 8 of 24 available for Domino
      - name: Node Memory
        status: failed
        Details:
          failure: Insufficient 96Gi required but only 30880736Ki of 92642208Ki available
            for Domino
      - name: 'Docker Daemon Available: 4.14.145+'
        status: passed
- name: gke-etienne-gke-1-compute-a5dfc474-g5s4
  status: passed
  Items:
  - name: domino-junit.xml
    status: passed
    Meta:
      file: results/gke-etienne-gke-1-compute-a5dfc474-g5s4/domino-junit.xml
    Items:
    - name: Domino Sonobuoy K8s Conformance Plugin
      status: passed
- name: gke-etienne-gke-1-platform-a70f6fe2-fcss
  status: passed
  Items:
  - name: domino-junit.xml
    status: passed
    Meta:
      file: results/gke-etienne-gke-1-platform-a70f6fe2-fcss/domino-junit.xml
    Items:
    - name: Domino Sonobuoy K8s Conformance Plugin
      status: passed