Configuration Reference¶
Key | Description | Required | Values |
---|---|---|---|
schema |
YAML schema version. | ✓ | 1.0 |
name |
Unique deployment name. This should contain the name of the deployment owner. | ✓ | [a-zA-Z0-9_-]+ |
version |
Domino version to install. | ✓ | Supported versions: 4.1.10 , 4.2.0 |
hostname |
Hostname Domino application will be accessed at. | ✓ | Valid FQDN |
pod_cidr |
If network policies are enabled, allow access from this CIDR. This range should cover addresses used by your cluster nodes and pods. | Valid CIDR range, e.g. 10.0.0.0/8 |
|
ssl_enabled |
Should Domino only be accessible using HTTPS. | ✓ | true , false |
ssl_redirect |
Should Domino only be accessible using HTTPS. | ✓ | true , false |
create_ingress_controller |
Create an NGINX ingress controller. | ✓ | true , false |
request_resources |
Create Kubernetes resource requests and limits for services. | ✓ | true , false |
enable_network_policies |
Use network policies for fine-grained service access. | ✓ | true , false , Note: requires a compatible CNI plugin e.g. Calico |
enable_pod_security_policies |
Enables pod security policies for locked down system capabilities. | ✓ | true , false |
create_restricted_pod_security_policy |
Creates pod security policies for locked down system capabilities. | ✓ | true , false |
kubernetes_distribution |
Determines resource compatibility with either OpenShift or CNCF Kubernetes | true |
cncf or openshift |
Istio¶
This section configures how and if an Istio service mesh is deployed by or integrated to Domino. A Domino-deployed Istio is for Domino use only. These configuration should only installed and/or enable if intra-cluster encryption in transit is required.
Key | Description | Required | Values |
---|---|---|---|
istio.enabled |
Enable Istio in deployment (i.e. sidecar injection) | ✓ | true , false |
istio.install |
Install Istio service with Domino | ✓ | true , false |
istio.cni |
Configures whether Istio installation is done with a CNI. If true , the installation is done with a CNI and requires fewer permissions; this is our preferred and recommended setting. If false , the installation will add required capabilities to every pod security policy: NET_ADMIN , NET_BIND_SERVICE , and NET_RAW . |
✓ | true , false |
istio.namespace |
Namespace of the Istio control plane. This field is not meant for a Domino-deployed Istio (i.e. istio.install=true ); it is available for integrating with an existing deployed Istio service within the cluster. |
✓ | true , false |
Ingress Controller¶
This section configures the NGINX ingress controller deployed by the
fleetcommand-agent
.
Key | Description | Required | Values |
---|---|---|---|
ingress_controller.create |
Whether to create the ingress controller. | ✓ | true , false |
ingress_controller.gke_cluster_uuid |
When running Domino on GKE you should supply the GKE cluster UUID here to configure GCP networking for ingress. | ✓ | Cluster UUID |
Namespaces¶
Namespaces are a way to virtually segment Kubernetes executions. Domino will create namespaces according to the specifications in this section, and the installer requires that these namespaces not already exist at installation time.
Key | Description | Required | Values |
---|---|---|---|
namespaces.platform.name | Namespace to place Domino services | ✓ | Kubernetes Name |
namespaces.compute.name | Namespace for user executions | ✓ | Kubernetes Name Note: may be the same as the platform namespace |
namespaces.system.name | Namespace for deployment metadata | ✓ | Kubernetes Name |
namespaces.*.annotations | Optional annotations to apply to each namespace | Kubernetes Annotation |
Storage Classes¶
Storage Classes are a way to abstract the dynamic provisioning of volumes in Kubernetes.
Domino requires two storage classes:
block
storage for Domino services and user executions that need fast I/Oshared
storage that can be shared between multiple executions
Domino supports pre-created storage classes, although the installer can create a shared
storage class backed by NFS
or a cloud NFS analog as long as the cluster can access the NFS system for read and write, and the installer can create
several types of block
storage classes backed cloud block storage systems like Amazon EBS.
Key | Description | Required | Values |
---|---|---|---|
storage_classes.block.create | Whether to create the block storage class | ✓ |
|
storage_classes.block.name | ✓ | Kubernetes Name Note: always required due to platform limitations, cannot be “” which indicates the default storage class | |
storage_classes.block.type | Name of the block storage class to utilize | ✓ |
|
storage_classes.block.base_path | Base path to
use on nodes as
a base when
using
hostpath
volumes |
||
storage_classes.block.default | Whether to set this storage class as the default | ✓ |
|
storage_classes.shared.create | Whether to create the shared storage class | ✓ |
|
storage_classes.shared.name | ✓ | Kubernetes Name | |
storage_classes.shared.type | Type of the shared storage class to utilize | ✓ |
|
storage_classes.shared.efs.region | EFS store AWS region | e.g. us-west-2 |
|
storage_classes.shared.efs.filesystem_id | EFS filesystem ID | e.g. fs-7a535bd1 |
|
storage_classes.shared.nfs.server | NFS server IP or hostname | ||
storage_classes.shared.nfs.mount_path | Base path to use on the server when creating shared storage volumes | ||
storage_classes.shared.nfs.mount_options | YAML List of additional NFS mount options | e.g. - mfsymlinks |
|
storage_classes.shared.azure_file.storage_account | Azure storage account to create filestores |
Blob Storage¶
Domino can store long-term, unstructed data in “blob storage”
buckets. Currently, only the shared
storage class described above (NFS) and S3 are supported.
To apply a default S3 bucket or shared storage type to all use-cases of
blob storage, it is only necessary to fill out the default
setting
and make sure enabled
is true. Otherwise, all other blob storage
uses (projects
, logs
, and backups
) should be filled out.
Key | Description | Required | Values |
---|---|---|---|
blob_storage.default.enabled | Whether the
default
configuration
should take
precedence
over individual
config keys |
✓ |
|
blob_storage.*.type | Which type of blob storage to use | ✓ |
|
blob_storage.*.s3.region | AWS region of the S3 bucket store | e.g. us-west-2 |
|
blob_storage.*.s3.bucket | S3 bucket name | e.g. domino-bucket-1 |
Autoscaler¶
For Kubernetes clusters without native cluster scaling in response to new user executions, Domino supports the use of the cluster autoscaler.
Key | Description | Required | Values |
---|---|---|---|
autoscaler.enabled | Enable cluster autoscaling | ✓ |
|
autoscaler.cloud_provider | Cloud provider Domino is deployed with |
|
|
autoscaler.aws.region | AWS region Domino is deployed into | e.g. us-west-2 |
|
autoscaler.azure.resource_group | Azure resource group Domino is deployed into | Azure resource group | |
autoscaler.azure.subscription_id | Azure subscription ID Domino is deployed with | Azure subscription ID |
AWS Auto-Discovery¶
The cluster autoscaler supports autodiscovery on AWS. Without any explicit configuration of specific autoscaling groups, it will detect all ASGs that have the appropriate tags and refresh them if their settings are updated directly. This means listing all ASGs with accurate min/max settings (or listing them at all) is not required as referenced below in the Groups section. ASG settings can be updated directly in AWS without having to update the cluster-autoscaler configuration or rerun the installer.
Key | Description | Required | Values |
---|---|---|---|
autoscaler.auto_discovery.cluster_name | K8s Cluster Name | exactly match the name in AWS | |
autoscaler.auto_discovery.tags | Optional. If filled in, cluster_name is ignored | e.g.
- my.tag
or
[] |
|
autoscaler.auto_discovery.groups | Must be set to
[] if using
auto_discovery |
By default, if no autoscaler.groups and autoscaler.auto_discovery.tags are specified, the cluster_name will be used to look for the following AWS tags:
k8s.io/cluster-autoscaler/enabled
k8s.io/cluster-autoscaler/{{ cluster_name }}
The tags
setting can be used to explicitly specify which resource tags the autoscaler service should look for.
If you would like to disable auto-discovery and continue using specific groups
, ensure that auto_discovery.cluster_name
is an empty value.
Groups¶
Autoscaling groups are not dynamically discovered. Each autoscaling group must be individually specified including the minimum and maximum scaling size.
Key | Description | Required | Values |
---|---|---|---|
autoscaler.groups.*.name | Autoscaling group name | Must exactly match the name in the cloud provider | |
autoscaler.groups.*.min_size | e.g. 0 |
||
autoscaler.groups.*.max_size | e.g. 10 |
External DNS¶
Domino can automatically configure your cloud DNS provider. More extensive documentation can be found on the external-dns homepage.
Key | Description | Required | Values |
---|---|---|---|
external_dns.enabled | Whether Domino should configure cloud DNS | ✓ |
|
external_dns.provider | Cloud DNS provider | e.g. aws |
|
external_dns.domain_filters | Only allow access to domains that match this filter | e.g. my-domain.example.com |
|
external_dns.zone_id_filters | Only allow updates to specific Route53 hosted zones |
Email Notifications¶
Domino supports SMTP for sending email notifications in response to user actions and run results.
Key | Description | Required | Values |
---|---|---|---|
email_notifications.enabled | Whether Domino should send email notifications | ✓ |
|
email_notifications.server | SMTP server hostname or IP | ||
email_notifications.port | SMTP server port | ||
email_notifications.encryption | Whether the SMTP server uses SSL encryption | ||
email_notifications.from_address | Email address to send emails from Domino with | e.g. domino
@example.com |
|
email_notifications.authentication.username | If using SMTP authentication, the username | ||
email_notifications.authentication.password | If using SMTP authentication, the password |
Monitoring¶
Domino supports in-cluster monitoring with Prometheus as well as more detailed, external monitoring through NewRelic APM and Infrastructure.
Key | Description | Required | Values |
---|---|---|---|
monitoring.prometheus_metrics | Install Prometheus monitoring | ✓ |
|
monitoring.newrelic.apm | Enable NewRelic APM | ✓ |
|
monitoring.newrelic.infrastructure | Enable NewRelic Infrastructure | ✓ |
|
monitoring.newrelic.license_key | NewRelic account license key |
Helm¶
Configuration for the Helm repository that stores Domino’s charts.
Key | Description | Required | Values |
---|---|---|---|
helm.version |
Which version of Helm to use. | ✓ | 2 or 3 |
helm.host |
Hostname of the chart repository | ✓ | For Helm 2 this should be quay.io or the address of your private appr server. For Helm 3 it should be gcr.io . |
helm.namespace |
Namespace to find charts in the repository. | Helm repo namespace. When using official Domino repositories this should be domino . For Helm 3 with gcr.io or mirrors.domino.tech , use domino-eng-service-artifacts . |
|
helm.prefix |
Prefix for the chart repository. | Application registry prefix. When using official Domino repositories this should be helm- . For Helm 3 with gcr.io or mirrors.domino.tech , this should be an empty string. |
|
helm.username |
Username for chart repository if authentication is required. When using Helm 3 with charts hosted in GCR this must be _json_key . |
Username | |
helm.password |
Password for chart repository if authentication is required. | For Helm 3 this is the base64 encoded JSON key that was provided by Domino. | |
helm.tiller_image |
URI of the Docker image for the Tiller service to use when running Helm 2. | ✓ | This must point to a version 2.16.1 Tiller image at gcr.io/kubernetes-helm/tiller:v2.16.1 or in your private registry. |
helm.cache_path |
Path to cached Helm 3 chart files. | Set to empty string ('' ) to use online chart data. |
Private Docker Registry¶
Configuration for the Docker repository that stores Domino’s images.
Key | Description | Required | Values |
---|---|---|---|
private_docker_registry.server | Docker registry host | ✓ |
|
private_docker_registry.username | Docker registry username | ✓ | |
private_docker_registry.password | Docker registry password | ✓ |
Internal Docker Registry¶
Configuration for the internal Docker registry deployed with Domino.
Override values are to allow the registry to use S3, GCS, or Azure blob store as a backend
store. GCS requires a service account already be bound into the
Kubernetes cluster with configuration to ensure the docker-registry
service account is properly mapped.
Key | Description | Required | Values |
---|---|---|---|
internal_docker_registry.s3_override.region | AWS region of the S3 bucket store | e.g. us-west-2 |
|
internal_docker_registry.s3_override.bucket | S3 bucket name | e.g. domino-bucket-1 |
|
internal_docker_registry.gcs_override.bucket | GCS bucket name | e.g. domino-bucket-1 |
|
internal_docker_registry.gcs_override.service_account_name | GCS service account with access to the bucket | ||
internal_docker_registry.gcs_override.project_name | GCP project name that Domino is deployed into | ||
internal_docker_registry.azure_blobs_override.account_name | Azure blobstore account name | ||
internal_docker_registry.azure_blobs_override.account_key | Azure blobstore account key | ||
internal_docker_registry.azure_blobs_override.container | Azure blobstore container name |
Telemetry¶
Domino supports user telemetry data to help improve the product.
Key | Description | Required | Values |
---|---|---|---|
intercom.enabled | Enable Intercom onboarding | ✓ | true|false |
mixpanel.enabled | Enable MixPanel | ✓ | true|false |
mixpanel.token | MixPanel API token |
GPU¶
If using GPU compute nodes, enable the following configuration setting to install the required components.
Key | Description | Required | Values |
---|---|---|---|
gpu.enabled | Enable GPU support | ✓ | true|false |
Fleetcommand¶
Domino supports upgrading minor patches through an internal tool named Fleetcommand.
Key | Description | Required | Values |
---|---|---|---|
fleetcommand.enabled | Enable ability for Domino staff to apply minor patches | ✓ | true|false |
fleetcommand.api_token | Deployment-specific API token (Domino staff will provide this) |
Node selectors¶
Domino will by default deploy some DaemonSets on all available nodes in the hosting cluster. When running in a multi-tenant Kubernetes cluster, where some nodes are available that should not be used by Domino, you can label nodes for Domino with a single, consistent label, then provide that label to the fleetcommand-agent with the below configuration to apply a selector to all Domino resources for that label.
Key | Description | Required | Values |
---|---|---|---|
global_node_selectors |
List of key/value pairs to use as the label for the selector. | Optional | See below example |
Example
global_node_selectors:
domino-owned: "true"
This example would apply a selector for domino-owned=true
to all Domino deployment resources.
Ingress controller class¶
The name of the Domino Ingress class can be changed with this setting. This should generally not need to change.
Key | Description | Required | Values |
---|---|---|---|
ingress_controller.class_name |
Name for the Domino Ingress class | ✓ | nginx |
Image caching¶
These settings control the Domino image caching service, which runs as a privileged pod and uses the host Docker socket to pre-pull popular Domino environment images onto compute workers. It can be disabled if desired.
Key | Description | Required | Values |
---|---|---|---|
image_caching.enabled |
Whether or not to deploy the image caching service | ✓ | Boolean |