domino logo
About DominoArchitecture
Kubernetes
Cluster RequirementsDomino on EKSDomino Kubernetes Version CompatibilityDomino on GKEDomino on AKSDomino on OpenShiftNVIDIA DGX in DominoDomino in Multi-Tenant Kubernetes ClusterEncryption in Transit
Installation
Installation ProcessConfiguration ReferenceInstaller Configuration ExamplesPrivate or Offline InstallationCustom Certificatesfleetcommand-agent release notes
Azure Deployments
Prepare for InstallationProvision Infrastructure and Runtime EnvironmentDeploy Domino
Google Cloud Deployments
Prepare for InstallationProvision Infrastructure and Runtime EnvironmentDeploy Domino
Amazon Web Services Deployments
Prepare for InstallationProvision Infrastructure and Runtime EnvironmentDeploy Domino
Configuration
Central ConfigurationNotificationsFeature FlagsChange The Default Project For New UsersProject Stage ConfigurationDomino Integration With Atlassian Jira
Compute
Manage Domino Compute ResourcesHardware Tier Best PracticesModel Resource QuotasPersistent Volume ManagementAdding a Node Pool to your Domino ClusterRemove a Node from Service
Keycloak Authentication Service
Operations
Domino Application LoggingDomino MonitoringSizing Infrastructure for Domino
Data Management
Data in DominoData Flow In DominoExternal Data VolumesConfigure Data Source AuthenticationDatasets AdministrationSubmit GDPR Requests
User Management
RolesManage UsersView User InformationRun a User Activity ReportSchedule a User Activity Report
Environments
Environment Management Best PracticesCache Environment Images in EKSImages From Authenticated External Registries
Backup and Restore
Backup StructureBackup LocationCustomize BackupsRun a Manual, On-Demand BackupRestore backups
Control Center
Control Center OverviewExport Control Center Data with The API
Troubleshooting
domino logo
About Domino
Domino Data LabKnowledge BaseData Science BlogTraining
Admin Guide
>
Data Management
>
Data Flow In Domino

Data Flow In Domino

The following are the ways for data to flow in and out of a Domino Run.

1) Domino File Store

Each Domino Run takes place in a project, and the files for the active revision of the project are automatically loaded into the local execution volume for a Job or Workspace according to the specifications of the Domino Service Filesystem. These files are retrieved from the Domino File Store, and any changes to these files are written back to the Domino File Store as a new revision of the project’s files.

2) Domino Datasets

Domino Runs might optionally be configured to mount Domino datasets for input or output. Datasets are network volumes mounted in the execution environment. Mounting an input Dataset allows for a Job or Workspace to both start quickly and have access to large quantities of data, since the data is not transferred to the local execution volume until user code performs read operations from the mounted volume. Any data written to an output Dataset is saved by Domino as a new snapshot.

3) External data systems

User code running in Domino can use Domino data sources or third-party drivers to interact with any external databases, APIs, and file systems to which the Domino-hosting cluster can connect. Users can read and write from these external systems, and they can import data into Domino from such systems by saving files to their project or writing files to an output Dataset.

The following diagram shows the series of operations that happens when a user starts a Job or Workspace in Domino, and illustrates when and how various data systems can be used.

data flow

Domino Data LabKnowledge BaseData Science BlogTraining
Copyright © 2022 Domino Data Lab. All rights reserved.