You can create new Domino environments and edit existing environments to meet your language and package needs. You can create or modify an environment when:
-
You must install a package for Python, R, Octave, or some other software dependency.
-
You want to cache a package or library so that it’s immediately available when you start a run.
-
You are managing an organization, and want to create a default environment across all projects.
A Domino environment is an abstraction on top of a Docker image. When Domino starts your run, it creates an isolated Docker container based on the environment associated with your project. This helps provide flexibility and versioning for your runs.
-
Go to the project for which you want to set a default environment. . In the navigation pane, click Settings.
-
From Compute environment, select the environment.
-
Click Manage Environment to open the environment’s Overview tab.
- Overview tab
The Overview tab lists all metadata about your environment. Click Edit Definition to make changes. After every save, your environment’s revision number increments by one, and your Domino deployment rebuilds the environment and pushes it to the local Docker registry.
- Revisions tab
Each time you edit an environment, you create a new revision. The Revisions tab lists all revisions of your compute environment, along with each revision’s build status, timestamp, and docker image URI. Click the gear icon to reveal additional options, including the ability to view build logs, cancel builds, or set a revision as Active.
- Projects and Models tabs
Projects and models are listed on their own tabs, after they have been assigned an environment. Use these tabs to see who to contact to update or archive an environment.
-
Click Duplicate Environment to clone an environment.
-
Click Edit Definition to edit an environment’s attributes.
-
Click Archive Environment to hide an environment. Projects already using this environment can continue to use it until a new project environment is selected.
In the navigation pane, click Environments to see the environments to which you have access, including: . Your deployment’s global environments. . Environments used by projects with which you are a collaborating. . Environments shared with organizations to which you belong.
Create an environment
-
Click Create Environment.
-
Name your new environment and define its visibility. Administrators can make the environment global, which makes it available to all users who have access to the deployment.
After creating your environment, the Revision tab opens.
-
Click Edit Definition to define the Dockerfile and supporting scripts and settings for the environment.
- Base Environment
Defines the FROM line in the Dockerfile that Domino constructs for you.
You can base your compute environment on your deployment default or on a custom Dockerfile URI (for example, registry.hub.docker.com/library/python:3.8-slim).
- Dockerfile Instructions
Enter your Dockerfile layers. See Docker’s official site. You can also view the Dockerfile best practices.
- Pluggable Notebooks / Workspace Sessions
Enter the interactive tools needed by any project using this environment. See Add Workspace IDEs.
- Scripts
-
Enter bash code that is executed at the specified step in your experiment’s lifecycle. Domino executes these commands at runtime and runs them as root.
- Pre-setup scripts
Run before the Python packages in your project’s requirements.txt are installed.
- Post-setup scripts
Run after the requirements.txt installation process.
- Pre Run scripts
Run after post-setup scripts.
- Post Run scripts
Run at the beginning of the Stopping run state. Because of the way that Domino shuts down runs and workspaces, these scripts are subject to a runtime limit. The default limit is one hour. If the script hasn’t completed by end of the runtime limit, it will time out and the process will terminate.
- Username
Specify a non-default username for the environment.
- Environment variables
Set environment variables at the environment level.
You might want to install packages directly to your environment. This is useful if your package installation takes a long time. Installations in a Domino environment are cached, so you won’t have to wait for the package to install every time. If you already have a Docker image you’d like to use, enter it in the preceding Base Environment field. If you don’t set this, the Domino default environment is used as your base image.
Consult the official Docker documentation to learn more about Dockerfiles:
Do not start your Dockerfile instructions in Domino with a FROM line. Domino includes the FROM line for you, pointing to the base image specified when setting up the environment.
The most common Dockerfile instructions you’ll use are RUN, ENV, and ARG:
RUN commands execute lines of bash. For example:
RUN wget http://d3kbcqa49mib13.cloudfront.net/spark-1.5.1-bin-hadoop2.6.tgz
RUN tar xvzf spark-1.5.1-bin-hadoop2.6.tgz
RUN mv spark-1.5.1-bin-hadoop2.6 /opt
RUN rm spark-1.5.1-bin-hadoop2.6.tgzARG commands set build-time variables, and
ENV commands set
container bash environment variables. They will be accessible from runs
that use this environment. For example:
ENV SPARK_HOME /opt/spark-1.5.1-bin-hadoop2.6If you set variables in the Environment variables section of your definition, you can use an ARG statement:
ARG SPARK_HOMEThis will be available for the build step. If you want the variable to be available in the final compute environment you also need to add an ENV statement referencing the argument name:
ENV SPARK_HOME=$SPARK_HOMEExamples: Package Installation
Click R Package or Python Package when editing your environment, to insert a line to install packages. Enter the names of the packages.
You can also add the commands, as in the following examples:
-
R Package Installation, example with the devtools package.
RUN R --no-save -e "install.packages('devtools')" -
Python Package Installation with Pip, for example with the… numpy package.
RUN pip install numpy
-
Docker optimizes its build process by keeping track of commands it has run and aggressively caching the results. This means that if it sees the same set of commands as in a previous build, it will assume that it can use the cached version. A single new command will invalidate the caching of all subsequent commands.
-
A Docker image can have up to 127 layers, or commands. To work around this limit, you can use
&&: to combine several commands into a single command.RUN \ wget http://d3kbcqa49mib13.cloudfront.net/spark-1.5.1-bin-hadoop2.6.tgz && \ tar xvzf spark-1.5.1-bin-hadoop2.6.tgz && \ mv spark-1.5.1-bin-hadoop2.6 /opt && \ rm spark-1.5.1-bin-hadoop2.6.tgz -
If you are installing multiple python packages via pip, it’s almost always best to use a single pip install command. This ensures that dependencies and package versions are properly resolved. If you install via separate commands, you may end up inadvertently overriding a package with the wrong version, due a dependency specified by a later installation. For example:
RUN pip install luigi nolearn lasagne