You might want to build models in Domino but host them in an environment outside Domino. For example, you might already have invested in an external production environment that supports high scale or low latency, your production data cannot be exported outside an environment for legal or compliance reasons, or you want to control deployments using a custom CI/CD pipeline.
Use Domino to export model images built in Domino to AWS Sagemaker as well as other external container registries.
These images include all the information needed to run the model such as the model code, artifacts, environment, and project files.
Your CI/CD pipeline or workflow can call Domino’s REST APIs to programmatically build and export the model image. By default, the images are built in Model API format. These images can be deployed in an environment that can run docker containers. However, you can also export images that are in AWS Sagemaker-compatible format so that they can be directly deployed in AWS Sagemaker.
The following APIs are now available:
Build Model Image API
This API builds a docker image for a model and stores it in Domino’s internal Registry. This can be fetched from the registry later (using other export APIs) by your CI/CD pipeline. Your runtime environment (outside Domino) can deploy the exported image. Your CI/CD pipeline can add layers to this image to do further customizations (such as adding Auth).
Export Model Image API
This API can push a model image to a third-party container registry outside Domino. It assumes the image was already built and available within Domino. As part of the API request, users must provide credentials for the registry so the image can be pushed to it. These credentials are not saved inside Domino and can have a time-to-live (TTL) attached.
The exported model runs on the port/path :8888/model.
Export Model Image For Sagemaker API
This API builds a docker image for a version of an Model API in an AWS Sagemaker-compliant format and then exports it to AWS ECR or any third-party container registry outside Domino. As part of the API request, users must provide credentials for their registry to push the image to it. These credentials are not saved inside Domino and can have a time-to-live (TTL) attached.
Sagemaker can train and then deploy a model to serve requests. Model export functionality in Domino only supports the serve use case because the train operation would have already happened in Domino. The image is ready to be deployed in the Sagemaker environment. All the files required to make predictions are packaged inside the container.
When you export a Domino model to Amazon Sagemaker and create an endpoint from the exported model, the Sagemaker endpoint might fail with the error The primary container for production variant variant-name-1 did not pass the ping health check.
Follow these steps to work around the issue:
-
Add
USER rootin the environmentDockerfileinstructions. -
Publish the model API from the new instructions.
-
Export the same image in Sagemaker and create an endpoint.