Scaling AI at the edge is a vital way to productionize data science work. This scaling is critical for varied use cases, such as anomaly detection at cellphone towers and quality control in manufacturing. Domino’s integration with NVIDIA Fleet Command simplifies model deployment to the edge.
-
Initiate a Model API build from any project, and Domino packages the necessary model files, inference code, dependent images, environments, and packages.
-
After the status is Ready to Run, invoke a Domino API to register the model image with NVIDIA.
-
After the API registers the packaged container image with NVIDIA’s Fleet Command registry, configure the edge API with the Fleet Command console to invoke predictions with this containerized model.
With Domino’s model export catalog, you can see a unified view of all exported models and their performance. With this view, you can track and manage all production assets from a single pane.
For data scientists and their IT counterparts, Domino simplifies the process of operationalizing a model at the edge to a few steps.
Use the /:modelId/:modelVersionId/exportImageForNvidia API endpoint to export a Domino Model in a format NVIDIA Fleet Command can deploy. Domino ensures the Model API container image and an associated Helm chart are registered with the Fleet Command repository.
-
Deploy a Model API on Domino.
This deployment ensures that the container image is built and ready for deployment externally. Use the Domino-hosted Model API to run tests before you deploy the container to Fleet Command. See Create a Model API for more information.
-
That’s it!
After the Model API starts to run, its image is ready for export. See the Model API details to learn how to invoke the export API to push the image to Fleet Command and check the status endpoint to track the progress of the export.
Invoke the export API
curl =
--location --request POST 'https://<deployment>/v4/models/<model-id>/<model-version-id>/exportImageForNvidia' \
--header 'Content-Type: application/json' \
--header 'X-Domino-Api-Key: <domino-api-key>' \
--data-raw '{
"containerRepository": "<path-to-container-repo>",
"tag": "<image-tag>",
"helmRepository": "<path-to-helm-repo>",
"helmVersion": "<version>",
"ngcApiKey": "<ngc-api-key>"
}'Here is a description of the parameters:
-
model-id: unique identifier for the Model API -
Model-version-id: version ID for the Model API -
containerRepository: Repository name for a private container repository in NVIDIA NGC. For example, <organization-name>/<team-name>/<container-repository-name>. The user can provide a container repository that already exists in NVIDIA NGC. Here are the acceptable elements for a Container repository name:-
Alphanumeric characters
-
Lowercase characters
-
Dashes and slashes (for organization and team names)
-
No spaces
-
-
tag: Tag for the container. The tag must be alphanumeric. It can also contain underscores, periods, and dashes. -
helmRepository: Repository name for a private helm repository in NVIDIA NGC. For example, <organization-name>/<team-name>/<container-repository-name>. The user can provide a helm repository that already exists in NVIDIA NGC. Here are the acceptable elements in a Helm repository name:-
Alphanumeric characters
-
Lowercase characters
-
Dashes and slashes (for organization and team names)
-
No spaces
-
-
helmVersion: Version of the Helm chart. Helm versions must be in Semver format. -
ngcApiKey: NVIDIA NGC API key. See generate an NVIDIA NGC API Key.Note
Track the status of the export
Use the export-id you get as the return value from the export endpoint to monitor the export status.
Here is a sample command:
curl --include \
--header 'Content-Type: application/json' \
--header 'Accept: application/json' \
--header 'X-Domino-Api-Key: <domino-api-key>' \
'https://<deployment>/v4/models/<export-id>/getExportImageStatus'