About Domino
Domino Data LabKnowledge BaseData Science BlogTraining

Remediate a Model API

After reviewing the data drift and model quality monitor, use the monitoring results and Cohort Analysis to determine whether there are concerns with your model. If you do have concerns, review your data.

If you want to review the model and its associated code to investigate further, you can reproduce the original code commits and artifacts in the environment in which you deployed the model. If necessary, you can update the model code or retrain it with the latest production data. Then you can deploy a new, improved version of the Model API.

Review the Model API predictions

Domino automatically creates a prediction dataset named prediction_data for every project that can be accessed from any workspace. The predictions are in Parquet format and are updated hourly as the Model API processes inputs. If there is no data in an hour, no file is created. If you configured data drift monitoring or Set up Model Quality Monitoring then the stored prediction data is automatically consumed by the Model Monitor.

By default, a daily job deletes data older than 30 days. Your administrator defines the retention policy for predictions. See the model monitoring configuration options.

Review your data:

  1. In your workspace, open the IDE.

  2. Use the following paths to read the data:

    • To load individual Parquet files:

/domino/datasets/local/prediction_data/<model_version_id>/$$date$$=<date_in_utc>/$$hour$$=<hour_in_utc>/predictions_<uuid>.parquet
/domino/datasets/local/prediction_data/<model_version_id>/$$date$$=<date_in_utc>/$$hour$$=<hour_in_utc>/predictions_<uuid>.parquet
Caution

Reproduce the environment

Prerequisites

For Git-based projects

  • This feature is only enabled for models published in Domino 5.0 and higher.

For Domino File System-based projects

  • This feature is available for pre-5.0 published models as well as newly-published models, as long as no additional Git repositories are involved.

Remediate the model

  1. From the navigation pane, click Model APIs.

  2. Click the model that you want to remediate and then click Open in Workspace.

  3. From the Open in New Workspace and Branch window, type a name for the workspace.

  4. Select a Hardware Tier.

  5. In New Branch Name, type a name for the code branch.

  6. Click Open. A Domino workspace opens and is ready for you to take remedial action.

    Note

Publish a new Model API

When you reproduce a workspace, as you did in the Reproduce the environment topic, Domino creates a branch in every repository involved in the project.

To publish a new model based on this reproduced branch, you must apply the commit to the master branch because Domino supports Git-based projects. The way that you do this depends on whether you are working with a Domino File System-based project or a Git-based project.

Commit projects based on the Domino File System (DFS):

  1. Go to the Project.

  2. Click Code. in the navigation bar.

  3. From the Branch list, select the reproduced branch.

  4. Click Revert Project to ensure that the commits made in this branch are added in the Master branch.

    Note

Commit Git-based projects:

  • In a Domino workspace or the Git tool of your choice, merge your latest code update into the master branch.

    Note

Go to the Model API section of your project to publish a new Model API or a new version of an existing Model API. See Publish the model API.