Git-based projects provide a full CodeSync experience for your code by using Git and a Git service provider of your choice. Integrated CodeSync technology ensures that all the common Git workflows, like committing, pushing changes, and more, are available to you natively within workspaces launched in Git-based projects. This makes it easy for you to engage in version controlled, code-based collaboration with fellow project team members, all from within Domino. Git-based projects also organize your projects' assets as either Code, Data, or Artifacts, an organizational structure intended to support common data science workflows.
If you want to use a private Git repository to store your code, then you must add the corresponding Git credentials in your Domino account settings prior to creating your project. After adding credentials, you’ll be able to easily create a Git-based project in Domino, thereby enabling CodeSync experience.
If you want to use a public Git repository to store your code, then you won’t need to add any Git credentials.
-
Click Projects in the navigation pane and then click New Project.
-
In the Create New Project window, enter a name for your project.
-
Set your project’s visibility.
-
Click Next to go to the wizard’s next page.
-
Under Hosted By, click Git Service Provider. On selection, more fields will be shown beneath the Hosted By field.
-
Under Git Service Provider, select the provider currently hosting the repository you wish to import (the "target repository").
-
Under Git Credentials, select credentials authorized to access the target repository.
-
Under Repository, you can either choose a repository from a list or enter a Git URL. If you are using a PAT credential with Github or GitLab, you can also create your own repository.
-
Click Create
ImportantIf the repository you’re using to store your code contains one or more files exceeding 2 GB in size, Domino will create your Git-based project, but you might fail at workspace setup. Consider using Domino File System in that case.
You can use the following tool to check the total size of a Git repository, as well as the size of individual files within the repository: git sizer.
-
During the project creation process, you can create a completely new repository for Github and GitLab. (These are the only Git providers currently supported by Domino.)
-
Select the Create new repository option under Repository.
-
Select the Owner/Organization associated with the repository, its visibility, and specify the name for the new repository.
In Domino, the Domino File System (DFS) is the traditional way of storing a project’s assets. DFS-based projects organize all of your project’s assets as either Data or Files. Git-based projects, however, organize your project’s assets as either Code, Data, or Artifacts, and apply CodeSync experience to Code assets.
Code – This section of your Git-based project organizes and lists all of the Git-based repositories used to store your project’s code, as well as any additional imported repositories. For more information, see Git repositories in Domino. Files within any of these repositories can be accessed from within a Domino workspace via CodeSync technology.
The common Git workflows, like committing, pushing, pulling, and more, are available to you when interacting with your code from within a Domino workspace. For more information, see Using Git in your workspace.
You can select from any branch and the latest 10 commits using drop-down lists, then browse the directories of linked Git repositories natively from the Code section on project page.
- Data
-
Similar to DFS projects, this section of your Git-based project organizes and lists all data sources used in your project, including Domino datasets, external data volumes, and dataset scratch spaces. For more information about how to use data with your project, see the Domino datasets documentation.
- Artifacts
-
Git-based projects introduce “Artifacts”. Artifacts are typically results or products from your research and analysis, like plots, charts, serialized models, and more. You can organize these outputs in this section, as well as import artifacts from other projects.
A Domino workspace is an interactive session where you can conduct research, analyze data, train models, and more. Workspaces let you work in a development environment of your choice, like Jupyter notebooks, RStudio, VS Code, and many other customizable environments.
Directory structure
Git-based projects with CodeSync use a different directory structure in workspaces than DFS-based projects. The directory structure is shown below.
The default working directory for your code is /mnt/code.
/mnt
│
├── /code # Git repository and default working directory
│
├── /data
│ │ # Project Datasets
│ ├── /{dataset-name} # latest version of dataset
│
│ # Project Artifacts
├── /artifacts
│
│ # External mounted volumes
├── /{external-volume-name}
│
└── /imported
│ # Imported Git Repos
├── /code
│ └── /{imported-repo-name}
│
├── /data
│ │ # Mounted Shared Datasets
│ └── /{shared-dataset-name} # contains contents of latest snapshot unless otherwise specified by yaml
│
│ # Imported Project Artifacts
└── /artifacts
└── /{imported-project-name}Switch branches in your workspace
Branches allow you to develop features, fix bugs, or safely experiment with new ideas in a contained area of your repository. You can switch branches easily inside your workspace for both the main code repository and any additional imported repositories. Maximum of 10 branches will be listed in the drop-down, in alphabetical order for local branches followed by remote branches. If your repositories have more than 10 branches, you can type to search for additional branches.
Resolve merge conflicts
Merge conflicts occur when competing changes are made to the same line of a file, or when one person edits a file and another person deletes the same file. You can resolve merge conflicts inside your workspace through guided UI for both the main code repository and any additional imported repositories.
Sync changes
When syncing changes in your workspace to remote Git repositories, we will first fetch the latest content from the remote branch (git fetch), then commit local changes on top of the updated branch (git rebase), and finally try to push the commit/s to the remote. When files are in conflict, you can choose to either resolve manually or force my changes. Force my changes will overwrite remote files with changes in your workspace. This means that the commit history on the remote will match the commit history in your workspace.
Pull changes
When pulling latest changes from remote to your workspace, we will first fetch the latest content from the remote branch (git fetch), then apply your changes on top of the updated branch (git rebase). When files are in conflict, you can choose to either resolve conflicts manually or use remote changes. Use Remote Changes will discard changes in your workspace and overwrite files in your workspace with remote changes.
Resolve manually
When Resolve Manually is selected, you can resolve conflicts by the filename. For each file in conflict, you can choose to Mark as resolved, Use my changes or Use origin repo changes:
-
Mark as resolved assumes that you’ve edited the files to resolve conflict markers. The latest change of the file will appear under Uncommitted changes and be pushed to remote when you continue sync.
-
Use my changes will overwrite remote files with changes in your workspace. The latest change of the file will appear under Uncommitted changes and be pushed to remote when you continue sync.
-
Use origin repo changes will discard changes in your workspace and overwrite the file with remote changes. The file won’t appear under Uncommitted changes because there is no change to commit. However, you still need to click Continue sync to complete the conflict resolution.
Work with artifacts in your workspace
|
Important
| All files in Artifacts are saved exclusively to the Domino File System (DFS). If you do not want to save a particular asset to the Domino File System, we recommend that you do not save it as an artifact. To learn more, see Syncing your work to Domino. |
Artifacts are results from your research, like plots, charts, serialized models, and more. In Domino, you can save these results in the Artifacts section of your project.
Saving artifacts and pushing changes
-
Click File Changes in the navigation pane of your workspace.
-
Under Artifacts, view changes by expanding File Changes.
-
Enter a commit message.
-
Click Sync to Domino. Domino will save your artifacts to the Domino File System (DFS).
Pull changes
-
Click the File Changes option in the sidebar menu of your workspace.
-
Under the “Artifacts” section, click Pull. Domino will pull the latest changes into your workspace.
|
Warning
| If you run a job in a Git-based project, CodeSync ensures that only artifacts will be automatically synced and saved to the Domino File System (DFS). Code, on the other hand, will not be automatically synced/pushed to the Git repository being used for the Git-based project. This is intentional and intended to support the "Code", "Data", and "Artifacts" workflow. To learn more, see running jobs. |