• Enable MLflow client integration
• Associating a candidate to a CI/CD job
• Supported MLflow client methods and caveats
• Limitations

MLflow client compatibility

MLflow is a popular open source tool for Machine Learning Experiment Tracking. GitLab Model experiment tracking is compatible with MLflow Client, logging experiments. The setup requires minimal changes to existing code.

GitLab plays the role of a MLflow server. Running mlflow server is not necessary.

Enable MLflow client integration

Prerequisites:

• A personal, project, or group access token with at least the Developer role and the api permission.
• The project ID. To find the project ID:
  1. On the left sidebar, at the top, select Search GitLab () to find your project.
  2. Select Settings > General.

To use MLflow client compatibility from a local environment:

1. Set the tracking URI and token environment variables on the host that runs the code. This can be your local environment, CI pipeline, or remote host. For example:

   export MLFLOW_TRACKING_URI="<your gitlab endpoint>/api/v4/projects/<your project id>/ml/mlflow"
   export MLFLOW_TRACKING_TOKEN="<your_access_token>"
   

2. If your training code contains the call to mlflow.set_tracking_uri(), remove it.

When running the training code, MLflow creates experiments, runs, log parameters, metrics, metadata and artifacts on GitLab.

After experiments are logged, they are listed under /<your project>/-/ml/experiments. Runs are registered as:

• Model Candidates, which can be explored by selecting an experiment.
• Tags, which are registered as metadata.

Associating a candidate to a CI/CD job

If your training code is being run from a CI/CD job, GitLab can use that information to enhance candidate metadata. To associate a candidate to a CI/CD job:

1. In the Project CI variables, include the following variables:
   • MLFLOW_TRACKING_URI: "<your gitlab endpoint>/api/v4/projects/<your project id>/ml/mlflow"
   • MLFLOW_TRACKING_TOKEN: <your_access_token>

2. In your training code within the run execution context, add the following code snippet:

    with mlflow.start_run(run_name=f"Candidate {index}"):
      # Your training code
   
      # Start of snippet to be included
      if os.getenv('GITLAB_CI'):
        mlflow.set_tag('gitlab.CI_JOB_ID', os.getenv('CI_JOB_ID'))
      # End of snippet to be included
   

Supported MLflow client methods and caveats

GitLab supports these methods from the MLflow client. Other methods might be supported but were not tested. More information can be found in the MLflow Documentation.

Limitations

• The API GitLab supports is the one defined at MLflow version 1.28.0.
• API endpoints not listed above are not supported.
• During creation of experiments and runs, ExperimentTags are stored, even though they are not displayed.
• MLflow Model Registry is not supported.