Dependency proxy for packages

Version history
On self-managed GitLab, by default this feature is not available. To make it available, an administrator can enable the feature flag named packages_dependency_proxy_maven. On GitLab.com, this feature is not available. The feature is not ready for production use.

The GitLab dependency proxy for packages is a local proxy for frequently pulled packages. It is implemented as a pull-through cache that works at the project level.

Packages are pulled from the upstream package registry and automatically published to the project’s package registry. Subsequent identical requests are fulfilled with the project’s package registry. You can use the dependency proxy for packages to reduce unnecessary traffic to the upstream registry.

Enable the dependency proxy

To use the dependency proxy for packages, ensure your project is configured properly, and that users who pull from the cache have the necessary authentication:

  1. In the global configuration, if the following features are disabled, enable them:
  2. In the project settings, if the package feature is disabled, enable it. It is enabled by default.
  3. Add an authentication method. The dependency proxy supports the same authentication methods as the package registry:

Advanced caching

When possible, the dependency proxy for packages uses advanced caching to store packages in the project’s package registry.

Advanced caching verifies the coherence between the project’s package registry and the upstream package registry. If the upstream registry has updated files, the dependency proxy uses them to update the cached files.

When advanced caching is not supported, the dependency proxy falls back to the default behavior:

  • If the requested file is found in the project’s package registry, it is returned.
  • If the file is not found, it is fetched from the upstream package registry.

Advanced caching support depends on how the upstream package registry responds to dependency proxy requests, and on which package format you use.

Maven
Package registry Advanced caching supported?
GitLab Yes
Maven Central Yes
Artifactory Yes
Sonatype Nexus Yes
GitHub Packages No

Permissions

When the dependency proxy pulls a file, the following occurs:

  1. The dependency proxy searches for a file in the project’s package registry. This is a read operation.
  2. The dependency proxy might publish a package file to the project’s package registry. This is a write operation.

Whether both steps are executed depends on user permissions. The dependency proxy uses the same permissions as the package registry.

Project visibility Minimum role Can read package files? Can write package files? Behavior
Public Anonymous No No Request rejected.
Public Guest Yes No Package file returned from either the cache or the remote registry.
Public Developer Yes Yes Package file returned from either the cache or the remote registry. The file is published to the cache.
Internal Anonymous No No Request rejected
Internal Guest Yes No Package file returned from either the cache or the remote registry.
Internal Developer Yes Yes Package file returned from either the cache or the remote registry. The file is published to the cache.
Private Anonymous No No Request rejected
Private Reporter Yes No Package file returned from either the cache or the remote registry.
Internal Developer Yes Yes Package file returned from either the cache or the remote registry. The file is published to the cache.

At a minimum, any user who can use the dependency proxy can also use the project’s package registry.

To ensure the cache is properly filled over time, you should make sure a user with at least the Developer role pulls packages with the dependency proxy.

Configure a client

Configuring a client for the dependency proxy is similar to configuring a client for the package registry.

For Maven packages

For Maven packages, all clients supported by the package registry are supported by the dependency proxy:

  • mvn
  • gradle
  • sbt

For authentication, the Maven dependency proxy access all authentication methods accepted by the Maven package registry. You should use the Basic HTTP authentication method as it is less complex.

To configure the client:

  1. Follow the instructions in Basic HTTP authentication.

    Make sure you use the endpoint URL https://gitlab.example.com/api/v4/projects/<project_id>/dependency_proxy/packages/maven.

  2. Complete the configuration for your client:

mvn

In the pom.xml file add a repository element: 

<repositories>
  <repository>
    <id>gitlab-maven</id>
    <url>https://gitlab.example.com/api/v4/projects/<project_id>/dependency_proxy/packages/maven</url>
  </repository>
</repositories>

Where:

  • <project_id> is the ID of the project to be used as a dependency proxy.
  • <id> contains the name of the <server> used in the authentication configuration.

By default, Maven Central is checked first through the Super POM. However, you might want to force mvn to check the GitLab endpoint first. To do this, follow the instructions from the request forward.

gradle

Add a repositories section to your build.gradle file.

  • In Groovy DSL: 

    repositories {
    maven {
        url "https://gitlab.example.com/api/v4/projects/<project_id>/dependency_proxy/packages/maven"
        name "GitLab"
        credentials(PasswordCredentials) {
            username = 'REPLACE_WITH_NAME'
            password = gitLabPrivateToken
        }
        authentication {
            basic(BasicAuthentication)
        }
    }
}

  • In Kotlin DSL: 

    repositories {
    maven {
        url = uri("https://gitlab.example.com/api/v4/projects/<project_id>/dependency_proxy/packages/maven")
        name = "GitLab"
        credentials(BasicAuthentication::class) {
            username = "REPLACE_WITH_NAME"
            password = findProperty("gitLabPrivateToken") as String?
        }
        authentication {
            create("basic", BasicAuthentication::class)
        }
    }
}

In this example:

  • <project_id> is the ID of the project to be used as a dependency proxy.
  • REPLACE_WITH_NAME is explained in the Basic HTTP authentication section.
sbt

In your build.sbt, add the following lines: 

resolvers += ("gitlab" at "https://gitlab.example.com/api/v4/projects/<project_id>/dependency_proxy/packages/maven")

credentials += Credentials("GitLab Packages Registry", "<host>", "<name>", "<token>")

In this example:

  • <project_id> is the ID of the project to be used as a dependency proxy.
  • <host> is the host present in the <endpoint url> without the protocol scheme or the port. Example: gitlab.example.com.
  • <name> and <token> are explained in the Basic HTTP authentication section.

Configure the remote registry

The dependency proxy must be configured with:

  • The URL of the remote package registry.
  • Optional. The required credentials.

To set those parameters:

  1. Use the related GraphQL mutation.

    A frontend to display and manage these settings is proposed by issue 410726.

  2. Complete the configuration for your package format:

Maven

Any Maven package registry can be connected to the dependency proxy. You can authorize the connection with the Maven package registry username and password.

To set or update the remote Maven package registry, update the following fields on the settings object:

  • mavenExternalRegistryUrl - The URL of the remote registry.
  • mavenExternalRegistryUsername - Optional. The username to use with the remote registry.
  • mavenExternalRegistryPassword - Optional. The password to use with the remote registry.

Example of the GraphQL mutation: 

mutation {
  updateDependencyProxyPackagesSettings(input: {  projectPath : <project full path>, enabled: true, mavenExternalRegistryUrl: <remote registry url>, mavenExternalRegistryUsername: <username>, mavenExternalRegistryPassword: <password> }) {
    dependencyProxyPackagesSetting {
      enabled
      mavenExternalRegistryUrl
      mavenExternalRegistryUsername
    }
    errors
  }
}

Troubleshooting

Manual file pull errors

You can pull files manually with cURL. However, you might encounter one of the following responses:

  • 404 Not Found - The dependency proxy setting object was not found because it doesn’t exist, or because the requirements were not fulfilled.
  • 401 Unauthorized - The user was properly authenticated but did not have the proper permissions to access the dependency proxy object.
  • 403 Forbidden - There was an issue with the GitLab license level.
  • 502 Bad Gateway - The remote package registry could not fulfill the file request. Verify the dependency proxy settings.
  • 504 Gateway Timeout - The remote package registry timed out. Verify the dependency proxy settings.
Maven
curl --verbose "https://<username>:<personal access token>@gitlab.example.com/api/v4/projects/<project_id>/dependency_proxy/packages/maven/<group id and artifact id>/<version>/<file_name>"
  • <username> and <personal access token> are the credentials to access the dependency proxy of the GitLab instance.
  • <project_id> is the project ID.
  • <group id and artifact id> are the Maven package group ID and artifact ID joined with a forward slash.
  • <version> is the package version.
  • file_name is the exact name of the file.

For example, given a package with:

  • group ID: com.my_company.
  • artifact ID: my_package.
  • version: 1.2.3.

The request to manually pull a package is: 

curl --verbose "https://<username>:<personal access token>@gitlab.example.com/api/v4/projects/<project_id>/dependency_proxy/packages/maven/com/my_company/my_package/1.2.3/my_package-1.2.3.pom"