Authenticating and reading secrets with HashiCorp Vault Premium All offerings
CI_JOB_JWT
was deprecated in GitLab 15.9
and the token is scheduled to be removed in GitLab 17.0. Use
ID tokens to authenticate with HashiCorp Vault
instead, as demonstrated on this page.This tutorial demonstrates how to authenticate, configure, and read secrets with HashiCorp’s Vault from GitLab CI/CD.
Prerequisites
This tutorial assumes you are familiar with GitLab CI/CD and Vault.
To follow along, you must have:
- An account on GitLab.
- Access to a running Vault server (at least v1.2.0) to configure authentication and to create roles and policies. For HashiCorp Vaults, this can be the Open Source or Enterprise version.
vault.example.com
URL below with the URL of your Vault server, and gitlab.example.com
with the URL of your GitLab instance.How it works
ID tokens are JSON Web Tokens (JWTs) used for OIDC authentication with third-party services. If a job has at least one ID token defined, the secrets
keyword automatically uses that token to authenticate with Vault.
The following fields are included in the JWT:
Field | When | Description |
---|---|---|
jti
| Always | Unique identifier for this token |
iss
| Always | Issuer, the domain of your GitLab instance |
iat
| Always | Issued at |
nbf
| Always | Not valid before |
exp
| Always | Expires at |
sub
| Always | Subject (job ID) |
namespace_id
| Always | Use this to scope to group or user level namespace by ID |
namespace_path
| Always | Use this to scope to group or user level namespace by path |
project_id
| Always | Use this to scope to project by ID |
project_path
| Always | Use this to scope to project by path |
user_id
| Always | ID of the user executing the job |
user_login
| Always | Username of the user executing the job |
user_email
| Always | Email of the user executing the job |
pipeline_id
| Always | ID of this pipeline |
pipeline_source
| Always | Pipeline source |
job_id
| Always | ID of this job |
ref
| Always | Git ref for this job |
ref_type
| Always | Git ref type, either branch or tag
|
ref_path
| Always | Fully qualified ref for the job. For example, refs/heads/main . Introduced in GitLab 16.0.
|
ref_protected
| Always |
true if this Git ref is protected, false otherwise
|
environment
| Job specifies an environment | Environment this job specifies (introduced in GitLab 13.9) |
environment_protected
| Job specifies an environment |
true if specified environment is protected, false otherwise (introduced in GitLab 13.9)
|
deployment_tier
| Job specifies an environment | Deployment tier of environment this job specifies (introduced in GitLab 15.2) |
environment_action
| Job specifies an environment |
Environment action (environment:action ) specified in the job. (Introduced in GitLab 16.5)
|
Example JWT payload:
{
"jti": "c82eeb0c-5c6f-4a33-abf5-4c474b92b558",
"iss": "gitlab.example.com",
"iat": 1585710286,
"nbf": 1585798372,
"exp": 1585713886,
"sub": "job_1212",
"namespace_id": "1",
"namespace_path": "mygroup",
"project_id": "22",
"project_path": "mygroup/myproject",
"user_id": "42",
"user_login": "myuser",
"user_email": "myuser@example.com",
"pipeline_id": "1212",
"pipeline_source": "web",
"job_id": "1212",
"ref": "auto-deploy-2020-04-01",
"ref_type": "branch",
"ref_path": "refs/heads/auto-deploy-2020-04-01",
"ref_protected": "true",
"environment": "production",
"environment_protected": "true",
"environment_action": "start"
}
The JWT is encoded by using RS256 and signed with a dedicated private key. The expire time for the token is set to job’s timeout, if specified, or 5 minutes if it is not. The key used to sign this token may change without any notice. In such case retrying the job generates new JWT using the current signing key.
You can use this JWT for authentication with a Vault server that is configured to allow
the JWT authentication method. Provide your GitLab instance’s base URL
(for example https://gitlab.example.com
) to your Vault server as the oidc_discovery_url
.
The server can then retrieve the keys for validating the token from your instance.
When configuring roles in Vault, you can use bound claims to match against the JWT claims and restrict which secrets each CI/CD job has access to.
To communicate with Vault, you can use either its CLI client or perform API requests (using curl
or another client).
Example
Let’s say you have the passwords for your staging and production databases stored in a Vault server that is running on http://vault.example.com:8200
. Your staging password is pa$$w0rd
and your production password is real-pa$$w0rd
.
$ vault kv get -field=password secret/myproject/staging/db
pa$$w0rd
$ vault kv get -field=password secret/myproject/production/db
real-pa$$w0rd
To configure your Vault server, start by enabling the JWT Auth method:
$ vault auth enable jwt
Success! Enabled jwt auth method at: jwt/
Then create policies that allow you to read these secrets (one for each secret):
$ vault policy write myproject-staging - <<EOF
# Policy name: myproject-staging
#
# Read-only permission on 'secret/myproject/staging/*' path
path "secret/myproject/staging/*" {
capabilities = [ "read" ]
}
EOF
Success! Uploaded policy: myproject-staging
$ vault policy write myproject-production - <<EOF
# Policy name: myproject-production
#
# Read-only permission on 'secret/myproject/production/*' path
path "secret/myproject/production/*" {
capabilities = [ "read" ]
}
EOF
Success! Uploaded policy: myproject-production
You also need roles that link the JWT with these policies.
One for staging named myproject-staging
:
$ vault write auth/jwt/role/myproject-staging - <<EOF
{
"role_type": "jwt",
"policies": ["myproject-staging"],
"token_explicit_max_ttl": 60,
"user_claim": "user_email",
"bound_claims": {
"project_id": "22",
"ref": "master",
"ref_type": "branch"
}
}
EOF
And one for production named myproject-production
:
$ vault write auth/jwt/role/myproject-production - <<EOF
{
"role_type": "jwt",
"policies": ["myproject-production"],
"token_explicit_max_ttl": 60,
"user_claim": "user_email",
"bound_claims_type": "glob",
"bound_claims": {
"project_id": "22",
"ref_protected": "true",
"ref_type": "branch",
"ref": "auto-deploy-*"
}
}
EOF
This example uses bound claims to specify that only a JWT with matching values for the specified claims is allowed to authenticate.
Combined with protected branches, you can restrict who is able to authenticate and read the secrets.
Any of the claims included in the JWT can be matched against a list of values in the bound claims. For example:
"bound_claims": {
"user_login": ["alice", "bob", "mallory"]
}
"bound_claims": {
"ref": ["main", "develop", "test"]
}
"bound_claims": {
"namespace_id": ["10", "20", "30"]
}
"bound_claims": {
"project_id": ["12", "22", "37"]
}
- If only
namespace_id
is used, all projects in the namespace are allowed. - If both
namespace_id
andproject_id
are used, Vault first checks if the project’s namespace is innamespace_id
. If not, it then checks if the project is inproject_id
.
token_explicit_max_ttl
specifies that the token issued by Vault, upon successful authentication, has a hard lifetime limit of 60 seconds.
user_claim
specifies the name for the Identity alias created by Vault upon a successful login.
bound_claims_type
configures the interpretation of the bound_claims
values. If set to glob
, the values are interpreted as globs, with *
matching any number of characters.
The claim fields listed in the table above can also be accessed for Vault’s policy path templating purposes by using the accessor name of the JWT auth within Vault. The mount accessor name (ACCESSOR_NAME
in the example below) can be retrieved by running vault auth list
.
Policy template example making use of a named metadata field named project_path
:
path "secret/data/{{identity.entity.aliases.ACCESSOR_NAME.metadata.project_path}}/staging/*" {
capabilities = [ "read" ]
}
Role example to support the templated policy above, mapping the claim field project_path
as a metadata field through use of claim_mappings
configuration:
{
"role_type": "jwt",
...
"claim_mappings": {
"project_path": "project_path"
}
}
For the full list of options, see Vault’s Create Role documentation.
project_id
or namespace_id
). Otherwise any JWT generated by this instance may be allowed to authenticate using this role.Now, configure the JWT Authentication method:
$ vault write auth/jwt/config \
oidc_discovery_url="https://gitlab.example.com" \
bound_issuer="https://gitlab.example.com"
bound_issuer
specifies that only a JWT with the issuer (that is, the iss
claim) set to gitlab.example.com
can use this method to authenticate, and that the oidc_discovery_url
(https://gitlab.example.com
) should be used to validate the token.
For the full list of available configuration options, see Vault’s API documentation.
In GitLab, create the following CI/CD variables to provide details about your Vault server:
-
VAULT_SERVER_URL
- The URL of your Vault server, for examplehttps://vault.example.com:8200
. -
VAULT_AUTH_ROLE
- Optional. The role to use when attempting to authenticate. If no role is specified, Vault uses the default role specified when the authentication method was configured. -
VAULT_AUTH_PATH
- Optional. The path where the authentication method is mounted. Default isjwt
. -
VAULT_NAMESPACE
- Optional. The Vault Enterprise namespace to use for reading secrets and authentication. If no namespace is specified, Vault uses the root (/
) namespace. The setting is ignored by Vault Open Source.
The following job, when run for the default branch, can read secrets under secret/myproject/staging/
, but not the secrets under secret/myproject/production/
:
job_with_secrets:
id_tokens:
VAULT_ID_TOKEN:
aud: https://example.vault.com
secrets:
STAGING_DB_PASSWORD:
vault: secret/myproject/staging/db/password@secrets # authenticates using $VAULT_ID_TOKEN
script:
- access-staging-db.sh --token $STAGING_DB_PASSWORD
In this example:
-
@secrets
- The vault name, where your Secrets Engines are enabled. -
secret/myproject/staging/db
- The path location of the secret in Vault. -
password
The field to be fetched within the referenced secret.
Limit token access to Vault secrets
You can control ID token access to Vault secrets by using Vault protections and GitLab features. For example, restrict the token by:
- Using Vault bound claims
for specific groups using
group_claim
. - Hard coding values for Vault bound claims based on the
user_login
anduser_email
of specific users. - Setting Vault time limits for TTL of the token as specified in
token_explicit_max_ttl
, where the token expires after authentication. - Scoping the JWT to GitLab protected branches that are restricted to a subset of project users.
- Scoping the JWT to GitLab protected tags, that are restricted to a subset of project users.