-
Manual secret creation (optional)
- Registry authentication certificates
- Registry sensitive notification headers
- SSH Host Keys
- Initial Enterprise license
- Initial root password
- Redis password
- GitLab Shell secret
- Gitaly secret
- Praefect secret
- GitLab Rails secret
- GitLab Workhorse secret
- GitLab Runner secret
- GitLab KAS secret
- GitLab KAS API secret
- GitLab Suggested Reviewers secret
- MinIO secret
- PostgreSQL password
- GitLab Pages secret
- Registry HTTP secret
- Registry notification secret
- Praefect DB password
-
External services
- OmniAuth
- LDAP Password
- SMTP password
- IMAP password for incoming emails
- IMAP password for Service Desk emails
- GitLab incoming email auth token
- GitLab Service Desk email auth token
- Zoekt basic auth password
- Microsoft Graph client secret for incoming emails
- Microsoft Graph client secret for Service Desk emails
- Microsoft Graph client secret for outgoing emails
- S/MIME Certificate
- Smartcard Authentication
- OAuth integration
- Next steps
- Rotating secrets
Configure secrets for the GitLab chart
GitLab requires a variety of secrets to operate:
GitLab Components:
- Registry authentication certificates
- SSH Host Keys and Certificates for GitLab Shell
- Passwords for individual GitLab services
- TLS certificate for GitLab Pages
Optional External Services:
- SMTP server
- LDAP
- OmniAuth
- IMAP for incoming emails (via mail_room service)
- IMAP for Service Desk emails (via mail_room service)
- Microsoft Graph with OAuth2 for incoming emails (via mail_room service)
- Microsoft Graph with OAuth2 for Service Desk email (via mail_room service)
- Microsoft Graph with OAuth2 for outgoing emails
- S/MIME certificate
- Smartcard authentication
- OAuth integration
Any secret not provided manually will be automatically generated with a random value. Automatic generation of HTTPS certificates is provided by Let’s Encrypt.
To utilize autogenerated secrets continue to next steps.
To specify your own secrets, proceed to manual secret creation.
Manual secret creation (optional)
Use gitlab
as the release name if you followed previous steps in this
documentation.
- TLS certificates
- Registry authentication certificates
- Registry sensitive notification headers
- SSH Host Keys
- Passwords:
- Initial root password
- Redis password
- GitLab Shell secret
- Gitaly secret
- Praefect secret
- GitLab Rails secret
- GitLab Workhorse secret
- GitLab Runner secret
- PostgreSQL password
- Praefect DB password
- MinIO secret
- Registry HTTP secret
- Registry notification secret
- GitLab Pages secret
- GitLab incoming email auth token
- GitLab Service Desk email auth token
- Zoekt basic auth password
- External Services
Registry authentication certificates
Communication between GitLab and Registry happens behind an Ingress so it is sufficient in most cases to use self-signed certificates for this communication. If this traffic is exposed over a network, you should generate publicly valid certificates.
In the example below, we assume that we require self-signed certificates.
Generate a certificate-key pair:
mkdir -p certs
openssl req -new -newkey rsa:4096 -subj "/CN=gitlab-issuer" -nodes -x509 -keyout certs/registry-example-com.key -out certs/registry-example-com.crt
Create a secret containing these certificates.
We will create registry-auth.key
and registry-auth.crt
keys inside the
<name>-registry-secret
secret. Replace <name>
with the name of the release.
kubectl create secret generic <name>-registry-secret --from-file=registry-auth.key=certs/registry-example-com.key --from-file=registry-auth.crt=certs/registry-example-com.crt
This secret is referenced by the global.registry.certificate.secret
setting.
Registry sensitive notification headers
Check documentation regarding configuring Registry notifications for more details.
The secret content should be a list of items, even if it contains a single item. If the content is just a string, the charts WILL NOT convert it to a list as needed.
Consider the example where registry-authorization-header
secret with value
RandomFooBar
is created.
kubectl create secret generic registry-authorization-header --from-literal=value="[RandomFooBar]"
By default, the key used within the secret is “value”. However, users can use a
different key, but must ensure that it’s specified as key
under the header map
item.
SSH Host Keys
Generate the OpenSSH certificate-key pairs:
mkdir -p hostKeys
ssh-keygen -t rsa -f hostKeys/ssh_host_rsa_key -N ""
ssh-keygen -t dsa -f hostKeys/ssh_host_dsa_key -N ""
ssh-keygen -t ecdsa -f hostKeys/ssh_host_ecdsa_key -N ""
ssh-keygen -t ed25519 -f hostKeys/ssh_host_ed25519_key -N ""
Create the secret containing these certificates. Replace <name>
with the name
of the release.
kubectl create secret generic <name>-gitlab-shell-host-keys --from-file hostKeys
This secret is referenced by the global.shell.hostKeys.secret
setting.
If this secret is rotated, all SSH clients will see hostname mismatch
errors.
Initial Enterprise license
Create a Kubernetes secret for storing the Enterprise license for the GitLab instance.
Replace <name>
with the name of the release.
kubectl create secret generic <name>-gitlab-license --from-file=license=/tmp/license.gitlab
Then use --set global.gitlab.license.secret=<name>-gitlab-license
to
inject the license into your configuration.
You can also use the global.gitlab.license.key
option to change the default
license
key pointing to the license in the license secret.
Initial root password
Create a Kubernetes secret for storing the initial root password. The password
should be at least 6 characters long. Replace <name>
with the name of the
release.
kubectl create secret generic <name>-gitlab-initial-root-password --from-literal=password=$(head -c 512 /dev/urandom | LC_CTYPE=C tr -cd 'a-zA-Z0-9' | head -c 32)
Redis password
Generate a random 64 character alpha-numeric password for Redis. Replace
<name>
with the name of the release.
kubectl create secret generic <name>-redis-secret --from-literal=secret=$(head -c 512 /dev/urandom | LC_CTYPE=C tr -cd 'a-zA-Z0-9' | head -c 64)
If deploying with an already existing Redis cluster, please use the password for accessing the Redis cluster that has been base64 encoded instead of a randomly generated one.
This secret is referenced by the global.redis.auth.secret
setting.
GitLab Shell secret
Generate a random 64 character alpha-numeric secret for GitLab Shell. Replace
<name>
with the name of the release.
kubectl create secret generic <name>-gitlab-shell-secret --from-literal=secret=$(head -c 512 /dev/urandom | LC_CTYPE=C tr -cd 'a-zA-Z0-9' | head -c 64)
This secret is referenced by the global.shell.authToken.secret
setting.
Gitaly secret
Generate a random 64 character alpha-numeric token for Gitaly. Replace <name>
with the name of the release.
kubectl create secret generic <name>-gitaly-secret --from-literal=token=$(head -c 512 /dev/urandom | LC_CTYPE=C tr -cd 'a-zA-Z0-9' | head -c 64)
This secret is referenced by the global.gitaly.authToken.secret
setting.
Praefect secret
Generate a random 64 character alpha-numeric token for Praefect. Replace <name>
with the name of the release:
kubectl create secret generic <name>-praefect-secret --from-literal=token=$(head -c 512 /dev/urandom | LC_CTYPE=C tr -cd 'a-zA-Z0-9' | head -c 64)
This secret is referenced by the global.praefect.authToken.secret
setting.
GitLab Rails secret
Replace <name>
with the name of the release.
cat << EOF > secrets.yml
production:
secret_key_base: $(head -c 512 /dev/urandom | LC_CTYPE=C tr -cd 'a-zA-Z0-9' | head -c 128)
otp_key_base: $(head -c 512 /dev/urandom | LC_CTYPE=C tr -cd 'a-zA-Z0-9' | head -c 128)
db_key_base: $(head -c 512 /dev/urandom | LC_CTYPE=C tr -cd 'a-zA-Z0-9' | head -c 128)
encrypted_settings_key_base: $(head -c 512 /dev/urandom | LC_CTYPE=C tr -cd 'a-zA-Z0-9' | head -c 128)
openid_connect_signing_key: |
$(openssl genrsa 2048 | awk '{print " " $0}')
ci_jwt_signing_key: |
$(openssl genrsa 2048 | awk '{print " " $0}')
EOF
kubectl create secret generic <name>-rails-secret --from-file=secrets.yml
This secret is referenced by the global.railsSecrets.secret
setting.
It is not recommended to rotate this secret as it contains the database encryption keys. If the secret is rotated, the result will be the same behavior exhibited when the secrets file is lost.
encrypted_settings_key_base
was added in GitLab 13.7
, and will be required for GitLab 14.0
.GitLab Workhorse secret
Generate the workhorse secret. This must have a length of 32 characters and
base64-encoded. Replace <name>
with the name of the release.
kubectl create secret generic <name>-gitlab-workhorse-secret --from-literal=shared_secret=$(head -c 512 /dev/urandom | LC_CTYPE=C tr -cd 'a-zA-Z0-9' | head -c 32 | base64)
This secret is referenced by the global.workhorse.secret
setting.
GitLab Runner secret
Replace <name>
with the name of the release.
kubectl create secret generic <name>-gitlab-runner-secret --from-literal=runner-registration-token=$(head -c 512 /dev/urandom | LC_CTYPE=C tr -cd 'a-zA-Z0-9' | head -c 64)
This secret is referenced by the gitlab-runner.runners.secret
setting.
GitLab KAS secret
GitLab Rails requires that a secret for KAS is present, even if one deploys this chart without installing the KAS sub-chart. Still, one can create this secret manually by following the below procedure or leave it to the chart to auto-generate the secret.
Replace <name>
with the name of the release.
kubectl create secret generic <name>-gitlab-kas-secret --from-literal=kas_shared_secret=$(head -c 512 /dev/urandom | LC_CTYPE=C tr -cd 'a-zA-Z0-9' | head -c 32 | base64)
This secret is referenced by the global.appConfig.gitlab_kas.key
setting.
GitLab KAS API secret
You can leave it to the chart to auto-generate the secret, or you can create this secret manually (replace <name>
with the name of the release):
kubectl create secret generic <name>-kas-private-api --from-literal=kas_private_api_secret=$(head -c 512 /dev/urandom | LC_CTYPE=C tr -cd 'a-zA-Z0-9' | head -c 32 | base64)
This secret is referenced by the gitlab.kas.privateApi.secret
setting.
GitLab Suggested Reviewers secret
GitLab Rails requires that a secret for Suggested Reviewers is present. You can
leave it to the chart to auto-generate the secret, or you can create this secret
manually (replace <name>
with the name of the release):
kubectl create secret generic <name>-gitlab-suggested-reviewers --from-literal=suggested_reviewers_secret=$(head -c 512 /dev/urandom | LC_CTYPE=C tr -cd 'a-zA-Z0-9' | head -c 32 | base64)
This secret is referenced by the global.appConfig.suggested_reviewers.secret
setting.
MinIO secret
Generate a set of random 20 & 64 character alpha-numeric keys for MinIO.
Replace <name>
with the name of the release.
kubectl create secret generic <name>-minio-secret --from-literal=accesskey=$(head -c 512 /dev/urandom | LC_CTYPE=C tr -cd 'a-zA-Z0-9' | head -c 20) --from-literal=secretkey=$(head -c 512 /dev/urandom | LC_CTYPE=C tr -cd 'a-zA-Z0-9' | head -c 64)
This secret is referenced by the global.minio.credentials.secret
setting.
PostgreSQL password
Generate a random 64 character alpha-numeric password. Replace <name>
with
the name of the release.
kubectl create secret generic <name>-postgresql-password \
--from-literal=postgresql-password=$(head -c 512 /dev/urandom | LC_CTYPE=C tr -cd 'a-zA-Z0-9' | head -c 64) \
--from-literal=postgresql-postgres-password=$(head -c 512 /dev/urandom | LC_CTYPE=C tr -cd 'a-zA-Z0-9' | head -c 64)
This secret is referenced by the global.psql.password.secret
setting.
Changing the PostgreSQL password for the bundled PostgreSQL subchart
The bundled PostgreSQL subchart only configures the database with the passwords from the secret when the database is initially created. Additional steps need to be taken to change the passwords in an existing database.
Please note this operation will be disruptive to users while the change is being made.
To rotate the PostgreSQL secret:
- Complete the general rotating secrets instructions for the PostgreSQL secret.
-
Exec into the PostgreSQL pod and update the passwords in the database:
# Exec into the PostgreSQL pod kubectl exec -it <name>-postgresql-0 -- sh # Inside the pod, update the passwords in the database sed -i 's/^\(local .*\)md5$/\1trust/' /opt/bitnami/postgresql/conf/pg_hba.conf pg_ctl reload ; sleep 1 echo "ALTER USER postgres WITH PASSWORD '$(echo $POSTGRES_POSTGRES_PASSWORD)' ; ALTER USER gitlab WITH PASSWORD '$(echo POSTGRES_PASSWORD)'" | psql -U postgres -d gitlabhq_production -f - sed -i 's/^\(local .*\)trust$/\1md5/' /opt/bitnami/postgresql/conf/pg_hba.conf pg_ctl reload
- Delete the
gitlab-exporter
,postgresql
,toolbox
,sidekiq
andwebservice
pods using thekubectl delete pod
command so the new pods are loaded with the new secret and allow them to connect to the database.
GitLab Pages secret
Generate the GitLab Pages secret. This must have a length of 32 characters and
base64-encoded. Replace <name>
with the name of the release.
kubectl create secret generic <name>-gitlab-pages-secret --from-literal=shared_secret=$(head -c 512 /dev/urandom | LC_CTYPE=C tr -cd 'a-zA-Z0-9' | head -c 32 | base64)
This secret is referenced by the global.pages.apiSecret.secret
setting.
Registry HTTP secret
Generate a random 64 character alpha-numeric key shared by all registry pods.
Replace <name>
with the name of the release.
kubectl create secret generic <name>-registry-httpsecret --from-literal=secret=$(head -c 512 /dev/urandom | LC_CTYPE=C tr -cd 'a-zA-Z0-9' | head -c 64 | base64)
This secret is referenced by the global.registry.httpSecret.secret
setting.
Registry notification secret
Generate a random 32 character alpha-numeric key shared by all registry pods, and the GitLab webservice pods.
Replace <name>
with the name of the release.
kubectl create secret generic <name>-registry-notification --from-literal=secret=[\"$(head -c 512 /dev/urandom | LC_CTYPE=C tr -cd 'a-zA-Z0-9' | head -c 32)\"]
This secret is referenced by the global.registry.notificationSecret.secret
setting.
Praefect DB password
Generate a random 64 character alpha-numeric password. Replace <name>
with
the name of the release:
kubectl create secret generic <name>-praefect-dbsecret \
--from-literal=secret=$(head -c 512 /dev/urandom | LC_CTYPE=C tr -cd 'a-zA-Z0-9' | head -c 64) \
This secret is referenced by the global.praefect.dbSecret
setting.
External services
Some charts have further secrets to enable functionality that can not be automatically generated.
OmniAuth
In order to enable the use of OmniAuth Providers with the deployed GitLab, please follow the instructions in the Globals chart
LDAP Password
If you need password authentication to connect with your LDAP server, you must store the password in a Kubernetes secret.
kubectl create secret generic ldap-main-password --from-literal=password=yourpasswordhere
Then use --set global.appConfig.ldap.servers.main.password.secret=ldap-main-password
to
inject the password into your configuration.
Secret
name, not the actual password when configuring the Helm property.SMTP password
If you are using an SMTP server that requires authentication, store the password in a Kubernetes secret.
kubectl create secret generic smtp-password --from-literal=password=yourpasswordhere
Then use --set global.smtp.password.secret=smtp-password
in your Helm command.
Secret
name, not the actual password when configuring the Helm property.IMAP password for incoming emails
GitLab uses authentication strings such as app passwords, tokens, or IMAP passwords to access incoming emails.
Find your email provider in the GitLab incoming email documentation and set its required authentication string as a Kubernetes secret.
kubectl create secret generic incoming-email-password --from-literal="password=auth_string_for_your_provider_here"
Then use --set global.appConfig.incomingEmail.password.secret=incoming-email-password
in your Helm command along with other required settings as specified in the docs.
Secret
name, not the actual password when configuring the Helm property.IMAP password for Service Desk emails
GitLab uses authentication strings such as app passwords, tokens, or IMAP passwords to access Service Desk emails.
Find your email provider in the GitLab incoming email documentation and set its required authentication string as a Kubernetes secret.
kubectl create secret generic service-desk-email-password --from-literal="password=auth_string_for_your_provider_here"
Then use --set global.appConfig.serviceDeskEmail.password.secret=service-desk-email-password
in your Helm command along with other required settings as specified in the docs.
Secret
name, not the actual password when configuring the Helm property.GitLab incoming email auth token
When incoming email is configured to use webhook delivery method, there should
be a shared secret between mail_room service and webservice. This must have a
length of 32 characters and base64-encoded. Replace <name>
with the name of
the release.
kubectl create secret generic <name>-incoming-email-auth-token --from-literal=authToken=$(head -c 512 /dev/urandom | LC_CTYPE=C tr -cd 'a-zA-Z0-9' | head -c 32 | base64)
This secret is referenced by the global.incomingEmail.authToken
setting.
GitLab Service Desk email auth token
When Service Desk email is configured to use webhook delivery method, there should
be a shared secret between mail_room service and webservice. This must have a
length of 32 characters and base64-encoded. Replace <name>
with the name of
the release.
kubectl create secret generic <name>-service-desk-email-auth-token --from-literal=authToken=$(head -c 512 /dev/urandom | LC_CTYPE=C tr -cd 'a-zA-Z0-9' | head -c 32 | base64)
This secret is referenced by the global.serviceDeskEmail.authToken
setting.
Zoekt basic auth password
You can leave it to the chart to auto-generate the secret, or you can create this secret manually (replace <name>
with the name of the release):
password=$(head -c 512 /dev/urandom | LC_CTYPE=C tr -cd 'a-zA-Z0-9' | head -c 32 | base64)
kubectl create secret generic <name>-zoekt-basicauth --from-literal=gitlab_username=gitlab --from-literal=gitlab_password="$password"
This secret is referenced by the gitlab.zoekt.gateway.basicAuth.secretName
setting.
Microsoft Graph client secret for incoming emails
To let GitLab have access to incoming emails store the password of the IMAP account in a Kubernetes secret:
kubectl create secret generic incoming-email-client-secret --from-literal=secret=your-secret-here
Then, use --set global.appConfig.incomingEmail.clientSecret.secret=incoming-email-client-secret
in your Helm command along with other required settings as specified in the docs.
Secret
name, not the actual password when configuring the Helm property.Microsoft Graph client secret for Service Desk emails
To let GitLab have access to service_desk emails store the password of the IMAP account in a Kubernetes secret:
kubectl create secret generic service-desk-email-client-secret --from-literal=secret=your-secret-here
Then, use --set global.appConfig.serviceDeskEmail.clientSecret.secret=service-desk-email-client-secret
in your Helm command along with other required settings as specified in the docs.
Secret
name, not the actual password when configuring the Helm property.Microsoft Graph client secret for outgoing emails
Store the password in a Kubernetes secret:
kubectl create secret generic microsoft-graph-mailer-client-secret --from-literal=secret=your-secret-here
Then, use --set global.appConfig.microsoft_graph_mailer.client_secret.secret=microsoft-graph-mailer-client-secret
in your Helm command.
Secret
name, not the actual password when configuring the Helm property.S/MIME Certificate
Outgoing email messages can be digitally signed using the S/MIME standard. The S/MIME certificate needs to be stored in a Kubernetes secret as a TLS type secret.
kubectl create secret tls smime-certificate --key=file.key --cert file.crt
If there is an existing secret as a opaque type, then the global.email.smime.keyName
and global.email.smime.certName
values will need to be adjusted for
the specific secret.
S/MIME settings can be set through the values.yaml
file or on the command
line. Use --set global.email.smime.enabled=true
to enable S/MIME and
--set global.email.smime.secretName=smime-certificate
to specify the
secret that contains the S/MIME certificate.
Smartcard Authentication
Smartcard authentication uses a custom Certificate Authority (CA) to sign client certificates. The certificate of this custom CA needs to be injected to the Webservice pod for it to verify whether a client certificate is valid or not. This is provided as a k8s secret.
kubectl create secret generic <secret name> --from-file=ca.crt=<path to CA certificate>
The key name inside the secret where the certificate is stored MUST BE
ca.crt
.
OAuth integration
For configuring OAuth integration of various services like GitLab Pages, secrets
containing OAuth credentials are required. The secret should contain an App ID
(by default, stored under the appid
key), and an App Secret (by default,
stored under the appsecret
key), both of which are recommended to be
alphanumeric strings, at least 64 characters long.
kubectl create secret generic oauth-gitlab-pages-secret --from-literal=appid=<app id> --from-literal=appsecret=<app secret>
This secret can be specified using the global.oauth.<service name>.secret
setting. If keys other than appid
and appsecret
are used, they can be
specified using global.oauth.<service name>.appIdKey
and
global.oauth.<service name>.appSecretKey
settings.
Next steps
Once all secrets have been generated and stored, you can proceed deploying GitLab.
Rotating secrets
The secrets can be rotated if required for security purposes.
- Back up your current secrets.
- For your convenience, create new secrets that are suffixed with
-v2
(for examplegitlab-shell-host-keys-v2
) by following the manual secret creation steps for each secret you wish to rotate. - Update the secret keys in your
values.yaml
file to point to the new secret names. Most secret names are documented under each respective secret in the manual secret creation section. - Upgrade the GitLab Chart release with the updated
values.yaml
file. - If you are rotating the PostgreSQL secret, there are additional steps to complete the rotation.
- Confirm that GitLab is working as expected. If it is, it should be safe to delete the old secrets.