GitLab Helm chart 3.0

We have bumped the chart version to 3.0 in order to take in several major changes in our chart dependencies, some of which require manual actions to complete the upgrade.

Summary of major changes

Problematic Helm 2.15

Helm v2.15.x has a severe bug, and absolutely should not be used.

If you are to use Helm 2, use 2.14.3 or >= 2.16.1.

Upgrade path from 2.x

In order to upgrade to the 3.0 version of the chart, you first need to upgrade to the latest 2.6.x release of the chart. Check the version mapping details for the latest patch.

If you don’t first upgrade to the latest 2.6.x patch, you will see the following error from helm upgrade 

Error: UPGRADE FAILED: Job failed: BackoffLimitExceeded

You can then confirm you are in this situation by looking for pods in error with the text gitlab-upgrade-check in the name.

If you check the logs for those pods will see the version upgrade error message: 

It seems you are upgrading the GitLab Helm chart from X (GitLab X) to 3.0.0 (GitLab 12.7.0).
It is required to upgrade to the last minor version in a major version series
first before jumping to the next major version.
Please follow the upgrade documentation at https://docs.gitlab.com/charts/releases/3_0.html
and upgrade to GitLab Helm chart version 2.6.0 before upgrading to 3.0.0.

Upgrade from 2.6.x

Upgrading to the 3.0 chart requires manual upgrade steps in order to update some of the components. Please follow the upgrade steps for 3.0 release.

Major Changes

PostgreSQL

As part of the 3.0.0 release of this chart, we upgraded the bundled PostgreSQL chart from 0.11.0 to 7.7.3. This updates the database version from 9.6 to 10.9. This is not a drop in replacement. Manual steps need to be performed to upgrade the database.

The 3.0 upgrade steps includes the manual steps required during upgrade.

Further details can be found in our PostgreSQL upgrade issue.

note
If you are using an external PostgreSQL chart, you do not need to make changes to your database for this release. 9.6 is still supported, though we recommend upgrading to PostgreSQL 10.

NGINX Ingress

We addressed issue #1710, and that change will fix future upgrades, but requires a manual intervention when upgrading from a version of the chart prior to 3.0.

The 3.0 upgrade steps includes the manual steps required during upgrade.

Further details on this can be found in our troubleshooting documentation, under Immutable Field Error, spec.clusterIP.

Redis

As part of our Redis upgrade we’ve dropped our fork of the Redis and Redis HA charts and have instead switched to using a newer version of the upstream Redis chart.

This brings with it an update to using Redis 5.x, with improved performance.

  • For users of the previous bundled Redis chart, there will be no changes required to upgrade to the new Redis version.
  • For users of the previous Redis HA chart, there are some additional flags you need to enable to put Redis in an HA configuration.
  • For users of an external Redis database. The syntax for disabling the bundled database has changed to redis.install=false. (From redis.enabled=false)
note
The bundled Redis will use a new different Persistent Volume from the previous installation. This means there will be downtime during the upgrade, and that all users sessions will be logged out once completed.

Prometheus

The Prometheus chart has been updated to 10.0.0. This brings in the latest changes for the chart, which include removing deprecated APIs, that were preventing installation into Kubernetes 1.16.

This component does not require any manual upgrade steps, but it was required that users have already upgraded to the 9.0 Prometheus chart before upgrading further. We included 9.0 in GitLab Helm chart 2.5.0, so we placed the new version in this 3.0 release, which requires users who are upgrading to be on the GitLab Helm chart 2.6.0 release or newer.

See our Prometheus upgrade issue for further details.

Sidekiq Selectors

Previously, the Sidekiq chart did not assign unique selectors to deployments. This prevented deployments from being able to properly identify their Sidekiq pods and clean up as necessary.

These selectors are immutable fields in the Deployment Spec, so in order to update them, the Sidekiq deployments need to be deleted, then recreated. As part of the 3.0.0 release, this is done automatically by Helm by appending -v1 to the name of the Sidekiq Deployments,HPAs, and Pods.

Additional details on the can be found in the troubleshooting documentation for Immutable Field Error, spec.selector.

Upgrade path from 1.x

You first need to upgrade to the 2.6.x release of the charts, before upgrading to 3.0. Please follow the 2.0 upgrade documentation.

Known issues and limitations

Below is a list of the known issues and limitations, although it may not be exhaustive.

Helm chart issues/limitations:

Features that are currently out of scope:

Release cadence

We will be releasing a new version of the chart with each new GitLab patch.

More information on how we are versioning the chart can be found in the release documentation.

Along with the issues and merge requests in this repository, a changelog is available to more easily follow along with updates.

Kubernetes deployment support

GitLab is tested against:

Other Kubernetes deployments should also work. In the event of a specific non-GKE deployment issue, please raise an issue.

We are currently using Kubernetes version 1.12.10 in our automated tests, and 1.13.11 for development.

Technical support

Before opening an issue please review the known issues and limitations, and search to see if a similar issue already exists.

We greatly appreciate the wider testing of the community, and encourage detailed issues to be reported so we can address them.

We welcome any improvements contributed in the form of Merge Requests.