Responding to security incidents

When a security incident occurs, you should follow the processes defined by your organization. However, you might consider some additional steps. These suggestions are intended to supplement existing security incident response processes within your organization.

Administrators can choose to provide public security contact information to support receiving reports of security issues from security researchers.

Suspected compromised user account

If you suspect that a user account or bot account has been compromised, consider taking the following steps:

  • Block the user to mitigate any current risk.
  • Review the audit events available to you to identify any suspicious account behavior. For example:
    • Suspicious sign-in events.
    • Creation or deletion of personal access tokens, project access tokens, and group access tokens.
    • Creation or deletion of SSH or GPG keys.
    • Creation, modification, or deletion of two-factor authentication.
    • Changes to repositories.
    • Changes to group or project configurations.
    • Addition or modification of runners.
    • Addition or modification of webhooks or Git hooks.
  • Reset any credentials the user might have had access to. For example, users with at least the Maintainer role can view protected CI/CD variables and runner registration tokens.
  • Reset the user’s password.
  • Get the user to enable two factor authentication (2FA), and consider enforcing 2FA at the instance or group level
  • After completing an investigation and mitigating impacts, unblock the user.

Suspected compromised instance

Self-managed GitLab customers and administrators are responsible for:

  • The security of their underlying hosts.
  • Keeping GitLab itself up to date.

It is important to regularly update GitLab, update your operating system and its software, and harden your hosts in accordance with vendor guidance.

If you suspect that your GitLab instance has been compromised, consider taking the following steps:

  • Review the audit events available to you for suspicious account behavior.
  • Review all users (including the Administrative root user), and follow the steps in Suspected compromised user account if necessary.
  • Review the Credentials Inventory, if available to you.
  • Change any sensitive credentials, variables, tokens, and secrets. For example, those located in instance configuration, database, CI/CD pipelines, or elsewhere.
  • Update to the latest version of GitLab and adopt a plan to update after every security patch release.

In addition, the suggestions below are common steps taken in incident response plans when servers are compromised by malicious actors.

caution
Use these suggestions at your own risk.
  • Save any server state and logs to a write-once location, for later investigation.
  • Look for unrecognized background processes.
  • Check for open ports on the system.
  • Rebuild the host from a known-good backup or from scratch, and apply all the latest security patches.
  • Review network logs for uncommon traffic.
  • Establish network monitoring and network-level controls.
  • Restrict inbound and outbound network access to authorized users and servers only.
  • Ensure all logs are routed to an independent write-only datastore.