• Installation options
  • For Linux package installations
    • Disable on a single node
      • Configure KAS to listen on a UNIX socket
    • Enable on multiple nodes
      • Agent server node settings
  • For GitLab Helm Chart
• Kubernetes API proxy cookie
• Troubleshooting
  • GitOps: failed to get project information
  • Configuration file not found
  • dial tcp <GITLAB_INTERNAL_IP>:443: connect: connection refused

Install the GitLab agent server for Kubernetes (KAS)

The agent server is a component installed together with GitLab. It is required to manage the GitLab agent for Kubernetes.

The KAS acronym refers to the former name, Kubernetes agent server.

The agent server for Kubernetes is installed and available on GitLab.com at wss://kas.gitlab.com. If you use self-managed GitLab, by default the agent server is installed and available.

Installation options

As a GitLab administrator, you can control the agent server installation:

• For Linux package installations.
• For GitLab Helm chart installations.

For Linux package installations

The agent server for Linux package installations can be enabled on a single node, or on multiple nodes at once. By default, the agent server is enabled and available at ws://gitlab.example.com/-/kubernetes-agent/.

Disable on a single node

To disable the agent server on a single node:

1. Edit /etc/gitlab/gitlab.rb:

   gitlab_kas['enable'] = false
   

2. Reconfigure GitLab.

Configure KAS to listen on a UNIX socket

If you use GitLab behind a proxy, KAS might not work correctly. You can resolve this issue on a single-node installation, you can configure KAS to listen on a UNIX socket.

To configure KAS to listen on a UNIX socket:

1. Create a directory for the KAS sockets:

   sudo mkdir -p /var/opt/gitlab/gitlab-kas/sockets/
   

2. Edit /etc/gitlab/gitlab.rb:

   gitlab_kas['internal_api_listen_network'] = 'unix'
   gitlab_kas['internal_api_listen_address'] = '/var/opt/gitlab/gitlab-kas/sockets/internal-api.socket'
   gitlab_kas['private_api_listen_network'] = 'unix'
   gitlab_kas['private_api_listen_address'] = '/var/opt/gitlab/gitlab-kas/sockets/private-api.socket'
   gitlab_kas['env'] = {
     'SSL_CERT_DIR' => "/opt/gitlab/embedded/ssl/certs/",
     'OWN_PRIVATE_API_URL' => 'unix:///var/opt/gitlab/gitlab-kas/sockets/private-api.socket'
   }
   

3. Reconfigure GitLab.

For additional configuration options, see the GitLab Kubernetes Agent Server section of the gitlab.rb.template.

Enable on multiple nodes

To enable the agent server on multiple nodes:

1. For each agent server node, edit /etc/gitlab/gitlab.rb:

   gitlab_kas_external_url 'wss://kas.gitlab.example.com/'
   
   gitlab_kas['api_secret_key'] = '<32_bytes_long_base64_encoded_value>'
   gitlab_kas['private_api_secret_key'] = '<32_bytes_long_base64_encoded_value>'
   gitlab_kas['private_api_listen_address'] = '0.0.0.0:8155'
   gitlab_kas['env'] = {
     'SSL_CERT_DIR' => "/opt/gitlab/embedded/ssl/certs/",
     'OWN_PRIVATE_API_URL' => 'grpc://<ip_or_hostname_of_this_host>:8155'
   }
   
   gitlab_rails['gitlab_kas_external_url'] = 'wss://gitlab.example.com/-/kubernetes-agent/'
   gitlab_rails['gitlab_kas_internal_url'] = 'grpc://kas.internal.gitlab.example.com'
   gitlab_rails['gitlab_kas_external_k8s_proxy_url'] = 'https://gitlab.example.com/-/kubernetes-agent/k8s-proxy/'
   

2. Reconfigure GitLab.

Agent server node settings

1. For example, wss://kas.gitlab.example.com/.
2. For example, wss://gitlab.example.com/-/kubernetes-agent/.

For GitLab Helm Chart

See how to use the GitLab-KAS chart.

Kubernetes API proxy cookie

KAS proxies Kubernetes API requests to the GitLab agent with either:

• A CI/CD job.
• GitLab user credentials.

To authenticate with user credentials, Rails sets a cookie for the GitLab frontend. This cookie is called _gitlab_kas and it contains an encrypted session ID, like the _gitlab_session cookie. The _gitlab_kas cookie must be sent to the KAS proxy endpoint with every request to authenticate and authorize the user.

Troubleshooting

If you have issues while using the agent server for Kubernetes, view the service logs by running the following command:

kubectl logs -f -l=app=kas -n <YOUR-GITLAB-NAMESPACE>

In Linux package installations, find the logs in /var/log/gitlab/gitlab-kas/.

You can also troubleshoot issues with individual agents.

GitOps: failed to get project information

If you get the following error message:

{"level":"warn","time":"2020-10-30T08:37:26.123Z","msg":"GitOps: failed to get project info","agent_id":4,"project_id":"root/kas-manifest001","error":"error kind: 0; status: 404"}

The project specified by the manifest (root/kas-manifest001) doesn’t exist or the project where the manifest is kept is private. To fix this issue, ensure the project path is correct and that the project’s visibility is set to public.

Configuration file not found

If you get the following error message:

time="2020-10-29T04:44:14Z" level=warning msg="Config: failed to fetch" agent_id=2 error="configuration file not found: \".gitlab/agents/test-agent/config.yaml\

The path is incorrect for either:

• The repository where the agent was registered.
• The agent configuration file.

To fix this issue, ensure that the paths are correct.

dial tcp <GITLAB_INTERNAL_IP>:443: connect: connection refused

If you are running self-managed GitLab and:

• The instance isn’t running behind an SSL-terminating proxy.
• The instance doesn’t have HTTPS configured on the GitLab instance itself.
• The instance’s hostname resolves locally to its internal IP address.

When the agent server tries to connect to the GitLab API, the following error might occur:

{"level":"error","time":"2021-08-16T14:56:47.289Z","msg":"GetAgentInfo()","correlation_id":"01FD7QE35RXXXX8R47WZFBAXTN","grpc_service":"gitlab.agent.reverse_tunnel.rpc.ReverseTunnel","grpc_method":"Connect","error":"Get \"https://gitlab.example.com/api/v4/internal/kubernetes/agent_info\": dial tcp 172.17.0.4:443: connect: connection refused"}

To fix this issue for Linux package installations, set the following parameter in /etc/gitlab/gitlab.rb. Replace gitlab.example.com with your GitLab instance’s hostname:

gitlab_kas['gitlab_address'] = 'http://gitlab.example.com'