This page contains information related to upcoming products, features, and functionality. It is important to note that the information presented is for informational purposes only. Please do not rely on this information for purchasing or planning purposes. As with all projects, the items mentioned on this page are subject to change or delay. The development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc.
Status Authors Coach DRIs Owning Stage Created
implemented @shinya.maeda @DylanGriffith @nagyv-gitlab @cbalane @hustewart @hfyngvason devops deploy 2022-11-23

View and manage resources deployed by GitLab agent For Kubernetes

Summary

As part of the GitLab Kubernetes Dashboard epic, users want to view and manage their resources deployed by GitLab agent For Kubernetes. Users should be able to interact with the resources through GitLab UI, such as Environment Index/Details page.

This blueprint describes how the association is established and how these domain models interact with each other.

Motivation

Goals

Non-Goals

Proposal

Overview

  • GitLab Environment and GitLab agent For Kubernetes have 1-to-1 relationship.
  • GitLab Environment tracks all resources produced by the connected agent. This includes not only resources written in manifest files but also subsequently generated resources (for example, Pods created by Deployment manifest file).
  • GitLab Environment renders dependency graph, such as Deployment => ReplicaSet => Pod. This is for providing ArgoCD-style resource view.
  • GitLab Environment has the Resource Health status that represents a summary of resource statuses, such as Healthy, Progressing or Degraded.
GitLab
Kubernetes
Organization
Project
production environment
staging environment
Production
Service
Deployment
Pod1
Pod2
Staging
Service
Deployment
Pod1
Pod2

Existing components and relationships

Issues

  • GitLab Environment should have ID of GitLab agent For Kubernetes as the foreign key.
  • GitLab Environment should have parameters how to group resources in the associated cluster, for example, namespace, lable and inventory-id (GitOps mode only) can passed as parameters.
  • GitLab Environment should be able to fetch all relevant resources, including both default resource kinds and other Custom Resources.
  • GitLab Environment should be aware of dependency graph.
  • GitLab Environment should be able to compute Resource Health status from the associated resources.

Example

This is an example of how the architecture works in push-based deployment. The feature is documented here as CI access mode.

GitLab
Production Kubernetes
Organization
OperationGroup
DevelopmentGroup
AgentManagementProject
DeploymentProject
Shared with
Deploy
Deploy
Deploy
FrontendAppProject
VueJS
Dockerfile
BackendAppProject
Golang
Dockerfile
Kubernetes Manifest Files
CI/CD pipelines
production frontend environment
production backend environment
Production agent
Kubernetes Manifest Files
production prometheus environment
CI/CD pipelines
Production
Service
Deployment
Pod1
Pod2
Staging
Service
Deployment
Pod1
Pod2
Monitoring
Service
Deployment
Pod1
Pod2

Further details

Multi-Project Deployment Pipelines

The microservice project setup can be improved by Multi-Project Deployment Pipelines:

  • Deployment Project can behave as the shared deployment engine for any upstream application projects and environments.
  • Environments can be created within the application projects. It gives more visibility of environments for developers.
  • Deployment Project can be managed under Operator group. More segregation of duties.
  • Users don’t need to set up RBAC to restrict CI/CD jobs.
  • This is especitially helpful for dynamic environments, such as Review Apps.
GitLab
Production Kubernetes
Organization
OperationGroup
DevelopmentGroup
DeploymentProject
FrontendAppProject
BackendAppProject
Trigger downstream pipeline
Trigger downstream pipeline
Deploy
Deploy
CI/CD pipelines
production environment
CI/CD pipelines
production environment
Production agent
Kubernetes Manifest Files
production prometheus environment
CI/CD pipelines
Frontend
Service
Deployment
Pod1
Pod2
Backend
Service
Deployment
Pod1
Pod2
Monitoring
Service
Deployment
Pod1
Pod2

Design and implementation details

Associate Environment with Agent

Users can explicitly set a GitLab agent For Kubernetes to a GitLab Environment in setting UI. Frontend will use this associated agent for authenticating/authorizing the user access, which is described in a latter section.

We need to adjust the read_cluster_agent permission in DeclarivePolicy for supporting agents shared by an external project (also known as the Agent management project).

Fetch resources through user_access

When user visits an environment page, GitLab frontend fetches an environment via GraphQL. Frontend additionally fetches the associated agent-ID and namespace.

Here is an example of GraphQL query:

{
  project(fullPath: "group/project") {
    id
    environment(name: "<environment-name>") {
      slug
      kubernetesNamespace
      clusterAgent {
        id
        name
        project {
          name
        }
      }
    }
  }
}

GitLab frontend authenticate/authorize the user access with browser cookie. If the access is forbidden, frontend shows an error message that You don't have access to an agent that deployed to this environment. Please contact agent administrator if you are allowed in "user_access" in agent config file. See <troubleshooting-doc-link>.

After the user gained access to the agent, GitLab frontend fetches specific Resource kinds (for example, Deployment, Pod) in the Kubernetes with the following parameters:

  • namespace#{environment.kubernetesNamespace}

If no resources are found, this is likely that the users have not embedded these lables into their resources. In this case, frontend shows an warning message There are no resources found for the environment. Do resources have GitLab preserved labels? See <troubleshooting-doc-link>.

Dependency graph

  • GitLab frontend uses Owner References to idenfity the dependencies between resources. These are embedded in resources as metadata.ownerReferences field.
  • For the resoruces that don’t have owner references, we can use Well-Known Labels, Annotations and Taints as complement. for example, EndpointSlice doesn’t have metadata.ownerReferences, but has kubernetes.io/service-name as a reference to the parent Service resource.

Health status of resources

  • GitLab frontend computes the status summary from the fetched resources. Something similar to ArgoCD’s Resource Health for example, Healthy, Progressing, Degraded and Suspended. The formula is TBD.