Multiple GitLab projects on the same Kubernetes Cluster (GitLab CI/CD)

My team has recently boarded the hype train on Kubernetes deployments. We can get rid of a bunch of staging servers and never look back. While doing so, we also don’t seek after high-cost cluster maintenance infrastructures.

Recently I got myself a basic standard-type 2-node cluster to do some tests and deploy some of the company’s projects onto it. While things seemed to progress extremely fast with the lovely yaml files that could be used as the base from one project to another, running lines and lines of kubectl did not seem like the neatest way to do things.

GitLab CI/CD is GitLab’s built-in tool for software development using continuous methodology []. GitLab supports Kubernetes cluster connections. You can learn how to connect your project to your cluster through this. All seemed good and dandy. The connection is safe and happens through a specific serviceaccount made by GitLab. Nothing seems to be the matter, we’re happy!

The time came for me to apply the same to my other projects (all of which have their own GitLab groups). I use the certificate, token and URL, but the cluster simply doesn’t like the new project. It begs the question, hmmm, is it even possible to connect my cluster to multiple projects? One would think so. So I went on and emailed GitLab and here’s what I heard back: Your best bet would be to use group-level clusters only for projects in the same group.

Disappointed as I was, I did not accept the defeat. Here is how I hacked my way into connecting multiple projects to the same Kubernetes cluster on GKE.

  1. Get the first project connected in the standard way

GitLab’s own on this does an okay job of leading you through the steps of getting the first cluster connected. Here is a summary of what they walk you through:

  • Create a gitlab-admin serviceaccount.
  • Create the gitlab-admin cluster role binding with RBAC authorization.
  • Get the gitlab-admin secret you just created in the kube-systemnamespace, and take the token out. This will be used as your TOKEN.
  • Get your project’s namespace’s default token secret and extract the certificate. This will be your CA.
  • Get your cluster’s public URL (which could be an IP address). This will be your API URL.
  • Your project’s namespace (which has to be unique — and not default) will be your NAMESPACE.
  • Now connect your cluster through the GitLab Kubernetes portal.

NOTE: make sure that this went through by clicking on install Helm Tillerand GitLab Runner.

Once this goes through, and those sweet sweet twirling things show you’ve completed installing Helm Tiller and Runner on your cluster, go ahead and look at the resources in the gitlab-managed-apps namespace on your cluster.

Note: here you might get a Kubernetes error, or “can’t start installation” error. This might be due to different reasons. I personally was forgetting to input the header and the footer of my certificate file. There are tons of issues on GitHub out there explaining how to fix this.

2. Connecting the second, third, fourth, …. nth project to your cluster!

This is really really simple. Ready? Here it goes.

Use the same CA, TOKEN, CLUSTER NAME, and API URL as your first project. What is now different is the namespace. This has to be your new project’s namespace and has to be unique. Now you will need to check the RBAC option as you are adding your cluster.

Once that is done, go ahead and delete the gitlab-managed-appsnamespace. YES, you heard me right!

$ kubectl delete namespace gitlab-managed-apps

Now add your cluster. What you need to do now is to update your kubeconfig file. If you are using Google Kubernetes Engine, you should run the following:

$ gcloud container clusters get-credentials <cluster name> — zone <zone> — project <project name>

Now you will need to install Helm Tiller and GitLab Runner again in your GitLab dashboard. Once that is done, you are likely golden!

The same process can be done multiple times on your new projects. This is not a standard process, it is pretty hacky and might be flakey. So make sure that your old projects do not get disconnected from the cluster if you make a mistake in the process.