GitOps & Argo CD Simplifying Kubernetes Deployments with Automation

Right now, we have GitLab in place but haven’t fully embraced GitOps. In other words, while Git helps us track code versions, we don’t yet have a system that enforces every change to infrastructure configuration through commits to a repository. We also need to design and implement a CI/CD pipeline that can automatically build, test, and deploy our code.

Our main goal is to deliver an application into a Kubernetes (K8s) cluster. There are multiple strategies for Kubernetes deployments, each with its own pros, cons, and commonly used tools.

Git Repositories and the Push Model

In a push model, building, testing, and deploying an application to the Kubernetes cluster occurs right after every git push. By pushing code changes, a developer sets off a chain of automated steps—ending with a fresh version of the app running for users. Our Git platform provides enough built-in capabilities to support these steps.

How the Push Model Works

1. A developer commits changes in a feature branch or merges them into the main branch.
2. That commit triggers a pipeline — a sequence of automated tasks such as building, testing, and deploying.
3. The build process typically uses Docker or a similar tool to create container images. Tests confirm that the new features work as expected and fit well with existing components.
4. Once the tests pass, the new image is pushed to a container registry, becoming an artifact ready for deployment.
5. Finally, deployment is carried out, placing the new version of the application into the chosen environment (production, staging, development, etc.).

Below are some of the most commonly used tools for Kubernetes deployment:

• Kubectl — A command-line utility for direct interaction with Kubernetes.
• Kustomize — A configuration manager that applies patches to YAML manifests.
• Helm — A package manager for K8s, enabling the packaging and updating of applications.

Kubectl

kubectl is the standard CLI tool for operating applications in a Kubernetes environment. It allows you to deploy, scale, update, and troubleshoot services. You must provide YAML manifests that describe the desired state of Kubernetes resources:

• Deployment controls the rollout process and replica counts.
• Pod is a running instance of your containerized app.
• Service makes the application accessible within or outside the cluster.
• Ingress routes external traffic into the cluster.
• ConfigMap and Secret store configuration data and sensitive credentials, respectively.

Below is an example .pipeline.yml (a placeholder name here) that could live in your repository. It illustrates a deploy stage using kubectl to apply all manifests in the my-demo-app folder:

deploy-kubectl:
  stage: deploy
  image: myregistry/tools/kubectl:latest
  before_script:
    - kubectl config use-context $MY_CLUSTER_CONTEXT
  script:
    - kubectl apply -f my-demo-app/
  rules:
    - if: $CI_COMMIT_BRANCH
      when: manual        

The kubectl apply command ensures the cluster aligns with whatever is defined in your YAML manifests.

Pros of Kubectl

• Versatile: You can create, update, delete, or patch Kubernetes resources and scripts for advanced scenarios like blue-green or canary releases.
• Flexible: It’s the official CLI tool, so it integrates well into various workflows.

Cons of Kubectl

• You need to learn various command parameters and YAML syntax for different resource types.
• Each step requires you to specify your parameters and resources manually.
• A single error can disrupt the entire deployment process.

Kustomize

Kustomize is a tool that manages Kubernetes configurations without templates. It modifies base YAML manifests using “patches” and transformations, and you can then apply the final configuration with kubectl.

Key Concepts in Kustomize

• Base: A directory holding your main YAML manifests.
• Overlay: A directory of patches and transformations that adapt the base to different environments.
• kustomization.yaml: Describes how to apply those bases, overlays, patches, and transformations.
• Patches: YAML files that specify merges or changes to resources.
• Transformations: Bulk modifications applied to all resources in the base.

For instance, you might have a base folder for core settings and an overlays folder for environment-specific patches (dev, staging, prod, etc.). Here’s a snippet of a job that uses Kustomize before deployment:

deploy-kustomize:
  stage: deploy
  image: myregistry/tools/kubectl-kustomize:latest
  before_script:
    - kubectl config use-context $MY_CLUSTER_CONTEXT
    - cd deploy/base
    - kustomize edit set namespace $CI_ENV_NAME-demo
    - kustomize edit set image demo-app=$CI_REGISTRY_IMAGE/demo-app:$CI_COMMIT_SHORT_SHA
    - kustomize build ../overlays/$CI_ENV_NAME > $CI_PROJECT_DIR/final.yaml
  script:
    - kubectl create ns $CI_ENV_NAME-demo || true
    - kubectl apply -f $CI_PROJECT_DIR/final.yaml
  rules:
    - if: $CI_COMMIT_BRANCH
      when: manual        

In this setup, kustomization.yaml instructs Kustomize on how to handle patching.

Why Choose Kustomize

• Declarative: Clearly states how configurations are layered.
• Reusability: Easily adapt the same base for different stages or environments.
• Straightforward syntax: No custom templating language.

Potential Downsides

• Requires a bit of a learning curve to master patching effectively.

Helm

Helm is the package manager for Kubernetes. It bundles everything needed for your application into what’s called a “chart” (Deployments, Services, ConfigMaps, etc.). Helm uses Go-based templating, where user-provided values (via a values.yaml file) help generate the final Kubernetes manifests. Charts can also contain sub-charts and are often stored in a registry.

When you run commands like helm install, Helm automatically sets up all the necessary resources:

deploy-helm:
  stage: deploy
  image: myregistry/tools/helm-kubectl:latest
  before_script:
    - kubectl config use-context $MY_CLUSTER_CONTEXT
  script:
    - helm upgrade --install my-demo-app ./chart \
        -f values.yaml \
        -n demo-namespace --create-namespace
  rules:
    - if: $CI_COMMIT_BRANCH
      when: manual        

Helm will deploy or update my-demo-app using the chart in the chart directory, along with the configurations in values.yaml.

Benefits of Helm

• Simplified deployment, with no need to manually manage multiple YAML files.
• Built-in versioning and the ability to roll back to a previous chart version.
• Great for reusing charts across various environments.
• Extensive community support and numerous official or community-maintained charts for popular applications.

Drawbacks

• Learning Go templating and chart structures can be challenging for newcomers.
• A misconfigured chart may introduce security or reliability issues.
• Secrets shouldn’t be stored in charts (use Kubernetes Secrets or external vaults instead).

Pull Model, GitOps, and Argo CD

Why Consider a Different Approach

When you deploy your application via a push model, Kubernetes stores resource definitions in its internal database, etcd. This can lead to configuration drift:

• Kubernetes attempts to reconcile the “desired state” in etcd with the actual running resources.
• If someone manually alters something in the cluster, the new state can inadvertently become the one that Kubernetes “remembers.”
• If a resource is removed directly from the cluster, it’s essentially lost unless re-applied from scratch.

A GitOps approach with Argo CD helps eliminate these issues by flipping the workflow to a pull model.

Pull Model

GitOps often implies that the cluster “pulls” in changes rather than having them pushed in after every commit. You still store all your infrastructure configuration in a Git repository. However, an automated agent (frequently running inside the cluster) keeps track of changes in that repo and updates the cluster whenever it detects differences.

The key advantage here is that any manual, out-of-band edits to cluster resources will be overwritten by the agent, which always reverts to the state described in Git. That forces all modifications to go through the repository, ensuring consistency.

Argo CD

Argo CD is a popular, declarative GitOps tool for continuous delivery in Kubernetes.

Argo CD’s Main Components

• API Server — Responsible for authentication and provides an API to interact with Argo CD.
• Application Controller — Continuously compares your cluster state to what’s in Git and reconciles any mismatch.
• Repository Server — Caches your Git repo, stores commit references, and includes tools to render manifests (Helm, Kustomize, etc.).

Unlike manually applying YAML files, Argo CD manages a higher-level concept called an Application, defining how and where to deploy resources (e.g., referencing a Helm chart, Kustomize folder, or plain YAML).

Example Application manifest to deploy my-sample-app:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: my-sample-app
  namespace: argocd
spec:
  source:
    path: sample-app
    repoURL: https://some-git-service.com/my-org/k8s-config.git
    targetRevision: HEAD
  destination:
    namespace: sample-app
    server: https://kubernetes.default.svc
  project: production        

With many services, manually creating an Application for each one is cumbersome. ApplicationSet helps automate this process based on “generators” that watch for changes in your repo structure:

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: all-tenants
  namespace: argocd
spec:
  generators:
  - git:
      repoURL: https://some-git-service.com/my-org/k8s-config.git
      revision: HEAD
      directories:
      - path: tenants/*        

Here, the Git generator checks for new folders under tenants/. If a folder appears, Argo CD automatically creates an Application for it.

Argo CD revolves around Applications and Projects, which help manage environments (dev, staging, prod) or multiple clusters:

• Application — A logical bundle of Kubernetes resources that references a specific path (Helm, Kustomize, plain YAML, etc.).
• Project — Organizes Applications, controlling which clusters and repos are accessible.

Argo CD includes a handy UI for viewing synchronization status, comparing versions (diff), and checking logs.

Unified Deployment

Modern development often involves different kinds of projects, some of which need to be migrated between clusters or revived after a long period without updates. It can get messy when each project has its own custom pipeline.

GitOps (particularly Argo CD) can standardize the process. Instead of designing elaborate, one-off pipelines, you only need to modify the Docker image tag in your GitOps repository, and Argo CD automatically updates your cluster.

Below is an example of a job that clones a GitOps repo, updates a values.yaml field (app.image.tag) with the current commit’s short SHA, and commits that change back:

.deploy_template:
  stage: deploy
  image: myregistry/tools/yq:latest
  script:
    - git clone "https://$GIT_USER:$GIT_TOKEN@$GITOPS_REPO_URL" my-gitops
    - cd my-gitops
    - yq -i ".app.image.tag = strenv(CI_COMMIT_SHORT_SHA)" path/to/myapp/values.yaml
    - git config --global user.email "auto-deploy@org.local"
    - git config --global user.name "CI Pipeline"
    - git commit -am "Deploy myapp - ${CI_COMMIT_SHORT_SHA}"
    - git push        

Once the new tag is in values.yaml, Argo CD notices the change and updates Kubernetes accordingly, ensuring your environment matches the config in Git.

Our Approach

Here’s how we structure our process:

1. Separate Repos: We intentionally divide the application’s source code and the environment configuration into two different repositories.
2. Umbrella Charts: For managing multiple dependencies, we store “umbrella” Helm charts in our internal registry. These charts orchestrate various components in a unified way.

The final workflow looks something like this:

Benefits of Argo CD

• Convenient web UI for monitoring deployments.
• Continuous synchronization, avoiding drift and maintaining availability.
• A single source of truth for configurations, ensuring consistency.
• Easy to scale up or replicate services across multiple clusters.

Drawbacks

• Initial setup requires time and effort, plus a good strategy for writing reusable Helm charts or other templates.

Conclusion

Adopting GitOps principles and implementing Argo CD enabled us to unify our pipelines, reducing maintenance and complexity across multiple projects. Configuration drift is no longer a problem because the live cluster state always matches the Git repository. Also, Kubernetes makes scaling or moving services straightforward under this model.

Argo CD offers powerful capabilties, but to really get the most out of it, your team needs a strong understanding of base in Kubernetes. Many companies opt for a hosted or managed Kubernetes solution to handle tasks like auto-scaling, self-healing or updates, and monitoring — freeing developers to focus on creating and delivering top-notch applications. By pairing a managed Kubernetes service with “Argo CD”, you will have an automated pipeline from code to your production, significantly reducing the effort to spent on day-to-day infrastructure management.