Kubernetes Config Management Landscape

Kubernetes Conﬁg
Management Landscape
Tomasz Tarczyński, Gigaset

Agenda
@ttarczynski
1. Conﬁguration Management
2. Kubernetes
3. Declarative application management
4. Tools Landscape

Conﬁguration Management
@ttarczynski
“Of all the problems we have confronted, the ones over
which the most brainpower, ink, and code have been
spilled are related to managing conﬁgurations.”
Borg, Omega, and Kubernetes (2016)
Brendan Burns, Brian Grant, David Oppenheimer, Eric Brewer, John Wilkes

@ttarczynski
● Infrastructure as Code
● GIT – as a single source of truth
● Tools:
CFEngine / Puppet / Chef / Ansible

@ttarczynski
● Tools:
○ CFEngine (1993) Mark Burgess
○ Puppet (2005) Luke Kanies
○ Chef (2009) Adam Jacob
○ Ansible (2012) Michael DeHaan

Puppet
@ttarczynski
● Declarative – describe the desired state
● Resources automation – Types / providers
● DSL – a simple and constrained language
● Templates – ERB
● Modules – public and private
● Code / data separation – Hiera

Kubernetes
@ttarczynski
● Control plane provides a Declarative API
● Declare the desired state
● Control plane makes sure that the actual
state converges to the desired state

Kubernetes
@ttarczynski
● Counting the YAMLs (real env example)
○ 54 standard k8s resource types
○ 50 CRD types
○ 790 API object (user-provided)
○ 5,500 API objects (total)
○ 107,000 lines of YAML (user-provided)
○ 507,000 lines of YAML (total)

Kubernetes
@ttarczynski
● “Are we all YAML engineers now?”
– Bob Walker at CfgMgmtCampt 2018

Kubernetes: Declarative application management
@ttarczynski
● “Declarative application management in
Kubernetes”
– article by Brian Grant, 2017

@ttarczynski
● Central Idea:
○ Composable tools
○ Manipulate conﬁguration data
○ Declarative data model
● NOT: Conﬁguration as Code

@ttarczynski
● Conﬁguration as Data
○ Code goes into controllers (control loops)
○ Data in native Kubernetes API format

@ttarczynski
● Declarative – k8s API
● Resources automation – k8s controllers
● DSL –
● Templates –
● Modules –
● Code / data separation – controllers / API

Tools Landscape
● List by Brian Grant
● Currently 125 tools on the list

Tools Landscape – k8s config mgmt
@ttarczynski
● Approaches to config:
○ Fork
○ Overlay / patch
○ Parametrization
○ Generation
○ DSL (domain-specific language)
○ GPL (general-purpose language)

Tools Landscape – tools (1–3)
@ttarczynski
● 3 most widely used tools:
○ Jsonnet (2014) 4,5k ⭐
○ Helm (2015) 18,6k ⭐
○ Kustomize (2018) 6,5k ⭐

@ttarczynski
● Jsonnet
○ Data templating language (DSL)
○ Generates JSON
○ Hermetic / side-effect free
○ Superset of JSON

@ttarczynski
● Helm
○ Template YAML (with Go template)
○ Package cfg into charts
○ Simple sharing (public / private repos)
○ Manage releases (in Kubernetes)

@ttarczynski
● Kustomize
○ Template-free / DSL-free
○ Plain YAML
○ Customize Kubernetes objects
○ Built into kubectl

@ttarczynski
● GitOps tools:
○ Flux CD (2016) 5,9k ⭐
○ Argo CD (2018) 4,6k ⭐

@ttarczynski
● GitOps tools:
○ Keep Kubernetes clusters in sync with Git
○ Use operator in the cluster
● Integrations:
○ Flux CD – Helm, Kustomize
○ Argo CD – Helm, Kustomize, Jsonnet

@ttarczynski
● More tools:
○ Kapitan (2017) 1,3k ⭐
○ Tanka (2019) 1,2k ⭐
○ Kpt (2019) 0,7k ⭐

@ttarczynski
● Kapitan:
○ Template engines – Jsonnet, Jinja2, Kadet, Helm
○ Inventory – YAML-based, hierarchical
○ Compile k8s manifests and store in git

@ttarczynski
● Tanka:
○ By Grafana Labs
○ Powered by Jsonnet
○ Jsonnet Kubernetes library
○ Replacement for ksonnet
○ Integrates with: Helm, Kustomize

@ttarczynski
● Kpt:
○ Treat conﬁguration as data
○ Provides a packaging solution (Git-based)
○ Run functions to: generate, transform, validate cfg
○ Integrates with: Kustomize, Helm, (anything via fns)

@ttarczynski
● More tools:
○ Ytt (2019) 0,7k ⭐
○ Cue (2019) 2,4k ⭐

@ttarczynski
● ytt:
○ YAML structure-aware templating
○ Based on Starlark (Python-like language)
○ Conceptually close to Jsonnet
○ Supports patch/overlay approach

@ttarczynski
● Cue:
○ Data constraint language
○ Superset of JSON
○ Data validation
○ Reduce boilerplate

@ttarczynski
● IaC tools:
○ Terraform (2014) 25,2k ⭐
○ Pulumi (2016) 7,3k ⭐
○ Cdk8s (2020) 1,9k ⭐

@ttarczynski
● Terraform:
○ HCL (HashiCorp Conﬁguration Language)
■ JSON compatible DSL
■ Declarative
○ Providers:
■ Kubernetes – 34 core resource types
■ Kubernetes-alpha – any resource (incl. CRDs)
■ Helm – manage charts

@ttarczynski
● Pulumi:
○ Infrastructure as Code SDK
○ General-purpose programming language:
■ JavaScript, TypeScript, Python, Go, .NET
○ Supports: Helm, Kustomize, plain YAML
○ SDK API:
■ 100% compatible with the Kubernetes API
■ Supports CRDs

@ttarczynski
● Cdk8s:
○ CDK for Kubernetes
○ General-purpose programming language:
■ Typescript, JavaScript, Python, and Java
○ Supports: Helm, plain YAML
○ Covers:
■ 100% of the Kubernetes API
■ CRDs

Tools Landscape – summary
@ttarczynski

Tools Landscape – CNCF Tech Radar
@ttarczynski

Links
@ttarczynski
1. Borg, Omega, and Kubernetes – Brendan Burns, Brian Grant, David Oppenheimer, Eric Brewer,
John Wilkes, 2016
2. Are we all YAML engineer now? – Bob Walker at CfgMgmtCampt, 2018
3. Declarative application management in Kubernetes – Brian Grant, 2017
4. Kubernetes application management tools – Brian Grant, 2017
5. CNCF End User Technology Radar – Continuous Delivery – CNCF, 2020
6. All materials used for this talk – me, 2021
7. Tools comparison spreadsheet – me, 2021

Kubernetes Config Management Landscape

More Related Content

What's hot (20)

Similar to Kubernetes Config Management Landscape (20)

More from Tomasz Tarczyński (9)

Recently uploaded (20)

Kubernetes Config Management Landscape