![@bridgetkromhout #helmsummit
kubeval: find invalid deployments
$ helm kubeval stable/nginx-ingress --set
controller.replicaCount=two
[…]
The file nginx-ingress/templates/controller-
deployment.yaml contains an invalid Deployment
---> spec.replicas: Invalid type. Expected:
[integer,null], given: string
The file nginx-ingress/templates/default-backend-
deployment.yaml contains a valid Deployment
[…]
Error: plugin "kubeval" exited with error](https://image.slidesharecdn.com/helmsummit-190912141835/85/Increasing-Reliability-via-Helm-Pre-Release-Checks-Helm-Summit-2019-12-320.jpg)
![@bridgetkromhout #helmsummit
kubeval: will a chart work with a given version?
$ helm kubeval stable/nginx-ingress -v 1.15.0
The file nginx-ingress/templates/controller-
serviceaccount.yaml contains a valid ServiceAccount
The file nginx-ingress/templates/default-backend-
serviceaccount.yaml contains a valid ServiceAccount
[…]](https://image.slidesharecdn.com/helmsummit-190912141835/85/Increasing-Reliability-via-Helm-Pre-Release-Checks-Helm-Summit-2019-13-320.jpg)
![@bridgetkromhout #helmsummit
$ helm conftest stable/nginx-ingress
FAIL - nginx-ingress-controller in the Deployment
release-name-nginx-ingress-controller does not have
a memory limit set
FAIL - nginx-ingress-controller in the Deployment
release-name-nginx-ingress-controller does not have
a CPU limit set
[…]
Error: plugin "conftest" exited with error
conftest: fail if non-compliant with policy](https://image.slidesharecdn.com/helmsummit-190912141835/85/Increasing-Reliability-via-Helm-Pre-Release-Checks-Helm-Summit-2019-16-320.jpg)
Increasing Reliability via Helm Pre-Release Checks (Helm Summit 2019)
- 1. @bridgetkromhout #helmsummit Increasing Reliability via Helm Pre-Release Checks
- 3. @bridgetkromhout #helmsummit Where is Waldo Lachie? (and he still wrote most of this talk, too!)
- 4. @bridgetkromhout #helmsummit Image credit: Vasa Museet failed Helm release circa 1628
- 5. @bridgetkromhout #helmsummit Image credit: Vasa Museet a successful Helm release …has gotten harder (because k8s is vast and contains multitudes)
- 6. @bridgetkromhout #helmsummit open-source tooling for more reliable Helm releases kubeval conftest kubectl auth can-i
- 7. @bridgetkromhout #helmsummit $ helm install stable/nginx-ingress Let’s choose a chart to use
- 8. @bridgetkromhout #helmsummit Helm Pre-Release Checks resource validity policy role based access control
- 9. @bridgetkromhout #helmsummit invalid k8s resources $ helm install stable/nginx-ingress --set controller.replicaCount=two Error: release estranged-arachnid failed: Deployment in version "v1beta1" cannot be handled as a Deployment: v1beta1.Deployment.Spec: v1beta1.DeploymentSpec.Replicas: readUint32: unexpected character: , error found in #10 byte of ...|eplicas":"two","revi|..., bigger context ...|default"},"spec":{"minReadySeconds": 0,"replicas":"two","revisionHistoryLimit": 10,"strategy":{},"temp|...
- 10. @bridgetkromhout #helmsummit resources don’t work!? (…on this k8s version) https://kubernetes.io/blog/2019/07/18/api-deprecations-in-1-16/ $ helm install stable/nginx-ingress Error: validation failed: unable to recognize "": no matches for kind "Deployment" in version "extensions/ v1beta1"
- 11. @bridgetkromhout #helmsummit $ helm plugin install https:// github.com/instrumenta/helm-kubeval kubeval: install as Helm plugin @garethr - kubeval.instrumenta.dev
- 12. @bridgetkromhout #helmsummit kubeval: find invalid deployments $ helm kubeval stable/nginx-ingress --set controller.replicaCount=two […] The file nginx-ingress/templates/controller- deployment.yaml contains an invalid Deployment ---> spec.replicas: Invalid type. Expected: [integer,null], given: string The file nginx-ingress/templates/default-backend- deployment.yaml contains a valid Deployment […] Error: plugin "kubeval" exited with error
- 13. @bridgetkromhout #helmsummit kubeval: will a chart work with a given version? $ helm kubeval stable/nginx-ingress -v 1.15.0 The file nginx-ingress/templates/controller- serviceaccount.yaml contains a valid ServiceAccount The file nginx-ingress/templates/default-backend- serviceaccount.yaml contains a valid ServiceAccount […]
- 14. @bridgetkromhout #helmsummit Helm Pre-Release Checks resource validity policy role based access control
- 15. @bridgetkromhout #helmsummit conftest openpolicyagent.org Open Policy Agent https://garethr.dev/2019/06/introducing-conftest/ Policy-based control specified declaratively & enforced automatically Write policy in OPA native query language Rego test locally against structured configuration data (uses Rego) (enforced server-side: PodSecurityPolicy, Gatekeeper, etc)
- 16. @bridgetkromhout #helmsummit $ helm conftest stable/nginx-ingress FAIL - nginx-ingress-controller in the Deployment release-name-nginx-ingress-controller does not have a memory limit set FAIL - nginx-ingress-controller in the Deployment release-name-nginx-ingress-controller does not have a CPU limit set […] Error: plugin "conftest" exited with error conftest: fail if non-compliant with policy
- 17. @bridgetkromhout #helmsummit conftest: succeed when explicitly setting limits $ helm conftest stable/nginx-ingress/ —set controller.resources.limits.cpu=100m,controller .resources.limits.memory=64Mi $
- 18. @bridgetkromhout #helmsummit Helm Pre-Release Checks resource validity policy role based access control
- 19. @bridgetkromhout #helmsummit RBAC tl;dr: if you don’t have permissions, you’ll have a failed deployment…
- 20. @bridgetkromhout #helmsummit … and spoiler alert: in Helm 3, with Tiller gone, you won’t have the “cluster admin” permissions anymore!
- 21. @bridgetkromhout #helmsummit $ for i in `helm template stable/nginx-ingress | grep -i Kind | awk -F: '{print $2}' | sort -u`; do echo "$i: `kubectl auth can-i create $i`"; done Warning: resource 'clusterroles' is not namespace scoped in group 'rbac.authorization.k8s.io' ClusterRole: no Warning: resource 'clusterrolebindings' is not namespace scoped in group 'rbac.authorization.k8s.io' ClusterRoleBinding: no Deployment: yes Role: yes RoleBinding: yes Service: yes ServiceAccount: yes kubectl auth can-i
- 22. @bridgetkromhout #helmsummit multiple options: kubectl auth can-i (https://kubernetes.io/docs/reference/access-authn- authz/authorization/#checking-api-access) who-can (https://github.com/aquasecurity/kubectl-who-can)
- 23. @bridgetkromhout #helmsummit great! what now? - add pre-release checks to your CI/CD pipelines: - kubeval - conftest - kubectl auth can-i - prevent broken releases - … - profit!
- 25. @bridgetkromhout #helmsummit To learn more… Cloud Native Tooling deislabs.io Helm FAQ v3.helm.sh/docs/faq Container Training container.training What is Kubernetes? aka.ms/k8slearning
- 26. @bridgetkromhout #helmsummit Thanks! Cloud Native Tooling deislabs.io Helm FAQ v3.helm.sh/docs/faq Container Training container.training What is Kubernetes? aka.ms/k8slearning