Container Days: Hijack a Kubernetes Cluster - a Walkthrough
0 likes25 views
The document outlines a talk about hijacking Kubernetes clusters and emphasizes awareness of common attack vectors and preventive measures. It covers best practices for securing application code, container images, deployment manifests, and enforcing Kubernetes policies and network restrictions. Additionally, it provides resources and tools for improving container runtime security and maintaining compliance within clusters.
Container Days: Hijack a Kubernetes Cluster - a Walkthrough
- 1. Hijack a Kubernetes Cluster – a Walkthrough ContainerDays 2022
- 2. Nico Meisenzahl • Head of DevOps Consulting & Operations at white duck • Microsoft MVP, GitLab Hero • Cloud Native, Kubernetes & Azure © white duck GmbH 2022 Email: nico.meisenzahl@whiteduck.de Twitter: @nmeisenzahl LinkedIn: https://www.linkedin.com/in/nicomeisenzahl Blog: https://meisenzahl.org
- 3. About this talk • this is not an in-depth security talk • it should make you aware of common attack vectors and how to prevent them • you will see demos on how to hijack a cluster • you will learn how to prevent those with common best practices • three more slide, then we will start hijacking • https://github.com/nmeisenzahl/hijack-kubernetes © white duck GmbH 2022
- 4. Why do we need to care about security? https://www.redhat.com/en/resources/state-kubernetes-security-report
- 5. What we will do © white duck GmbH 2022
- 6. Log4Shell https://www.splunk.com/en_us/surge/log4shell-log4j-response-overview.html
- 7. Security quick wins through the DevOps cycle © white duck GmbH 2022
- 8. Ensure secure application code • automate and enforce code checks • schedule dependency scanning • e.g. Dependabot • enforce Static Application Security Testing (SAST) in PRs • scans your code to identify potential security vulnerabilities • more details: https://owasp.org/www- community/Source_Code_Analysis_Tools © white duck GmbH 2022
- 9. Build secure container images • build secure/small container images – less is more • do only include required dependencies (no debugging tools!) • use self-contained binaries or “distroless” if possible • https://github.com/GoogleContainerTools/distroless • otherwise, use a small and secure Linux distro • use and enforce SAST for validating your Dockerfiles • scan your container images (on build and regularly) © white duck GmbH 2022
- 10. Build secure container images • build secure/small container images – less is more • do only include required dependencies (no debugging tools!) • use self-contained binaries or “distroless” if possible • https://github.com/GoogleContainerTools/distroless • otherwise, use a small and secure Linux distro • use and enforce SAST for validating your Dockerfiles • scan your container images (on build and regularly) © white duck GmbH 2022 Would have made it much harder to hijack the container and further expend Would have shown the possibility of code injection
- 11. Ensure secure deployment code • as important as secure application code and Dockerfiles • validate your deployment manifests using SAST • and enforce them via PRs • can help you to implement best practices like denying • containers running as root • mounting hostPath • … © white duck GmbH 2022
- 12. Ensure secure deployment code • as important as secure application code and Dockerfiles • validate your deployment manifests using SAST • and enforce them via PRs • can help you to implement best practices like denying • containers running as root • mounting hostPath • … © white duck GmbH 2022 Would have made it much harder to hijack the node
- 13. SAST Tooling • Source code • https://codeql.github.com • https://security-code-scan.github.io • https://securego.io • Kubernetes manifests • https://kubesec.io • https://github.com/aquasecurity/trivy • https://github.com/bridgecrewio/checkov • Dockerfiles • https://github.com/aquasecurity/trivy • https://github.com/bridgecrewio/checkov • Terraform • https://github.com/tfsec/tfsec • https://github.com/aquasecurity/trivy • https://github.com/bridgecrewio/checkov © white duck GmbH 2022
- 14. Kubernetes policies • enforce compliance and governance within clusters • verifying manifests is not enough! • examples include enforcement of • read-only filesystems • denying hostPath mounts • denying containers running as root • … © white duck GmbH 2022
- 15. Kubernetes policies • enforce compliance and governance within clusters • verifying manifests is not enough! • examples include enforcement of • read-only filesystems • denying hostPath mounts • denying containers running as root • … © white duck GmbH 2022 Would have made it much harder to further hijack the nodes and cloud resources
- 16. Kubernetes policy tooling • Pod Security Admission • stable since 1.25 • https://kubernetes.io/docs/concepts/security/pod-security- admission • Open Policy Agent Gatekeeper • https://github.com/open-policy-agent/gatekeeper • Kyverno • https://kyverno.io © white duck GmbH 2022
- 17. Network Policies • granular deny or explicitly allow between containers and ingress/egress of the cluster • limit egress access to the internet • limit access between applications/namespaces • deny access to the Cloud provider metadata service • https://kubernetes.io/docs/concepts/services- networking/network-policies © white duck GmbH 2022
- 18. Network Policies • granular deny or explicitly allow between containers and ingress/egress of the cluster • limit egress access to the internet • limit access between applications/namespaces • deny access to the Cloud provider metadata service • https://kubernetes.io/docs/concepts/services- networking/network-policies © white duck GmbH 2022 Would have denied network connections (reverse shell, Redis, Internet, metadata service)
- 19. Container Runtime Security • helps to detect malicious threads and workloads • untrusted process within container • a shell is running inside a container • container process mounting a sensitive path • a process making outbound network connections • container runtime security tools like Falco of Tetragon can help © white duck GmbH 2022
- 20. Container Runtime Security • helps to detect malicious threads and workloads • untrusted process within container • a shell is running inside a container • container process mounting a sensitive path • a process making outbound network connections • container runtime security tools like Falco of Tetragon can help © white duck GmbH 2022
- 21. Container Runtime Security • helps to detect malicious threads and workloads • untrusted process within container • a shell is running inside a container • container process mounting a sensitive path • a process making outbound network connections • container runtime security tools like Falco of Tetragon can help © white duck GmbH 2022 Would have detect all our “work” within the containers
- 22. Further best practises • do not • share service accounts between applications • enable higher access levels for the default service account if not required • mount service account token if not required • https://kubernetes.io/docs/tasks/configure-pod-container/configure-service- account/#use-the-default-service-account-to-access-the-api-server • changed with 1.24 • review all third-party snippets before applying them • implement a Web Application Firewall (WAF) to further secure your application © white duck GmbH 2022
- 23. Further best practises • do not • share service accounts between applications • enable higher access levels for the default service account if not required • mount service account token if not required • https://kubernetes.io/docs/tasks/configure-pod-container/configure-service- account/#use-the-default-service-account-to-access-the-api-server • changed with 1.24 • review all third-party snippets before applying them • implement a Web Application Firewall (WAF) to further secure your application © white duck GmbH 2022 Wouldn’t have allowed us to talk to the API server Would have denied our code injection
- 24. ContainerDays 2022 Post-Conference Meetup • https://www.meetup.com/hamburg-kubernetes-meetup/events/287713443 © white duck GmbH 2022
- 25. Questions? • Slides: https://www.slideshare.net/nmeisenzahl • Demo: https://github.com/nmeisenzahl/hijack-kubernetes © white duck GmbH 2022 Email: nico.meisenzahl@whiteduck.de Twitter: @nmeisenzahl LinkedIn: https://www.linkedin.com/in/nicomeisenzahl Blog: https://meisenzahl.org