Kubernetes Security Workshop
2 likes240 views
This document summarizes a workshop on Kubernetes security presented by Avinash Desireddy and Anoop Kumar. The workshop covered Role-Based Access Control (RBAC) to grant users access to Kubernetes resources, using Open Policy Agent (OPA) and Gatekeeper to enforce cluster-wide policies, and network policies to control traffic between pods. It provided demonstrations of creating RBAC roles, restricting node port usage and enforcing resource limits with OPA policies, and allowing traffic between applications with network policies. The key takeaways were to enforce policies, build an RBAC strategy, start with a zero-trust approach, and use network policies.
![8
https://github.com/avinashdesireddy/k8s-security-workshop.git
ClusterRoleBinding
RoleBinding
Role Based Access Control
- List
- Get
- Watch
- Create
- Update
- Patch
- Delete
Verbs / Actions
Subjects
Resources
Role
ClusterRole
belongs to
binds
binds
connects
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: role-blue
namespace: blue
rules:
- apiGroups: ["", "apps"]
resources: ["deployments", "services"]
verbs: ["create", "delete", "list"]
apiVersion:
rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: blue-rb
namespace: blue-ns
subjects:
- apiGroup: rbac.authorization.k8s.io
kind: User
name: blue
roleRef:
kind: Role
name: role-blue
apiGroup: rbac.authorization.k8s.io](https://image.slidesharecdn.com/mirantis-protalk-kubernetes-security-workshop-220419214742/85/Kubernetes-Security-Workshop-8-320.jpg)
Kubernetes Security Workshop
- 1. Copyright © 2021 Mirantis, Inc. All rights reserved PRO TALK: Kubernetes Security Workshop Avinash Desireddy Sr. Solutions Architect
- 2. 2 Sr. Solutions Architect @ Mirantis /avinashdesireddy /avinashdesiredd /avinashdesireddy AVINASH DESIREDDY SPEAKER Director, Professional Services @ Mirantis /anokun7 /anooplive /anoopkumarv ANOOP KUMAR MODERATOR
- 3. 3 Kubernetes: Adoption, Security & Market Trends Source: The State of Containers and Kubernetes Security Report - Survey by StackRox(RedHat) in 2021 In the past 12 months, what security incidents or issues related to containers and/or Kubernetes have you experienced? Have you ever delayed or slowed down application deployment into production due to container or Kubernetes security concerns? What is your biggest concern about your company’s container strategy?
- 4. 4 Code Code Best Practices Vulnerability scanners Container Restrict Images, Privileged The 4 C’s of Cloud-Native Security Cluster Authentication, Authorization, Admission, Network Policy Cloud Datacenter, Network, Servers 2 4 3 1
- 5. 5 https://github.com/avinashdesireddy/k8s-security-workshop.git API ETCD Overview: Onboard Apps Securely Blue Red Green Worker Node Master Node SCHED C-M Worker Node Worker Node ● Isolate App teams Access rights? ● Protect clusters from restricted workloads? ● Protect communication layer in the cluster? Apps & App Teams Platform Engineer
- 6. 6 https://github.com/avinashdesireddy/k8s-security-workshop.git Worker Node Master Node API ETCD SCHED C-M Worker Node Worker Node Scenario #1 - Grant access to Users Blue Red Green Worker Node Master Node API ETCD SCHED C-M Worker Node Worker Node Role-Based Access Control A way of granting users access to Kubernetes API resources ❏ What API Resources a user should access? ❏ What Operations(Verbs) can be performed? ❏ Who can grant access? Apps & App Teams Platform Engineer
- 7. 7 https://github.com/avinashdesireddy/k8s-security-workshop.git ClusterRoleBinding RoleBinding Role Based Access Control - List - Get - Watch - Create - Update - Patch - Delete Verbs / Actions Subjects Resources Role ClusterRole belongs to binds binds connects
- 8. 8 https://github.com/avinashdesireddy/k8s-security-workshop.git ClusterRoleBinding RoleBinding Role Based Access Control - List - Get - Watch - Create - Update - Patch - Delete Verbs / Actions Subjects Resources Role ClusterRole belongs to binds binds connects apiVersion: rbac.authorization.k8s.io/v1 kind: Role metadata: name: role-blue namespace: blue rules: - apiGroups: ["", "apps"] resources: ["deployments", "services"] verbs: ["create", "delete", "list"] apiVersion: rbac.authorization.k8s.io/v1 kind: RoleBinding metadata: name: blue-rb namespace: blue-ns subjects: - apiGroup: rbac.authorization.k8s.io kind: User name: blue roleRef: kind: Role name: role-blue apiGroup: rbac.authorization.k8s.io
- 9. 9 https://github.com/avinashdesireddy/k8s-security-workshop.git Demo: RBAC ● Create Namespaces ● Grant Access to App Users to respective Namespaces ● Deploy 3 applications Environment Kubernetes IDE Access the cluster /avinashdesireddy/k8s-securi ty-workshop.git Mirantis Kubernetes Engine 1 Manager, 3 Worker Version - 1.21.3 Infrastructure Nodes, LB, DNS, etc
- 10. 10 https://github.com/avinashdesireddy/k8s-security-workshop.git Worker Node Master Node API ETCD SCHED C-M Worker Node Worker Node Happy Users!!! POD POD POD POD POD POD POD POD POD POD POD Green Blue Red Apps & App Teams Platform Engineer
- 11. 11 https://github.com/avinashdesireddy/k8s-security-workshop.git All of a sudden, Pods belong to App Blue started consuming a lot of memory in the cluster. How do we fix it? Resource Limits Worker Node Master Node API ETCD SCHED C-M Worker Node Worker Node Scenario #2 POD POD POD POD POD POD POD POD POD POD POD
- 12. 12 https://github.com/avinashdesireddy/k8s-security-workshop.git Scenario #2: Coordinating changes ● Identify Application Owner ● Ask Owner of App Blue to specify Memory & CPU Limits on Containers ● Configure Resource Quota & Limits on Namespaces
- 13. 13 https://github.com/avinashdesireddy/k8s-security-workshop.git Scenario #2: Challenges ● How can we enforce these across all the applications in the cluster? ○ Reach out to multiple application to make changes? ○ Define Best Practices? ○ Monthly Audits?
- 14. Do you find it a challenge while agreeing on Cluster Best Practices with App Teams? ⓘ Start presenting to display the poll results on this slide.
- 15. 15 https://github.com/avinashdesireddy/k8s-security-workshop.git Worker Node Master Node API ETCD SCHED C-M Worker Node Worker Node Scenario #2 - Policy Enforcement Blue Red Green POD POD POD POD POD POD POD POD POD POD POD Apps & App Teams Platform Engineer
- 16. 16 https://github.com/avinashdesireddy/k8s-security-workshop.git Worker Node Master Node API ETCD SCHED C-M Worker Node Worker Node Scenario #2 - Policy Enforcement Blue Red Green POD POD POD POD POD POD POD POD POD POD POD Policies ❏ Set up Resource Limits on containers ❏ Set Resource Quota & Limits on Namespaces ❏ Images from approved registries ❏ Limit NodePort Services ❏ Specific labels Apps & App Teams Platform Engineer
- 17. 17 https://github.com/avinashdesireddy/k8s-security-workshop.git Worker Node Master Node API ETCD SCHED C-M Worker Node Worker Node Scenario #2 - Policy Enforcement Blue Red Green POD POD POD POD POD POD POD POD POD POD POD Policies ❏ Set up Resource Limits on containers ❏ Set Resource Quota & Limits on Namespaces ❏ Images from approved registries ❏ Limit NodePort Services ❏ Specific labels Apps & App Teams Platform Engineer
- 18. 18 https://github.com/avinashdesireddy/k8s-security-workshop.git Open Policy Agent (OPA) ● CNCF Graduated ● General Purpose Policy Engine ● Empowers admins with more CONTROL over the system ● REGO Language ● Gatekeeper → Admission Controller implementation of OPA
- 19. 19 https://github.com/avinashdesireddy/k8s-security-workshop.git app.yaml Policy ConstraintTemplate Policy Constraint OPA in Kubernetes Admission Controller Gatekeeper/OPA defines kubectl apply validates Approve / deny
- 20. 20 Demo: OPA ● Restrict NodePort Usage ● Enforce Container Resource Limits Environment Kubernetes IDE Access the cluster /avinashdesireddy/k8s-securi ty-workshop.git Mirantis Kubernetes Engine 1 Manager, 3 Worker Version - 1.21.3 Open Policy Agent / Gatekeeper
- 21. 21 https://github.com/avinashdesireddy/k8s-security-workshop.git Worker Node Master Node API ETCD SCHED C-M Worker Node Worker Node Happy Users… Happy Cluster!!! Blue Red Green POD POD POD POD POD POD POD Apps & App Teams Platform Engineer
- 22. 22 https://github.com/avinashdesireddy/k8s-security-workshop.git Worker Node Master Node API ETCD SCHED C-M Worker Node Worker Node Scenario #3: Network Security POD POD POD POD POD POD POD New features are added to App Blue, the pods must connect to an external MySql DB and to an exposed API in Green App Pod How do we control Network Traffic to/from Pods? Network Policies
- 23. 23 Network Policy POD POD POD apiVersion : networking.k8s.io/v1 kind: NetworkPolicy metadata: name: test-network-policy namespace: default spec: podSelector : matchLabels : role: db policyTypes : - Ingress - Egress ingress: - from: - ipBlock: cidr: 172.17.0.0/16 except: - 172.17.1.0/24 - namespaceSelector : matchLabels : project: myproject - podSelector : matchLabels : role: frontend ports: - protocol: TCP port: 6379 egress: - to: - ipBlock: cidr: 10.0.0.0/24 ports: - protocol: TCP port: 5978 ● Control Traffic to/from pods ● Traffic between pods are non-Isolated ● Namespace scoped ● Can be defined based on - ○ Pod, Namespace or IP Range
- 24. Who are using Network Policies? ⓘ Start presenting to display the poll results on this slide.
- 25. 25 Default Deny Policy apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: name: default-deny-all spec: podSelector: {} policyTypes: - Ingress - Egress egress: - to: - namespaceSelector : matchLabels: kubernetes.io/metadata.name : kube-system podSelector: matchLabels: k8s-app: kube-dns ports: - protocol: UDP port: 53 - protocol: TCP port: 53 POD
- 26. 26 https://github.com/avinashdesireddy/k8s-security-workshop.git Scenario #3: App Blue connecting to MySQL 13.56.49.134 172.31.0.0/24 App Blue POD POD 192.168.96.4 192.168.96.3 App Green POD POD 192.168.206.7 192.168.206.9 1 Open firewall rule on DB to allow connections from 172.31.0.0/24 on port 3306 1 Create Egress Network Policy based on PodSelector 3 Open firewall rule on Cluster Nodes to allow traffic to 13.56.49.134 on port 3306 2 apiVersion : networking.k8s.io/v1 kind: NetworkPolicy metadata: name: mysql-port-egress spec: podSelector : matchLabels : app: blue backend: mysql policyTypes : - Egress egress: - to: - ipBlock: cidr: 13.56.49.134/32 ports: - protocol: TCP port: 3306
- 27. 27 https://github.com/avinashdesireddy/k8s-security-workshop.git Scenario #3: App Blue connecting to App Green 172.31.0.0/24 App Blue POD POD 192.168.96.4 192.168.96.3 App Green POD POD 192.168.206.7 192.168.206.9 Create Egress Network Policy based on Green PodSelector 1 Create Ingress Network Policy based on Blue PodSelector 2 apiVersion : networking.k8s.io/v1 kind: NetworkPolicy metadata: name: to-green-egress namespace: blue spec: podSelector : matchLabels : app: blue policyTypes : - Egress egress: - to: - podSelector : matchLabels : app: green ports: - protocol: TCP port: 8080 apiVersion : networking.k8s.io/v1 kind: NetworkPolicy metadata: name: from-blue-ingress namespace: green spec: podSelector : matchLabels : app: green policyTypes : - Ingress ingress: - from: - podSelector : matchLabels : app: blue
- 28. 28 Demo: Network Policies ● Create Default Network Policies ● Allow access for “Blue” App to MySQL on Port 3306 ● Allow access for “Blue” App to access “Green” Application’s API Environment Mirantis Kubernetes Engine Kubernetes IDE Access the cluster /avinashdesireddy/k8s-securi ty-workshop.git Mirantis Kubernetes Engine 1 Manager, 3 Worker Version - 1.21.3 Kubernetes Network Policies
- 29. 29 https://github.com/avinashdesireddy/k8s-security-workshop.git CNIs with Network Policy Support ● Weave ● Calico ● Cilium ● Kube-router ● Istio
- 30. 30 https://github.com/avinashdesireddy/k8s-security-workshop.git Apps & App Teams Platform Engineer Takeaways… Worker Node Master Node API ETCD SCHED C-M Worker Node Worker Node Blue Red Green Enforce Policies Build RBAC Strategy Start with Zero-Trust Network Policy