Securing your Kubernetes cluster_ a step-by-step guide to success !

Securing your Kubernetes cluster:
Your step-by-step guide to success!
25/04/2024
Katia HIMEUR

Who am I?
• 🪪 Katia HIMEUR
• 💻 Computer scientist
• ☁ CTO & co-founder of Cockpit io, cloud & DevOps specialist
• 󰟲 SRE/Cloud/DevOps consultant for several years
❤ #Cloud #DevOps #Containers #Serverless #GitOps #IaC #CICD ❤
Securing your Kubernetes cluster: a step-by-step guide to success! 3 25/04/2024

Kubernetes, the leading container orchestrator
25/04/2024
Source : https://marketsnresearch.com/report/1649/global-kubernetes-market
5
Securing your Kubernetes cluster: a step-by-step guide to success!

The new cloud OS
25/04/2024
Source : https://www.dynatrace.com/news/blog/kubernetes-in-the-wild-2023/
6

25/04/2024
Attackers on the lookout
7

Safety not always a priority
25/04/2024
8

9
Why is it important to
secure your cluster?

K8s's popularity increases attacker interest
25/04/2024
10

Flexibility ⥤ Complexity ⥤ Errors
25/04/2024
Configuration errors are
a security risk
Complexity can lead to
configuration errors
Its flexibility leads to
complex configurations
Kubernetes is extremely
flexible
11

Extended attack surface
25/04/2024
https://kubernetes.io/docs/concepts/overview/components/
12

Scalability and resilience can be a vector for
vulnerability propagation
25/04/2024
13

Company background
Need :
● Compliance with speciﬁc standards and regulations
● Protect sensitive and critical applications and infrastructure
● Secure the sensitive data that may pass through
25/04/2024
14

15
Community and
ecosystem support

Community and ecosystem support
● Strong focus on safety
● Continuous improvement and disclosure of vulnerabilities
● Goals :
○ Share security best practices
○ Encouraging corrective action
○ Stay ahead of potential threats
25/04/2024
16

17
What will we see during
this talk?

What will we see during this talk?
● Understand the different types of attackers and attack vectors
● A non-exhaustive list of concrete actions you can take to secure your
cluster
● Focus on managed Kubernetes clusters
● The list of tools provided is for information only.
25/04/2024
18

Attacker types
External
attackers
Compromised
users
Internal
attackers
Compromised
containers
25/04/2024
20

Main attack vectors
25/04/2024
Insecure Server API
Misconﬁgured access
controls
Compromised
containers
Exposed dashboards
Incorrect network
conﬁguration
Compromised nodes
Compromised secrets
Supply chain
(dependencies...)
…
22

API server focus
● Critical cluster component
● External and internal communications gateway
● Manage
○ authentication and authorization
○ Data validation and storage in etcd
○ Orchestration and resource management
○ Scalability and performance
25/04/2024
24

25
How do you secure your
Kubernetes cluster?

Zero trust architecture
26 25/04/2024

● Approach to designing and implementing IT systems where trust is
totally eliminated
● Never trust, always verify
25/04/2024
27

Source : https://blog.cockpitio.com/cloud/approche-zero-trust-dans-le-cloud/
25/04/2024
28

Limiting API Server
exposure
25/04/2024
29

Limit API Server exposure?
● Restrict access
● Encrypt all ﬂows with API Server
25/04/2024
30
Private
clusters
Security
groups
Endpoint
access
control
Network ACLs

Authentication
25/04/2024
31

Authentication
● Use a strong authentication mechanism to control access to
Kubernetes
● Use an SSO portal to connect to your clusters (oAuth2, OpenID
Connect or LDAP).
● Enable multifactor authentication
25/04/2024
32

Authentication
25/04/2024
33

Using RBAC
34 25/04/2024

Using RBAC
● What is it?
○ RBAC: Role-Based Access Control
○ Resource access control based on user roles and service accounts
○ Deﬁnes who can do what
35 25/04/2024

Using RBAC
● Good practice
○ Apply the principle of least privilege to users and service accounts
36 25/04/2024

Example of an RBAC role
37 25/04/2024

Pod Security Admission
25/04/2024
38

Pod Security Standards Policies
39
Restricted
● Very restrictive policy
● Follows good curing
practices
Privileged
● No restrictions
● Climbing possibilities
Baseline
● Minimum restriction
policy
● Prevents the most
common climbs
25/04/2024

Pod Security Admission
● Kubernetes native admission controller to enforce Pod Security
standards policies
● Pod security restrictions are applied at the namespace level
● Enabled by default in the latest versions of Kubernetes
40 25/04/2024

Apply a safety policy
41 25/04/2024

Alternatives
42 25/04/2024

Using network policies
43 25/04/2024

Network status report
● Default : pods are not isolated, all trafﬁc ﬂows are allowed (ingress and
egress).
● Best practice: Restrict pod-to-pod communication to the strict
minimum
44 25/04/2024

What are network policies?
● Allows you to deﬁne how pods are authorized to communicate with :
○ Other pods
○ Other namespaces
○ IP blocks
45 25/04/2024

Example of a policy that prohibits all outbound ﬂow
46 25/04/2024

Alternatives
47 25/04/2024
Network
policies
Security Network
Observability
Performance

Securing your secrets
48 25/04/2024

Secrets... Not very secret 😱
● A Kubernetes secret is an object used to store sensitive data
(passwords, tokens, SSH keys, etc.).
● Default :
○ No encryption
○ Stored unencrypted
○ Base64 encoded value
49 25/04/2024

● Use RBAC to secure access
● Rotate secret
● Audit access to secrets
● Regularly review and update access policies
● Prevent secrets from ending up in logs
● Don't hard code secrets
50 25/04/2024
Gitleaks

Enable encryption
at REST
51 25/04/2024

Turn secrets into volume 👍
Exposing them as environment variables👎
52 25/04/2024
ServiceAccount secrets can
only be mounted on
speciﬁc resource types.

Use external secret management tools
53
AWS Secrets Manager
Azure Key Vault Google Cloud Secret
Manager
25/04/2024

● The Secrets Store CSI Driver enables integration of external secrets
managers with Kubernetes
● Integration is via a CSI (Container Storage Interface) volume.
● This driver allows you to mount several secrets, keys, certiﬁcates, etc.
stored in these external secret managers.
● Once the volume has been attached, the data is mounted in the
container's ﬁle system.
54 25/04/2024

Audit logging
55 25/04/2024

Audit logging
● Record every request made to the API server
● Analyze logs to detect suspicious and unusual activity
● Use security tools to analyze and react to logs in real time
● Define audit policies to fine-tune the configuration of events to be
recorded
25/04/2024
56

Example of an audit policy
25/04/2024
57

Update and patch management
25/04/2024
58

Update and patch management
● Apply update and security patches, automatically if possible, and
quickly :
○ Kubernetes clusters
○ all components deployed on clusters
○ nodes
● Use tools that scan all Kubernetes cluster components, including
images
● Respect best operating system practices
25/04/2024
59

Securing containers
25/04/2024
60

Securing containers
● Use only trusted images
● Reduce image size to reduce attack surface
● Use immutable containers to avoid runtime changes
25/04/2024
61

Securing containers
● Scanning your container images: before and after deployment
● Sign your container images
● Secure and protect your containers using tools that react to the
detection of abnormal events
25/04/2024
62

Securing containers
25/04/2024
63

Securing containers
25/04/2024
64

Admission controllers
25/04/2024
65

Admission controllers
● Controllers that intercept requests to the API server before objects are
persisted in the etcd database
● Two types of controller :
○ Validating admission controller : Validates or rejects requests
○ Mutating admission controller : Modiﬁes the requests it accepts
25/04/2024
66

Conﬁguration example
25/04/2024
67
This admission controller
prevents the API server
from being overloaded by
requests applying rate
limiting.

What is a service
mesh?
25/04/2024
69

● Dedicated infrastructure layer added to applications
● Designates both the tools and the network domain created
● Allow us to :
○ understand trafﬁc
○ make decisions based on trafﬁc type or origin
○ reduce the complexity of network management in a
microservice context
What is a service mesh?
25/04/2024
70

Service Mesh
functionalities
25/04/2024
71

Service mesh functionalities
25/04/2024
72
● Observability
● Trafﬁc management
● Security
● A/B testing
● Canary deployment
● Rate limiting
● Access control
● Encryption (including mTLS)
● End-to-end authentication
● Service discovery

25/04/2024
73
Some services mesh

Using services mesh
25/04/2024
74

75
Container-speciﬁc operating
systems for Kubernetes

Container-speciﬁc operating systems for
Kubernetes
25/04/2024
76
● Operating systems optimized for running containers and Kubernetes
● Minimize attack surfaces
● Increase cluster security
● Based on READ-ONLY systems

Container-speciﬁc operating systems for
Kubernetes
Talos Linux
Elemental
25/04/2024
77

Requests & limits
25/04/2024
79
● Best practices
○ Set memory requests lower than or equal to limits
○ Limit CPU on sensitive workloads

Requests & limits
25/04/2024
80
● Why?
○ Restrict resources used by pods on nodes
○ Avoid the effects of a denial-of-service attack

81
Linux kernel security
modules

Linux kernel security modules (1/3)
25/04/2024
82
● Deﬁning security proﬁles
● Mechanisms for controlling access and limiting the functionality of
processes running on the host system
● Damage limitation for compromised accounts

25/04/2024
83
● Increase insulation between containers
● Defense in depth
● Audit and ensure compliance of our environments

SELinux AppArmor
25/04/2024
84

Seccomp
25/04/2024
86
● Abbreviation for Secure computing mode
● Linux kernel security features
● Limits the number of system calls (Syscalls) containers can make
● Reduces attack surface
● Kubernetes automatically applies seccomp proﬁles loaded on a node
to pods and containers

Seccomp - Example
25/04/2024
87

Real-time protection
25/04/2024
89
● Detect abnormal behavior, security threats and compliance violations
● Be alerted in real time.
● Some tools rely on kernel events, enriched with container and
Kubernetes metadata, to succeed in their protection missions.

Real-time protection
25/04/2024
90

Follow best practices and
recommendations
25/04/2024
92

Follow best practices and recommendations
25/04/2024
93
● Follow recommendations and best practices
○ Security Checklist from the Kubernetes community:
https://kubernetes.io/docs/concepts/security/security-checklist/
○ CIS (Center for Internet Security (CIS) ) Kubernetes Benchmark
○ Recommendations from cloud providers
○ Keeping up to date

● Introduction to Vault (FR)
○ https://blog.cockpitio.com/devops/vault-overview/
● Managing secrets in Kubernetes with Vault Agents (FR)
○ https://blog.cockpitio.com/devops/vault-agent-kubernetes/
● Securing the API server in Kubernetes with Keycloak(FR)
○ https://blog.cockpitio.com/devops/kubernetes-secure-access-using-keycloak-oidc/
● Introduction to the zero trust approach (FR)
○ https://blog.cockpitio.com/cloud/approche-zero-trust-dans-le-cloud/
● Right-sizing your Kubernetes deployments (FR)
○ https://presentations.verchere.fr/Requests_Limits_OSXP_2023/
● Kubernetes security checklist
○ https://kubernetes.io/docs/concepts/security/security-checklist/
25/04/2024
95
Some resources

Conclusion
● Securing Kubernetes environments is fundamental to protecting your
company's interests, ensuring compliance and maintaining trust.
● As Kubernetes continues to evolve, so will security strategies, requiring
constant attention and adaptation.
● Today's list is not exhaustive. No list can be, as attackers show their creativity.
● Safety is a matter of continuous improvement and constant attention
25/04/2024
97

Thank you
@katia_tal
/in/katiahimeur
/
🔗 blog.cockpitio.com
🔗 www.cockpitio.com
@cockpitio42
/company/cockpit-io/
Keep in touch
17/04/2024
98

A team of enthusiasts,
guided by DevOps
culture,
to bring you the best of
the Cloud!

Securing your Kubernetes cluster_ a step-by-step guide to success !

More Related Content

Similar to Securing your Kubernetes cluster_ a step-by-step guide to success ! (20)

Recently uploaded (20)

Securing your Kubernetes cluster_ a step-by-step guide to success !