Containerised applications workload security

What is container security?

Containerised application workloads are often more complex than traditional application workloads. Therefore, deploying such workloads in a production environment may involve launching a large number of containers. As a result, system administrators and security specialists need more time and effort, along with more complex procedures, to secure containerised workloads compared to traditional workloads. 

Container security is the process of implementing and maintaining security controls that protect containers and their underlying infrastructure during the application development lifecycle. Those processes need to be integrated into the development pipeline to ensure that all application components are secure, from the initial development phase until the end-of-development lifecycle.

Container security challenges

While the process of containerising applications has benefits, it can introduce security challenges that could sometimes be challenging to overcome. Some of these challenges include:

1. Larger attack surface of containerised applications

Because a containerised application may need to launch a large number of containers derived from the same base images, for example a web server image, and if that particular base image has known vulnerabilities, then all containers based on that image will inherit the vulnerability. In that case, the attack surface is multiplied by the number of containers. Similarly, when containers are based on different base images, for example two web server images derived from different operating system releases, then the attack surface will be proportional to the total number of vulnerabilities in all base images combined.

2. Shared kernel architecture

Since the container host’s kernel is shared by all containers running on it by design, additional configuration is required to limit container permissions and ensure proper isolation of containers. For example, containers by default have access to the host’s network, where the internet or other private networks might be reachable. In most cases, containers of a microservices application are only supposed to interact with each other within an isolated network. Therefore, configuring container network isolation is necessary, and this can be achieved by creating isolated Docker networks when launching containers with Docker Compose, or creating a network policy in Kubernetes. Another example is the default kernel capabilities assigned by Docker when launching containers, such as CAP_NET_RAW, where this capability is not needed for most application containers to function properly, and therefore can be removed.

3. Visibility in a dynamic environment

Containerised workloads are highly dynamic and their orchestration platforms abstract the underlying infrastructure, including the network. As a result, traditional monitoring tools may not be able to see which containers are running, what they are running, or analyse their network behaviour. For example, containers communicate with each other over network overlays managed by the container orchestration platform. Therefore, traditional network monitoring tools that analyse network flows going through the container’s host network interfaces, or through network infrastructure devices, will not be able to monitor the required network metrics.

Container security best practices

1. Securing images

• Container image security is enhanced by minimising included components, with every extraneous tool or library introducing potential vulnerabilities. Building the application as a statically compiled binary, embedding all dependencies, effectively addresses this.
• Use trustworthy images. If the application container is not built from scratch and is based on an image, then the base image needs to be chosen from a trustworthy source. Public image repositories, such as Docker Hub, can be used by anyone to host images that might include malware.

2. Securing container registries

• Use access control with private registries to define who can access and publish images. This can be easily achieved with Identity and Access Management (IAM) and Role-based Access Control (RBAC) when using container registry services available from public cloud providers. For example, developers should only be able to push and pull images to specific repositories within the registry, but shouldn’t be able to modify the registry configuration.
• Sign images to help end users verify the origin of images, ensuring the authenticity and integrity of the code within the image. This can be accomplished with tools like Cosign, Docker Content Trust (DCT), and Notary Project.
• Scan images for vulnerabilities using the vulnerability scanning facility built into some container registries, or an external vulnerability scanning tool like Trivy.
• Prune registries of unsafe and vulnerable images that should no longer be used. This can be automated with time triggers or labels associated with images.

3. Securing deployment

• Secure the target environment by hardening the container host operating system or using a purpose-built container-optimised operating system, such as Flatcar or Fedora CoreOS.
• Use orchestration platforms, as they provide features like secrets management, RBAC, resource isolation (with namespaces and network policies), audit logging, and monitoring.
• Use immutable deployments. They completely eliminate vulnerabilities in older versions of dependencies, unlike the typical process of patching the application in place, which might leave older vulnerable dependencies in the container.
• Refer to specific versions of images during deployment using version tags, such as “app:1.2.3”. This ensures that specific and known-to-be-good images are being deployed.
• Separate deployment of workloads with different sensitivity levels, as orchestration platforms may place them by default on the same host. This can be achieved by utilising the orchestration platform’s scheduling features, such as node selectors and affinity rules, or simply by having separate clusters for each sensitivity level.

4. Securing container runtime

• Create separate virtual networks for each application to isolate its containers and restrict connectivity to only be within the virtual network. This can be easily achieved in a container orchestration platform through the use of the network policy feature provided by some Container Network Interface (CNI) plugins, such as Calico.

• Only expose ports that serve the application. This rule also applies to the container host itself.
• Use Transport Layer Security (TLS) to secure communications between services. This encrypts traffic and only allows authorised endpoints to connect. This can be easily achieved in a container orchestration platform through the use of a service mesh, such as Istio .
• Run containers with their root filesystem in “read-only” mode and use a temporary file system (tmpfs) to restrict non-persistent writes to specific directories.

5. Monitoring container activity

As containerised workloads are highly dynamic, where many containers may be launched from the same or different images and new versions of those images may get introduced at a high pace, it is critical to identify security or other operational issues quickly and remediate them at the source. Therefore, containerised workloads require a granular level of monitoring to provide visibility for IT and security teams into elements running in the environment. Monitoring tools can help with identifying anomalous behaviour and responding to events in a timely manner. For example, the timely identification of a faulty or vulnerable image ensures the ability to fix the image and rebuild all containers based on it.

Common container security mistakes to avoid

1. Failure to implement basic security practices

Even though containers are relatively new technology that require the implementation of new security practices and tools, certain basic security principles and practices still apply. These include keeping operating systems and container runtimes updated and patched.

2. Failure to implement logging, monitoring, and testing

This is especially the case for highly distributed systems spanning multiple or hybrid clouds, as this would result in loss of visibility into the health of applications and environments, leading to critical security risks.

3. Failure to secure all stages of CI/CD pipelines

If security testing is performed only at the end of the development lifecycle, that would result in unnecessary delays with the deployment of application releases to production. Those delays can be eliminated by implementing security early in the development lifecycle, and by using appropriate security tools at every stage within the pipeline.

By avoiding these common mistakes and following container security best practices, the risk of security breaches can be significantly decreased for a containerised application and its infrastructure.