Webinar–Vulnerabilities in Containerised Production Environments
1 like111 views
This document discusses vulnerabilities in containerized production environments. It begins by outlining how security mindsets must change when adopting container technologies. It then discusses how data centers are targets for attacks, with the average cost of a data breach being over $3 million. The document outlines how certifications and regulations currently guide security processes, but these may not be fully aware of new technologies like containers. It stresses that container technologies are largely open source, so managing risk requires understanding components beyond just proprietary code. The document provides several best practices for securing containers throughout the development and deployment process. It emphasizes the importance of image health and outlines challenges around managing open source vulnerabilities in containers. Finally, it discusses implications of serverless architectures and provides key takeaw
![CONFIDENTIAL© 2019 Synopsys, Inc.14
Example: How image layer cache affects security
[root@localhost project]# docker history scanner_base:4.2.0
IMAGE CREATED CREATED BY SIZE
COMMENT
395dfd09d6d7 16 hours ago /bin/sh -c #(nop) EXPOSE 9036/tcp 0 B
6ccb88892a15 16 hours ago /bin/sh -c #(nop) ENTRYPOINT ["/scanner" 0 B
604fcdb13b57 16 hours ago /bin/sh -c #(nop) LABEL name=Base scanner 0 B
5bf60b767227 2 days ago /bin/sh -c #(nop) COPY file:0319ebe1148b5cefa 682 B
fe23aeab7fcc 2 days ago /bin/sh -c #(nop) COPY file:e822182ba43798ba0 11.36 kB
9cdc179735ad 2 days ago /bin/sh -c #(nop) COPY file:e822182ba43798ba0 11.36 kB
57bd5e62be14 2 days ago /bin/sh -c #(nop) COPY dir:d9dc3b531575096c83 241.6 MB
a1cb8fd37a68 2 days ago /bin/sh -c #(nop) COPY file:98c69c969ee05b51b 6.14 MB
13855a218a3e 7 days ago /bin/sh -c #(nop) ENV PATH=/scan.cli-4.2.0/b 0 B
885efab8f9b5 7 days ago /bin/sh -c #(nop) ENV APP_HOME=/scan.cli-4.2 0 B
1ed791e999b5 7 days ago /bin/sh -c #(nop) ARG BUILD 0 B
9dcb95a5ceb4 7 days ago /bin/sh -c #(nop) ARG BUILDTIME 0 B
8ada27a4da06 7 days ago /bin/sh -c #(nop) ARG LASTCOMMIT 0 B
7461b836791f 7 days ago /bin/sh -c #(nop) ARG bds_ver 0 B
4020be54fb0f 7 days ago /bin/sh -c yum -y update-minimal --security - 2.632 MB
208a012b6fe4 7 days ago /bin/sh -c #(nop) MAINTAINER Black Duck 0 B
196e0ce0c9fb 6 weeks ago /bin/sh -c #(nop) CMD ["/bin/bash"] 0 B
<missing> 6 weeks ago /bin/sh -c #(nop) LABEL name=CentOS Base Ima 0 B
<missing> 6 weeks ago /bin/sh -c #(nop) ADD file:1ed4d1a29d09a636dd 196.6 MB](https://image.slidesharecdn.com/emeawbcontainertrust-190530123013/85/Webinar-Vulnerabilities-in-Containerised-Production-Environments-12-320.jpg)
Webinar–Vulnerabilities in Containerised Production Environments
- 1. CONFIDENTIAL© 2019 Synopsys, Inc.1 Vulnerabilities in Containerised Production Environments Tim Mackey, Senior Technology Evangelist, Synopsys Changes in Security Mindset When Adopting Container Technologies
- 2. CONFIDENTIAL© 2019 Synopsys, Inc.2 Data centers are prime targets for attack Average cost of data breach: $3.62 million Lost business: $4.03 million Average time to identify and contain a breach: 257 days Source: 2017 Cost of Data Breach Study (US Data), Ponemon Institute Average cost of data breach: $3.86 million Lost business: $4.20 million Average time to identify and contain a breach: 266 days Source: 2018 Cost of Data Breach Study (US Data), Ponemon Institute 2017 2018
- 3. CONFIDENTIAL© 2019 Synopsys, Inc.3 Certifications and regulations guide current processes On-prem infrastructure policies • DISA STIG, OVAL Definitions, and XCCDF • Managed via Chef, Puppet, Ansible, raw SSH tooling • Private cloud adopts similar policies Public cloud infrastructure policies • Provider responsible for infrastructure security • Tenant remains responsible for VM security Some operational risk transferred to provider Focus for certifications and regulations • PCI, PII, and PHI • Process-centric and often not technology aware • Developers assume compliance and no feedback loop • NIST 800-137 focused on process • Continuous monitoring isn’t prescriptive NIST 800-137
- 4. CONFIDENTIAL© 2019 Synopsys, Inc.4 Equifax breach focused attention on open source
- 5. CONFIDENTIAL© 2019 Synopsys, Inc.5 Container technologies are largely open source Managing risk when you don’t own the stack
- 6. CONFIDENTIAL© 2019 Synopsys, Inc.6 Modern application = Proprietary code + Open source components + API usage + Application behavior and configuration
- 7. CONFIDENTIAL© 2019 Synopsys, Inc.8 Risk is a function of the full stack—not just the app “CNCF Interactive Landscape” application Cloud Native Computing Foundation provides a web-based application to browse for partner technology providers. Application details • 40K source lines of code • Node.js application framework • Containerized and deployed on Kubernetes • 0.05% of code in use is custom • 99.5% of code is in the stack • Risks present at all layers of the stack Source: CNCF Presentation: How good is our code?
- 8. CONFIDENTIAL© 2019 Synopsys, Inc.10 Cloud-native design rules 1. Containers are immutable Create once, run many instances 2. Containers are ephemeral A container’s life should be only as long as absolutely required 3. Containers can be sacrificed - Orchestration systems may terminate a container if needed - No guarantee of life span - Don’t store data or logs in containers 4. Containers limit resource access - Define groups for CPU/RAM access - Avoid use of ROOT credentials - Layered file system helps manage storage
- 9. CONFIDENTIAL© 2019 Synopsys, Inc.11 Designing a container is like developing a new car DevelopmentOperations • Engineers design using internal and external components • Production assembles components into a vehicle • Vehicle safety and assembly tests ensure compliance • Vehicle delivery occurs using trusted carriers to dealerships • Vehicle deployment occurs at time of purchase • Repair occurs using validated components • Regulators define governance and compliance criteria
- 10. CONFIDENTIAL© 2019 Synopsys, Inc.12 At each stage of development, question everything 1. Where does your base image actually come from? 2. What is the health of that base image? 3. You’re updating it at build time, but from what cache? 4. You trust your build servers, but who controls them? 5. Is there any way a foreign container can start in your environment? 6. Who has rights to modify container images? 7. What happens if the base image registry goes away? 8. What happens if the base image tag goes away? 9. When a security disclosure happens, what’s the process to determine impact? 10. How are images being updated and deployed in the face of new security disclosures? Container usage can trigger IP governance issues— i.e., `docker push` is potentially a distribution.
- 11. CONFIDENTIAL© 2019 Synopsys, Inc.13 Image health is critical to application security Docker Hub Container Scanning Red Hat Container Catalog Health Index
- 12. CONFIDENTIAL© 2019 Synopsys, Inc.14 Example: How image layer cache affects security [root@localhost project]# docker history scanner_base:4.2.0 IMAGE CREATED CREATED BY SIZE COMMENT 395dfd09d6d7 16 hours ago /bin/sh -c #(nop) EXPOSE 9036/tcp 0 B 6ccb88892a15 16 hours ago /bin/sh -c #(nop) ENTRYPOINT ["/scanner" 0 B 604fcdb13b57 16 hours ago /bin/sh -c #(nop) LABEL name=Base scanner 0 B 5bf60b767227 2 days ago /bin/sh -c #(nop) COPY file:0319ebe1148b5cefa 682 B fe23aeab7fcc 2 days ago /bin/sh -c #(nop) COPY file:e822182ba43798ba0 11.36 kB 9cdc179735ad 2 days ago /bin/sh -c #(nop) COPY file:e822182ba43798ba0 11.36 kB 57bd5e62be14 2 days ago /bin/sh -c #(nop) COPY dir:d9dc3b531575096c83 241.6 MB a1cb8fd37a68 2 days ago /bin/sh -c #(nop) COPY file:98c69c969ee05b51b 6.14 MB 13855a218a3e 7 days ago /bin/sh -c #(nop) ENV PATH=/scan.cli-4.2.0/b 0 B 885efab8f9b5 7 days ago /bin/sh -c #(nop) ENV APP_HOME=/scan.cli-4.2 0 B 1ed791e999b5 7 days ago /bin/sh -c #(nop) ARG BUILD 0 B 9dcb95a5ceb4 7 days ago /bin/sh -c #(nop) ARG BUILDTIME 0 B 8ada27a4da06 7 days ago /bin/sh -c #(nop) ARG LASTCOMMIT 0 B 7461b836791f 7 days ago /bin/sh -c #(nop) ARG bds_ver 0 B 4020be54fb0f 7 days ago /bin/sh -c yum -y update-minimal --security - 2.632 MB 208a012b6fe4 7 days ago /bin/sh -c #(nop) MAINTAINER Black Duck 0 B 196e0ce0c9fb 6 weeks ago /bin/sh -c #(nop) CMD ["/bin/bash"] 0 B <missing> 6 weeks ago /bin/sh -c #(nop) LABEL name=CentOS Base Ima 0 B <missing> 6 weeks ago /bin/sh -c #(nop) ADD file:1ed4d1a29d09a636dd 196.6 MB
- 13. CONFIDENTIAL© 2019 Synopsys, Inc.15 Example: System behavior with image consistency 1. Push and tag image version “1.0.1” 2. Pull and run “latest” 3. Scale to 2 replicas 4. Pull and run tag “1.0.1” 5. Scale to 2 replicas 6. Pull and run pull spec “sha256:12345” 7. Scale to 2 replicas 8. Delete tag “1.0.1” in registry 9. Scale “1.0.1” to 3 replicas 10. Push and tag image version “1.0.2” 11. Add node 2 12. Down node 1 Cluster Node 1 Cluster Node 2 “latest” “latest” 1.0.1 1.0.1 Sha256:12345 Sha256:12345 1.0.1 “latest” “latest” Sha256:12345 Sha256:12345
- 14. CONFIDENTIAL© 2019 Synopsys, Inc.16 Open source vulnerability management is a challenge 64 vulnerabilities per codebase—a 134% increase over last year Source: Synopsys Open Source Security and Risk Report 2018
- 15. CONFIDENTIAL© 2019 Synopsys, Inc.17 • Primary objective: Facilitate application deployment –Abstracts replication controllers and pods from users • DeploymentConfig –Defines the conditions for a new deployment – Configuration and image changes supported –Defines deployment strategy – Rolling, recreate, blue/green, A/B, custom – Requires “readiness” probes for pods –Rollback via DeploymentConfigRollback Planning your container patch strategy kind: "DeploymentConfig" apiVersion: "v1" metadata: name: "frontend" spec: template: metadata: labels: name: "frontend" spec: containers: - name: "helloworld" image: "openshift/helloworld" triggers: - type: "ConfigChange" - type: "ImageChange" imageChangeParams: automatic: true containerNames: - "helloworld" from: kind: "ImageStreamTag" name: “helloworld:latest" strategy: type: "Rolling" Patch version worse Patch version worse Patch version worse
- 16. CONFIDENTIAL© 2019 Synopsys, Inc.18 Container security paradigm shift Container security antipatterns • Allowing interactive login or SSH access • Requiring elevated privileges • Installing software or patching existing software • Attaching to a running container instance • Executing multiple processes • Writing any log data to local disk Container security patterns • Limit execution life span • Use exception-based network access with default blocked • Limit resource consumption • Limit image dependencies to required components • Restrict privileges on running containers • Allow for multiple replicas and concurrent container operations • Define a base image approval and update process • Use a secrets vault for all sensitive configuration information
- 17. CONFIDENTIAL© 2019 Synopsys, Inc.19 Example: Security information flow aids compliance Suggestion: usage of red and green for the outline confuses the message, i.e. start/stop
- 18. CONFIDENTIAL© 2019 Synopsys, Inc.20 Augmenting perimeter scan solutions Threat agents • Scan networks for weaknesses using toolkits • Success is a numbers game—zero knowledge of target • Perimeter defenses can be false positives • Use multifactors for attack reconnaissance Infiltration • Occurs through at least one vector • Creates beachheads supporting infection, C&C, and lateral movement • Exploits latent vulnerabilities and misconfigurations Mitigation powered by information flow • Can’t exploit what doesn’t exist • Focus attention on unpatched services • Open source originates from multiple channels; patches must match • Recognizes that attack landscape evolves Global IP space Managed systems Accessible systems Vulnerability present
- 19. CONFIDENTIAL© 2019 Synopsys, Inc.21 Example: The tale of CVE-2017-5638 and Equifax 1,649 days Code bug introduced August 2012 Struts 2.3 released November 2012 Struts 2.5 released May 2016 Patches available March 6 2017 7 days March 7 2017 Disclosure published NVD details March 14 2017 78 days Hacks successful May 13 2017 Hacks discovered July 29 2017
- 20. CONFIDENTIAL© 2019 Synopsys, Inc.22 Does the “serverless” model change things? Definition: A model under which an application is decomposed into stateless activities or functions run on cloud infrastructure Or: RPC.Next Process • Cloud provider defines a runtime • Runtime is implemented in a container • Developer creates an app-function targeting runtime • Developer defines an API endpoint for the app-function • Cloud provider supplies API gateway Security implications • Authorization and authentication required for protected data accesses • All data passes on “the wire,” which affects certifications • API contracts are critical, as are data reviews • Security response process must include security awareness from cloud provider Web App Authorizer Data StoreUser Service External APIContent Service 10100 001100 API Gateway
- 21. CONFIDENTIAL© 2019 Synopsys, Inc.23 Key takeaways Reassess processes when technology stacks change • Identify opportunities to reduce business risk with new technologies • Cloud-native paradigms are perfect to derisk in the face of adversity • Containers aren’t VMs and offer opportunities to improve risk posture Reduce risks of noncompliance • Implement continuous monitoring of all deployed apps, complete with dependency inventory • Leverage immutability properties of containers to simplify security • Validate running containers against orchestration to eliminate rogue activities Define clear technology risk acceptance criteria • Ensure criteria are understood in Ops, Development, and Procurement • Train all development and operations teams to identify changes in risk • Document decisions affecting risk acceptance at all points in the SDLC
- 22. Thank You