Containers 101 Meetup - VMs vs Containers
- 1. Containers 101 Meetup “Everything you always wanted to know about containers and VMs, but were too afraid to ask…” Presenter: Ben Corrie (@bensdoings)
- 2. Containers and VMs are Complementary • Virtualization of mixed workloads still just as compelling • VMs as flexible failure / isolation / tenant domains • Blurring the lines between the two – VMs that behave like containers?! 2
- 3. 3 Docker HubDocker Client LINUX HOST Container Control & TTY / REST Pull ImageDocker Daemon Docker Daemon Layered file system (AUFS) S D N Docker ImagesNET TRAFFIC TO CONTAINERS Linux Kernel 1. An executable process 2. Resource constraints / private namespace 3. Binary dependencies: Application, runtime, OS 4. A shared Linux kernel for running the executable 5. Ephemeral and persistent storage layers STATELESS PORTABLE FAST SECURE What is a Container?
- 4. 4 What is a Container Host? Docker Client LINUX HOST LXC Container Docker Daemon Container Container Layered file system (AUFS) S D N Docker ImagesNET TRAFFIC TO CONTAINERS Linux Kernel Control & TTY / REST Docker Hub Pull Image 1. Control plane & lifecycle management for containers 2. Resource scheduling and a container abstraction 3. Infrastructure abstractions: Storage, networking etc 4. A single Linux kernel manages everything 5. A static size and a resource reservation when virtual STATEFUL LONG-RUNNING SINGLE USER SINGLE USE
- 5. 5 What is a Hypervisor? vSphere Client ESX HOST VM Linux vCenter VM Windows VMFS / vSAN S D N VMDKs ESX Kernel SOAP / XML HTTP MyVMware Pull Image 1. Control plane & lifecycle management for VMs 2. Resource scheduling and a VM abstraction 3. Infrastructure abstractions: Storage, networking etc 4. A hypervisor kernel manages everything except apps 5. A static size, but no resource concerns unless nested STATEFUL LONG-RUNNING MULTI USER MULTI USE
- 6. 6 Characteristics • Speed – Start time vs throughput. Benefits transactional containers – Hello World vs Tomcat. Less benefit for long-running apps • Efficiency – Less memory consumption? Depends when virtualized – Network traffic NAT’d through guest vs straight to vNIC • Portability – Docker image abstraction and API is very portable – Subtle issues with kernel versions and patches • Isolation – Shared kernel significantly reduces runtime isolation • Granularity - Containers are great for granular services. Group in a VM as isolation domain.
- 7. 7 Containers and Security • Integrity – Reducing attack surface – includes vulnerability scanning – Isolation domains – runtime, network, storage – State management – lifecycle, propagation and garbage collection • Confidentiality – Privilege escalation – security patching, capabilities (cgroups), system permissions – Access control – authentication, secrets and credentials, docker cp and exec – Secure transport and encryption – Image provenance – image signing, lifecycle, propagation and GC • Availability – Data integrity – Resource management – importance of resource limits – Fault tolerance – Monitoring – health and performance
- 8. 8 Isolation Domains and Data Persistence • How do I isolate workloads from each other? – Runtime isolation – resource limits, kernel panic, ESX host failure, rack or region failure – Network isolation – traffic sniffing, firewalls, encryption, rate limiting – Storage isolation – data persistence, backup, networking, RBAC • Stateful vs Stateless / Cattle vs Pets – Different classifications of data. Where should it go? • Image state, container state, volume state. What’s the difference? – Should data lifespan be inherently tied to compute (VM / container)? – Without live migration, shared storage and HA, stateless looks attractive • The question is not “what plumbing do I need?” – What characteristics or business value do I need for my application? – Better question than, “do I need a container or a VM?”
- 9. 9 Types of Containers • Long-running – Can be stateless or stateful – Eg. Application servers, databases, load-balancers, KV stores etc – Typically a need for strong isolation • Transactional – Runs for a period and transforms some data – Eg. Runs a build. Processes a web request. Batch processing – Should only consume resource when running • Sidecar / micro-service – Augments the capabilities of a service or provides a helper function – Eg. Logging, monitoring, caching – Scales with the service. Potentially hindered by strong isolation
- 10. 10 Business Value as Plumbing • Jenkins Cluster case study • Characteristics of various workloads • Cluster size is a measure of compute capacity – reservation or limit? Jenkins Master Jenkins Slave Jenkins Slave Long-running Stateful Long-running Long-running Docker Images Docker Images Container Container Container Container Container Container Database Web Server
- 11. 11 The Resource Management Problem • P to V offered a shift away from underutilized hardware • V to C offers a shift away from underutilized VMs • Why do we still have this problem? – Long running workloads – Lack of self-provisioning – Stateful reconfiguration and cost of re-provisioning – Problem translated to container hosts • "Pets vs Cattle" != "VMs vs Containers". – Container hosts are the new pets J
- 12. 12 Demo 1: Workflow Efficiencies • Controlling vSphere infrastructure from a Docker client – Compute – Storage – Networking • Image management and portability – Modify existing image – Push to registry – Deploy to VIC • VMs as Cattle – Spin up VMs for specific tasks – Build VIC with VIC
- 13. 13 So What is VIC and How does it help? • VIC brings all of the container workflow efficiencies to vSphere infrastructure – Control vSphere infrastructure from a Docker client without having any vSphere credentials – Treat VMs as ephemerally as containers – No more OVAs, VMDKs, Templates, Cloning. Push / pull your state from secure registries • VIC allows you to translate business value into plumbing – Eg. I need to deploy Wordpress with MySQL. – Do I want strong isolation between these workloads? Container as a VM – Do I want strong isolation from other tenants? Container in a VM • VIC helps to draw clear lines between admins and users • Future integrations take this a step further. Eg. Kubernetes on vSphere
- 14. 14 Revisit: What is a Container Host? Docker Client LINUX HOST LXC Container Docker Daemon Container Container Layered file system (AUFS) S D N Docker ImagesNET TRAFFIC TO CONTAINERS Linux Kernel Control & TTY / REST Docker Hub Pull Image 1. Control plane & lifecycle management for containers 2. Resource scheduling and a container abstraction 3. Infrastructure abstractions: Storage, networking etc 4. A single Linux kernel manages everything 5. A static size and a resource reservation when virtual STATEFUL LONG-RUNNING SINGLE USER SINGLE USE
- 15. 15 Docker HubDocker Client ESX HOST / HYPERVISOR ContainerVM Control & TTY / REST Pull Image vCenter Docker Daemon VM Layered file system (VMFS) S D N Docker ImagesNET TRAFFIC TO CONTAINERS ESX Kernel STATELESS PORTABLE FAST SECURE What is a “ContainerVM”? Linux Kernel 1. An executable process 2. Resource constraints / private namespace 3. Binary dependencies: Application, runtime, OS 4. A private Linux kernel for running the executable 5. Ephemeral and persistent storage layers
- 16. 16 What is a Virtual Container Host? vSphere Cluster Layered file system (VMFS) S D N Docker Images Docker Hub Pull Image 1. Control plane & lifecycle management for ContainerVMs 2. Resource scheduling and a container abstraction 3. Infrastructure abstractions: Storage, networking etc 4. A Linux kernel per container, separate from control plane 5. Dynamic size and a resource limit, not reservation! VIRTUAL SINGLE USER MULTI USE MULTI HOST ESX Host ESX Host ESX Host VCH 1 VCH 2 Docker Daemon Docker Daemon vCenter Docker Client ContainerVM ContainerVM NET TRAFFIC TO CONTAINERS
- 17. 17 Demo 2: ContainerVMs as Cluster Nodes • Make container hosts as ephemeral as containers