Streamlining HPC Workloads with Containers
1 like6,660 views
The document discusses the advantages of using containers for high-performance computing (HPC) workloads, highlighting their efficiency over traditional virtual machines. Containers provide fast startup, high density, and minimal overhead while running directly on Linux, offering exclusive access to system resources. The document emphasizes the performance benefits and operational simplicity of containers in HPC environments.
Streamlining HPC Workloads with Containers
- 1. Streamlining HPC Workloads with Containers @DustinKirkland
- 4. Or perhaps in China...
- 5. Google image search shows...
- 8. Your DevOps engineer launches... x1.32xlarge
- 10. what do all of these have in common?
- 11. a lot, actually
- 13. directly on the bare metal itself
- 16. big problems are distributed across a cluster
- 17. everyone prefers a clean environment
- 18. virtual machines always involve overhead
- 19. VM MonitorVMXON VMXOFF Guest VM EntryVM Exit
- 20. oh, and let’s reboot a datacenter
- 22. BIOS is checking memory for problems… Scanning 1,199,511,627,776 bytes… This may take several minutes… Running test 1 of 8: 1.0% complete Overall test status: 0.1% complete Time elapsed: 17m23s Status: No problems have been found yet.
- 24. so let’s have a look at containers
- 30. LXD
- 33. ➢ Ultra fast “vm-lite” guests (bare metal speed) ➢ Any distribution of Linux - e.g. Ubuntu, CentOS ➢ Starts in less than 1 second ➢ 15x density of KVM or ESX for idle workloads host A nova-lxd lxc cli lxdkernel other restful apps lxc machine LXD REST API host B lxc machine lxdkernel host C host D host ... lxc machine lxc machine lxc machine lxdkernel lxdkernel lxdkernel
- 34. ➢ Ultra fast “vm-lite” guests (bare metal speed) ➢ Any distribution of Linux - e.g. Ubuntu, CentOS ➢ Starts in less than 1 second ➢ 15x density of KVM or ESX for idle workloads host A nova-lxd lxc cli lxdkernel other restful apps LXD REST API host B lxdkernel host C host D host ... lxdkernel lxdkernel lxdkernel lxc machine lxc machine lxc machine lxc machine lxc machine
- 35. CPU Cores CPU Cycles Memory Disk Space Disk IO Network IO One LXD container, with 100% of the system: “alloy” mode
- 36. exclusive access to system resources
- 37. but secured from the underlying hardware and OS
- 38. cgroups, user namespaces, apparmor, seccomp
- 39. instant startup
- 40. looks like a machine, Linux on Linux
- 41. zero latency
- 42. zero overhead
- 44. snapshot restore
- 45. live migration
- 46. demo