Introductio to Docker and usage in HPC applications
Download as PPTX, PDF0 likes129 views
The document discusses Docker as an open-source platform for application containerization, emphasizing its advantages over traditional virtualization methods, such as reduced performance overhead and better resource management. It includes a literature survey, problem statement, proposed system architecture, and performance evaluation of Docker containers relative to virtual machines. The conclusion highlights Docker's impact on cloud computing and data-intensive applications, along with its potential for future developments in virtualization technology.
Introductio to Docker and usage in HPC applications
- 1. Richie Varghese Roll no : 9 S7 IT1
- 2. Table of contents Introduction Literature Survey Problem statement Proposed system System architecture Module description Application Merits Demerits Future scope Conclusion Reference 2
- 3. 3 Introduction • Docker is an open-source platform to build and distribute application with all its dependencies into a standard form called containers (they holds the whole kit required for an application) • It eases managing containers for cloud-based environment with tools like docker engine, docker registry, docker images, docker-server & client, docker container • VMs virtualize at hardware abstraction level, but containers emulates at OS level giving it a better performance results
- 4. Literature survey Details Description High Performance Linpack (HPL) "The linkpack benchmark: past, present, future" - J. J. Dongarra, P. Luszczek, A. Petitet Graph 500 "Introducing the graph 500" - R. C. Murphy, K. B. Wheeler, B. W. Barrett, J. A. Ang Docker "An introduction to Docker & analysis of its performance" - B. B. Rad, H. J. Bhatti, Mohammad Ahmadi 4
- 5. Problem Statement 5 Traditional approaches in Virtualization technology (Hardware virtualization) : • Performance overhead • Huge system resource usage • Limited number of instances for VM's • Low flexibility • Deploying HPC applications on VM's
- 6. Proposed System 6 • A container is a lightweight environment, docker provides simpler access to containers • Better system resource management • Sharing libraries and other core system components Virtualization at OS-level using Containers (docker containers) :
- 7. Shared Bins/LibsBins/Libs Docker Engineguest OS guest OS Bins/Libs Hypervisor 7 Infrastructure Host Operating System App 1 App 3App 2 Bins/Libs System Architecture Docker Registry Docker images Docker Server Docker Client dockerd docker hub guest OS Virtual Machine
- 8. 8 System Architecture (contd...) 1. Docker Server & client - The host machine that gets request from docker client and processes it accordingly. Client may be local or remote machine. server : dockerd | client : docker 2. Docker Images - Docker provides an automated way to create containers. This can be done either by "docker build" or by 'committing change' when exiting a container environment.
- 9. 9 System Architecture (contd...) 3. Docker Registries - Docker images are stored in docker registries from which an image can be pulled or to which image can be pushed. Registries may be private or public. 4. Docker daemon (services) - The Docker daemon listens for RESTful API requests and manages Docker objects such as images, containers, networks, and volumes. A daemon can also communicate with other daemons to manage Docker services.
- 10. Module Description 10 I/O devices & networking Main Memory System interconnect (bus) Memory translati on Execution hardware Operating System Libraries Application Programs API - Application Programming Interface ABI - Application Binary Interface ISA - Instruction Set Architecture Computer system architecture CPU : 16 cores / 32 logical cores RAM : 64 GB Network (Gigabit ethernet): 1Gbps OS (host & guest) : CentOS 7 Full virtualizer : QEMU 2.4.0.1 / KVM OS-level virtualizer : Docker 1.7.1
- 11. 11 No. of instances Virtual Machine (xCPU, xRAM, no. of process) Docker 2 16, 32, 16 16 4 8, 16, 8 8 8 4, 8, 4 4 16 2, 4, 2 2 32 1, 2, 1 1 Scenarios of configuration in performance evaluation 1) Test environment for deploying HPC applications in VM and docker containers Module Description (contd.)
- 12. ContainerContainerContainer 12 Infrastructure Host Operating System Docker Engine Shared Bins/Libs App 1 App 3App 2 Infrastructure Host Operating System Hypervisor VM OS Bins/Libs Apps VM OS Bins/Libs Apps VM OS Bins/Libs Apps 2) Proposed Model for deploying distributed applications Docker Virtual Machine Module Description (contd.)
- 13. 13 3) Docker container with computing intensive application : Module Description (contd.) Based on GFLOPs and % of RAM used Native performance standsout and is considered as the base of analysis (peak value of 308 GFLOPs) Docker performance is best at 75-80% Virtual Machines' performance reduces as RAM usage increases, its best when usage is <65%
- 14. 14 Module Description (contd.) Based on GFLOPs and the number of virtualized instances Docker dominates in performance in almost all scenarios Best performance is at 16 instances When running 32 instances (maximum possible containers), the performance drops due to controlling of computing workload by docker This causes more over-head in managing and distributing containers
- 15. 15 Module Description (contd.) 4) Docker container with data intensive application : Benchmark is performed in 2 phases : 1. Generate undirected graph 2. Traverse through using BFS There are 5 problem classes : 1. Toy (17 GB) 2. Mini (140 GB) 3. Small (1 TB) 4. Medium (17 TB) 5. Large (140 TB) 6. Huge (1.1 PB) Graph size is adjusted based on M = E * 2 SCALE M --> Number of edges E --> Edge factor 2 SCALE --> Number of vertices
- 16. 16 Module Description (contd.) Range of SCALE : 20 - 26 Edge factor : 16 (default) Docker can run large scale and problem size is limited for VM's Overhead of docker is low Hypervisor has to deploy more necessary bin/libs to guest OS With no hypervisor, docker can retrieve data faster Container starts instantly and share the docker image from layered filesystem
- 17. 17 Increasing number of docker containers donot contribute to further over-head 16-containers acheives the best performance Docker engine need only manage resources in terms of data traceability that needs to be used Module Description (contd.) When 32 containers are run, it has the drawback in controlling resources as they share the same kernel
- 18. Applications 18 • Docker containers are most suitable for virtualization of cloud platforms (most common in PaaS) • Developer productivity - Collaborate, develop and deploy applications. • Server consolidation • Accelerate app pipeline automation and deployement and ship applications much faster
- 19. Merits & Demerits Merits 1. Speed 2. Portability 3. Scalability 4. Rapid delivery 5. Density 6. Less impact on system performance 19 Demerits 1. Partial virtualization 2. No complete isolation 3. No support for 32-bit machines 4. Complete virtualized environment must be provided for non-linux platforms 5. Less control over resources
- 20. Future Scope • Docker has revolutionized LXC (Linux Containers) and may completely replace LXC for the betterment of virtualization technology • Better support for Network protocols and Xorg server for application & desktop virtualizations • More usage in cloud computing platforms like IaaS and SaaS 20
- 21. Conclusion 21 • Docker containers are becoming a widespread platform on cloud computing field, especially in data intensive applications • VM's and Docker provide greater portability, convinience and scalability • Docker helps reduce overheads, by sharing the Host OS kernel • Problem size, Application type and system limitations help decide whether to run a Docker-Container or a VM
- 22. Reference • "An introduction to Docker & analysis of its performance" - B. B. Rad, H. J. Bhatti, Mohammad Ahmadi • "Using Docker in High Performance Computing Applications" - M. T. Chung, N. Quang-Hung, Manh-Thin Nguyen, N. Thoai • https://docs.docker.com/get-started/ • https://docker.com/what-docker 22
- 23. 23 thank youdocker Find more at : https://docs.docker.com/