Docker and the K computer
1 like746 views
The document discusses using Docker on the K computer supercomputer. Docker could help with compiling and running software by providing consistent software environments. At compile time, Docker containers could be used on login nodes to install cross-compilers and compile software in a consistent environment. At runtime, Docker could potentially be installed on K computer compute nodes, but the operating system kernel would need to be updated from 2.6.25 to 3.8 to support Docker, requiring patching 84 files. Issues include compiler licenses and administrative policies. A Docker-based IaaS approach is proposed to allow compiling software in containers on a server to overcome dependency and installation challenges.
Docker and the K computer
- 1. Docker and the K computer Bryzgalov Peter @ HPC Usability Research Team RIKEN AICS petr.bryzgalov@riken.jp 2015.08.19 Docker Meetup Tokyo #5
- 2. K computer
- 3. K computer Nodes > 80 000 Cores > 700 000 Linpack Performance 10,510 TFlop/s
- 4. Docker merits for HPC • Runtime — customise software stack, easy to test new software, more efficient resource management, administrators do not have to worry about security patches for user-space software, etc. • Compile time — eliminate dependencies on compilers and libraries versions, etc. 4
- 5. Docker merits for HPC • Runtime — customise software stack, easy to test new software, more efficient resource management, administrators do not have to worry about security patches for user-space software, etc. • Compile time — eliminate dependencies on compilers and libraries versions, etc. 5
- 6. Docker at Compile time • Install cross-compiler in container 6
- 7. Docker at Compile time • Install cross-compiler in container Why cross-compiler? 7
- 8. K architecture • Login nodes and Compute nodes • Compilation on Login nodes • Execution on Compute nodes 8
- 9. K architecture • Login nodes and Compute nodes • Compilation on Login nodes • Execution on Compute nodes • Login nodes — x86_64 • Compute nodes — sparc64fx 9
- 10. Docker at Compile time Done : • Installed cross-compilers in container. • Developed system for using it online. 10
- 11. Docker at Compile time Workflow : • User runs a script instead of “make” command on his computer. • Source code folder automatically mounted into container with compilers. • “make” command automatically executed in container. 11
- 12. Docker at Compile time 1.Container with cross-compilers runs on server. 12
- 13. Docker at Compile time 2.Source code folder automatically mounted into container. 13
- 14. Docker at Compile time 3.“make” command automatically executed in container. 14
- 15. Docker at Compile time Result : • Binaries are built and saved on user computer. 15
- 16. Docker at Compile time Issues : • Compiler license. • Customise software stack (instal third-party software in container). 16
- 17. Docker merits for HPC • Runtime — customise software stack, easy to test new software, more efficient resource management, administrators do not have to worry about security patches for user-space software, etc. • Compile time — eliminate dependencies on compilers and libraries versions, etc. 17
- 18. Docker at Runtime • Install Docker on Compute nodes 18
- 19. Docker at Runtime • Install Docker on Compute nodes • K compute nodes: 19
- 20. Docker at Runtime • Install Docker on Compute nodes • K compute nodes: 20
- 21. Docker at Runtime • Install Docker on Compute nodes • K compute nodes: 20 CPU sparc64fx OS kernel 2.6.25
- 22. Docker at Runtime • Install Docker on Compute nodes • K compute nodes: • Docker requires 3.8 (3.10 for latest version) 21 CPU sparc64fx OS kernel 2.6.25
- 23. Update OS kernel on Compute Nodes • 2.6.25 -> 3.8 22
- 24. Update OS kernel on Compute Nodes • 2.6.25 -> 3.8 • OS kernel on K CN: vanilla 2.6.25.8 + Fujitsu patch • Apply Fujitsu patch to vanilla 3.8 23
- 25. Patch vanilla kernel • Take a new vanilla kernel source code and apply Fujitsu patch 24
- 26. Patch vanilla kernel • Take a new vanilla kernel source code and apply Fujitsu patch • Patch fails starting from 2.6.26 25
- 27. Patch vanilla kernel • How many files need to be fixed? 26
- 29. Patch vanilla kernel 28 84f i l e s
- 30. Patch vanilla kernel 29 We need help!
- 31. Maeda Toshiyuki Team leader Bryzgalov Peter Researcher Thank you! HPC Usability Research Team www.hpcu.aics.riken.jp comp.photo777.org petr.bryzgalov@riken.jp This presentation has more slides but I don’t have time to show them.
- 32. Docker merits for HPC • Runtime — customise software stack, easy to test new software, more efficient resource management, administrators do not have to worry about security patches for user-space software, etc. • Compile time — eliminate dependencies on compilers and libraries versions. 31
- 33. Docker at Compile time K workflow • Edit Source code • Cross-compile on Login Nodes • Execute on Compute Nodes 32
- 34. Docker at Compile time K workflow • Edit Source code • Cross-compile on Login Nodes • Execute on Compute Nodes Dependencies • Compilers, libraries, … — one environment for all, it changes over time, application that compiles now may not compile in the future (like ABYSS). 33
- 35. Docker at Compile time K workflow • Edit Source code • Cross-compile on Login Nodes • Execute on Compute Nodes Dependencies • Compilers, libraries, … — one environment for all, it changes over time, application that compiles now may not compile in the future (like ABYSS). 33 Cross-compile in Docker container
- 36. Compile in container • Run container on User Computer • Run container on Login Node • Run container on Server 34
- 37. Compile in container • Run container on User Computer • Run container on Login Node • Run container on Server 34 Compiler licence limitations
- 38. Compile in container • Run container on User Computer • Run container on Login Node • Run container on Server 34 Compiler licence limitations “K” administration policies
- 39. Compile in container • Run container on User Computer • Run container on Login Node • Run container on Server 35
- 40. Run compiler on server in Docker container Requirements • Multiple environments (containers) • Multiple users • One machine • Limited resources — stop containers when they are not used Tasks • Copy source code into containers • Copy binaries from containers 36
- 41. Docker-based IaaS Overview • One host multi tenant IaaS • One user — one container • Users login with SSH • Mount local folders into containers with SSHFS 37 github.com/pyotr777/dockerIaaSTools
- 43. Docker-based IaaS Application Challenge • K-scope — tool for Fortran program static analysis. • Depends on Omni XMP compiler front-end, which produces AST (XML files). • Omni XMP depends on many packages and is hard to install. 39 omni-compiler.org/xcalablemp.html www.aics.riken.jp/ungi/soft/kscope github.com/pyotr777/dockerIaaSTools
- 44. Docker-based IaaS Application Our solution • Script that mounts local source code folder in container and executes “make” command inside the container. • K-scope executes the script instead of make command. 40 github.com/pyotr777/dockerIaaSTools
- 46. Docker-based IaaS Application Our solution • Script that mounts local source code folder in container and executes “make” command inside the container. • K-scope executes the script instead of make command. • After script finishes Intermediate code files are in local source folder — same effect as calling “make” command locally. 42 github.com/pyotr777/dockerIaaSTools