Singularity Containers for Scientific Compute
0 likes134 views
This document discusses Singularity containers for scientific computing. Singularity allows containers to be portable and reproducible across environments, solving issues like dependency conflicts. It can import containers from Docker images or build files. The Singularity Hub provides a registry and build system for Singularity containers, which can be pulled and run on HPC systems. Users can get started by trying a container from Docker Hub or building their own using Singularity build files.
Singularity Containers for Scientific Compute
- 1. SINGULARITYCONTAINERS FOR SCIENTIFIC COMPUTE Vanessa Sochat, PhD Research Software Engineer Research Computing Stanford School of Medicine
- 3. SINGULARITY: CONTAINERS FOR HPC
- 4. Scientists need containers too. SINGULARITY: CONTAINERS FOR HPC
- 5. SINGULARITY: CONTAINERS FOR HPC LIFE’S WORK
- 6. Why do we use them? SINGULARITY: CONTAINERS FOR HPC - Science is founded on reproducibility of work - Different environments require mobility of compute - Running a thing many times → scalability - Please, let’s not go to isolation / dependency hell
- 7. HELLO, SINGULARITY! SINGULARITY: CONTAINERS FOR HPC
- 8. The problems Singularity solves SINGULARITY: CONTAINERS FOR HPC Reproducibility and archive of software and environments A container is a file. It’s portable and sharable. Integration with traditional HPC The user inside the container is the user outside
- 9. Singularity, integrate! SINGULARITY: CONTAINERS FOR HPC You can import from a Docker image, a tarball, mirror url Add to an import using bootstrap Automatically do the above with Singularity Hub a la Github
- 10. SINGULARITY: CONTAINERS FOR HPC
- 11. SINGULARITY: CONTAINERS FOR HPC ./container.img
- 13. https://www.singularity-hub.org SINGULARITY HUB: CONTAINER COLLECTIONS github repo
- 14. https://www.singularity-hub.org SINGULARITY HUB: CONTAINER COLLECTIONS github repo Bootstrap: docker From: ubuntu:14.04
- 15. https://www.singularity-hub.org SINGULARITY HUB: CONTAINER COLLECTION push
- 17. https://www.singularity-hub.org SINGULARITY HUB: CONTAINER OPERATING SYSTEM
- 18. https://www.singularity-hub.org SINGULARITY HUB: CONTAINER USAGE COMMANDS
- 20. SINGULARITY HUB: BUILD INFRASTRUCTURE https://www.singularity-hub.org
- 21. SINGULARITY HUB: BUILD INFRASTRUCTURE https://www.singularity-hub.org module load singularity/jan2017master singularity pull shub://vsoch/analysis singularity shell shub://vsoch/analysis singularity run shub://vsoch/analysis singularity exec shub://vsoch/analysis
- 22. SINGULARITY HUB: THE BUILD SPECIFICATION Singularity sudo singularity create analysis.img sudo singularity bootstrap analysis.img Singularity Bootstrap: docker From: python:latest
- 23. SINGULARITY HUB: THE BUILD SPECIFICATION Singularity sudo singularity create analysis.img sudo singularity bootstrap analysis.img Singularity Bootstrap: docker From: python:latest %post echo “This is just run once after bootstrap!” echo “Install and add stuff here!” apt-get update apt-get install -y vim wget
- 24. SINGULARITY HUB: THE BUILD SPECIFICATION Singularity sudo singularity create analysis.img sudo singularity bootstrap analysis.img Singularity Bootstrap: docker From: python:latest %post echo “This is just run once after bootstrap!” echo “Install and add stuff here!” apt-get update apt-get install -y vim wget %runscript exec /usr/bin/python “$@”
- 25. How to get started? SINGULARITY: GETTING STARTED module load singularity/jan2017master singularity shell docker://python:latest 1. Try it out.
- 26. How to get started? SINGULARITY: GETTING STARTED module load singularity/jan2017master singularity shell docker://python:latest Bootstrap: docker From:python:latest 1. Try it out. 2. Make your own image a. Docker hub b. Singularity build file
- 27. How to get started? SINGULARITY: GETTING STARTED module load singularity/jan2017master singularity shell docker://python:latest Bootstrap: docker From:python:latest 1. Try it out. 2. Make your own image a. Docker hub b. Singularity build file sudo singularity create ubuntu.img sudo singularity bootstrap docker://python:latest
- 28. How to get started? SINGULARITY: GETTING STARTED module load singularity/jan2017master singularity shell docker://python:latest Bootstrap: docker From:python:latest 1. Try it out. 2. Make your own image a. Docker hub b. Singularity build file https://www.singularity-hub.org sudo singularity create ubuntu.img sudo singularity bootstrap docker://python:latest
- 29. HPC Admin, Developers, and Scientists SINGULARITY: CONTAINERS FOR HPC
- 30. SINGULARITY: CONTAINERS FOR HPC https://singularity.lbl.gov
- 31. SINGULARITY: CONTAINERS FOR HPC http://singularity.lbl.gov/singularity-tutorial
- 32. SINGULARITY: CONTAINERS FOR HPC vsochat@stanford.edu