容器與資料科學應用
10 likes1,687 views
This document outlines a course on container technology for data science applications taught by Philipz. The course covers using containers with R, Python, Jupyter notebooks, TensorFlow with GPUs, Docker Compose, and more. It emphasizes how containers can integrate data, methods, and computing platforms to easily reproduce research. Example code is provided for using RStudio, Jupyter, and composing multiple services with Docker Compose. The instructor concludes by discussing best practices for using containers at different stages of work and how they can save more valuable time than money.
容器與資料科學應用
- 2. Philipz (鄭淳尹) Docker.Taipei 共同發起人 2014 COSCUP/iThome Summit 講者 2015 Microsoft Azure 開發者大會 講者 2016 COSCUP Docker 進階工作坊 2016 元智大學資工系 Docker 專題演講 2016 義守大學資工系 Docker 研習營 2017 逢甲大學資工系 Docker 研習班
- 7. 課程大綱 1. 容器與資料科學 2. Rocker - RStudio 3. Python & Jupyter 4. Nvidia-docker 工具 5. Docker & TensorFlow GPU 6. Docker Compose for Data Science 7. 結語
- 8. 1. 容器與資料科學
- 12. Singularity A Container for HPC Singularity HPC Container Technology Moves Out of the Lab
- 13. 2. Rocker - RStudio
- 14. R & RStudio https://github.com/rocker-org/rocker/wiki/Using-the-RStudio-image $ docker run -d -p 8787:8787 rocker/rstudio ● username: rstudio ● password: rstudio http://ropenscilabs.github.io/r-docker-tutorial/ http://tutorials.iq.harvard.edu/R/Rgraphics/Rgraphics.html
- 15. Commit to New Docker image install.packages("ggplot2") docker commit CONTAINER_ID docker run NEW_DOCKER_IMAGE
- 17. 3. Python & Jupyter
- 19. How to Use $ docker run -it --rm -p 8888:8888 jupyter/scipy-notebook
- 20. Volume to Save $ docker run -it --rm -p 8888:8888 -v YOUR_FOLDER:/home/jovyan/work jupyter/scipy-notebook Docker Volume vSphere, NFS, AWS EFS, Azure File Storage
- 22. Docker + TensorFlow + GPU ● Machine Learning, Deep Learning ● TensorFlow Docker images ● nvidia-docker, All-in-one DL image Deep Learning
- 23. Docker & NVIDIA GPUs
- 24. 5. Docker & TensorFlow GPU
- 25. Just ONE Line nvidia-docker run -it --rm -p 8888:8888 tensorflow/tensorflow:1.3.0-gpu Only Support Linux
- 26. 6. Docker Compose for Data Science
- 27. Compose File Sample (1/3) version: '2' services: db: image: mysql:5.7 volumes: - db_data:/var/lib/mysql restart: always environment: MYSQL_ROOT_PASSWORD: wordpress MYSQL_DATABASE: wordpress MYSQL_USER: wordpress MYSQL_PASSWORD: wordpress
- 28. Compose File Sample (2/3) wordpress: depends_on: - db image: wordpress:latest ports: - "80:80" restart: always environment: WORDPRESS_DB_HOST: db:3306 WORDPRESS_DB_PASSWORD: wordpress
- 29. Compose File Sample (3/3) ropensci: depends_on: - wordpress image: rocker/ropensci:latest ports: - "8787:8787" restart: always volumes: db_data:
- 30. RStudio with MySQL DB
- 31. 7. 結語
- 32. 容器思維 1. 各階段如何使用容器 2. 直接用容器，不再重頭安裝 3. 官方映像檔或依需要客製 4. 營運環境亦使用容器 5. 在易用性與尺寸取得平衡 6. 使用標籤(tag)區分版本 7. 擺脫程式語言限制，善用各語言優點
- 34. FaaS & Machine Learning https://github.com/jmkhael/faas-tensorflow