Amazon SageMaker 紹介 & ハンズオン（2018/07/03 実施）
6 likes9,542 views
The document outlines the Amazon SageMaker agenda and features, detailing its built-in algorithms, framework support, and integration with AWS services like Greengrass and Glue. It includes links to documentation and examples for using various machine learning models such as TensorFlow, MXNet, and Chainer within SageMaker. The document also touches on pricing and provides URLs for further resources and example notebooks.
Amazon SageMaker 紹介 & ハンズオン（2018/07/03 実施）
- 2. 7/3/2018 Amazon SageMaker 16:00-17:00 Amazon SageMaker - - Amazon SageMaker 3 - Amazon SageMaker - Build-in Algorithm - Native Framework (TF/MXNet/Chainer) - Bring your own Algorithm - Amazon SageMaker - Amazon SageMaker - Greengrass 17:30-19:00 Amazon SageMaker - Build-in Algorithm - Chainer 17:00- Q&A and Break 19:00-19:30 Q&A | Close
- 3. • • • ⎼ Amazon Athena ⎼ AWS Glue ⎼ Amazon SageMaker
- 11. SageMaker
- 12. SageMaker 5
- 14. Jupyter Notebook • • • 4 ml.t2, ml.m4, ml.p2, ml.p3 • VPC ENI VPC
- 15. CreateTrainingJob API Docker • • 2 • m4, m5, c4, c5, p2, p3 • S3 https://docs.aws.amazon.com/sagemaker/latest/dg/API_CreateTrainingJob.html
- 18. / : SageMaker Tensorflow / MXNet / Chainer github • SageMaker pull / • instance_type ’local’ https://github.com /aw s/sagem aker-python-sdk#local-m ode https://github.com /aw s/sagem aker-tensorflow -containers https://github.com /aw s/sagem aker-m xnet-containers https://github.com /aw s/sagem aker-chainer-container
- 19. CreateEndpoint API Docker • AB • • https://docs.aw s.am azon.com /sagem aker/latest/dg/API_CreateEndpointConfig.htm l https://docs.aw s.am azon.com /sagem aker/latest/dg/API_CreateEndpoint.htm l https://docs.aw s.am azon.com /ja_jp/sagem aker/latest/dg/API_UpdateEndpoint.htm l https://docs.aw s.am azon.com /ja_jp/sagem aker/latest/dg/API_UpdateEndpointW eightsAndCapacities.htm l
- 20. : AWS Greengrass SageMaker / . .- . / - ./ .- . • SageMaker S3 • DeepLens
- 21. SageMaker
- 22. SageMaker 2 AWS SDK • • SageMaker SDK • • AWS SDK SageMaker SDK AWS SDK scikit-learn • Python Spark • Jupyter Notebook https://github.com/aws/sagemaker-python-sdk https://github.com/aws/sagemaker-spark
- 23. SageMaker 2 AWS SDK • • SageMaker SDK • • AWS SDK SageMaker SDK AWS SDK scikit-learn • Python Spark • Jupyter Notebook https://github.com/aws/sagemaker-python-sdk https://github.com/aws/sagemaker-spark
- 24. create-endpoint create-notebook-instance create-training-job delete-endpoint delete-notebook-instance describe-endpoint describe-notebook-instance … estimator = TensorFlow(…) estimator.set_hyperparameters(…) estimator.fit(…) predictor = estimator.deploy(…) Predictor.predict(…)
- 26. SageMaker 1. SageMaker → 2. Tensorflow/MXNet/ Chainer → 3. →
- 27. SageMaker
- 28. • Linear Learner • Factorization Machines • XGBoost • Image Classification • seq2seq • K-means • PCA • LDA • Neural Topic Model • DeepAR Forecasting • BlazingText (word2vec) • Random Cut Forest New! New! https://docs.aws.amazon.com/sagemaker/latest/dg/algos.html New!
- 29. Image Classification ResNet • ResNet CNN • ILSVRC 2015 1 • ImageNet • use_pretrained_model 1 0 https://github.com/awslabs/amazon-sagemaker-examples/blob/master/introduction_to_amazon_algorithms/imageclassification_caltech/Image-classification-transfer-learning.ipynb https://www.cv-foundation.org/openaccess/content_cvpr_2016/papers/He_Deep_Residual_Learning_CVPR_2016_paper.pdf https://docs.aws.amazon.com/ja_jp/sagemaker/latest/dg/image-classification.html https://docs.aws.amazon.com/ja_jp/sagemaker/latest/dg/IC-Hyperparameter.html
- 30. SageMaker S3
- 34. Chainer / PyTorch / Tensorflow / MXNet
- 37. Tensorflow • model_fn • estimator_fn tensorflow.estimator • keras_model_fn tf.keras • train_input_fn • eval_input_fn • serving_input_fn • input_fn • output_fn https://docs.aws.amazon.com/sagemaker/latest/dg/tf-training-inference-code-template.html
- 38. MXNet • train • save • model_fn: • transform_fn: • https://docs.aws.amazon.com/sagemaker/latest/dg/mxnet-training-inference-code-template.html
- 49. SageMaker 3 1: • SageMaker 2: GPU • AWS
- 50. AWS SDK SageMaker SDK • SageMaker SDK Jupyter Notebook • AWS SDK
- 51. AWS SDK SageMaker SDK • SageMaker SDK Jupyter Notebook • AWS SDK SageMaker SDK AWS SDK
- 52. ML • SageMaker 1 ML • • 0.14 USD/GB/ • • 0.016 USD/GB https://aws.amazon.com/jp/sagemaker/pricing/
- 53. 3 SageMaker Example Notebooks • https://github.com/awslabs/amazon-sagemaker-examples SageMaker SDK • https://github.com/aws/sagemaker-python-sdk (Doc : https://readthedocs.org/projects/sagemaker/) SageMaker • https://docs.aws.amazon.com/ja_jp/sagemaker/latest/dg/whatis.html
- 55. Tensorflow • Tensorflow • https://github.com/awslabs/amazon-sagemaker-examples/blob/master/sagemaker-python- sdk/tensorflow_abalone_age_predictor_using_keras/tensorflow_abalone_age_predictor_using_keras.ipynb • Tensorflow Tensorboard • https://github.com/awslabs/amazon-sagemaker-examples/tree/master/sagemaker-python- sdk/tensorflow_resnet_cifar10_with_tensorboard • Tensorflow • https://github.com/awslabs/amazon-sagemaker-examples/blob/master/sagemaker-python- sdk/tensorflow_distributed_mnist/tensorflow_distributed_mnist.ipynb • Keras + Tensorflow • https://github.com/awslabs/amazon-sagemaker-examples/tree/master/sagemaker-python- sdk/tensorflow_abalone_age_predictor_using_keras • MXNet • https://github.com/awslabs/amazon-sagemaker-examples/tree/master/sagemaker-python-sdk
- 56. • scikit-learn SageMaker SageMaker • https://github.com /aw slabs/am azon-sagem aker- exam ples/blob/m aster/advanced_functionality/scikit_bring_your_ow n/scikit_bring_your_ow n.ipynb • SageMaker R • https://github.com /aw slabs/am azon-sagem aker- exam ples/blob/m aster/advanced_functionality/install_r_kernel/exam ple_r_notebook.ipynb • SageMaker • https://github.com /aw slabs/am azon-sagem aker- exam ples/blob/m aster/advanced_functionality/handling_km s_encrypted_data/handling_km s_encrypted_data.ipynb • XGBoost SageMaker • https://github.com /aw slabs/am azon-sagem aker- exam ples/blob/m aster/advanced_functionality/xgboost_bring_your_ow n_m odel/xgboost_bring_your_ow n_m odel.ipynb • Tensorflow Sagemaker • https://github.com /aw slabs/am azon-sagem aker- exam ples/blob/m aster/advanced_functionality/tensorflow _iris_byom /tensorflow _BYO M _iris.ipynb • Redshift MXNet R • https://github.com /aw slabs/am azon-sagem aker-exam ples/tree/m aster/advanced_functionality
- 57. Amazon Web Services Japan, K. K. Amazon SageMaker
- 58. SageMaker
- 60. IAM S3 S3
- 61. • VPC, •
- 62. Jupyter Notebook • InService Jupyter Notebook • New → Terminal
- 63. Terminal cd SageMaker/ wget http://bit.ly/sagemaker-notebooks -O sagemaker-notebooks.zip unzip sagemaker-notebooks.zip O( )
- 65. • MNIST • • Amazon SageMaker 1 XGBoost • Chainer MLP ( ) Amazon SageMaker