Amazon SageMaker を中心とした持続的な ML システム
1 like8,999 views
The document discusses Amazon SageMaker, an AWS service for building, training, and deploying machine learning models. It covers how SageMaker can be used for ML workflows including training models, creating endpoints, and integrating with other AWS services like Step Functions, Glue and Athena. Examples are provided of using SageMaker for various use cases like real-time inference and processing streaming data. Security and access control features of SageMaker are also summarized.
Amazon SageMaker を中心とした持続的な ML システム
- 1. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Makoto Shimura, Analytics Specialist Solution Architect Amazon Web Services Japan, K. K. Amazon SageMaker ML
- 2. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. • • • • Amazon Athena • AWS Glue • Amazon SageMaker
- 3. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Agenda • ML Ops • Amazon SageMaker • Amazon SageMaker ML • ML
- 4. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ML Ops
- 5. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ML
- 6. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ML
- 7. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ML 1. • • 2. ML • • ML 3. ML • •
- 8. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ML
- 9. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Kinesis S3 EMR Glue Redshift Athena QuickSight FargateECS SageMaker
- 10. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ML Kinesis S3 EMR Glue Redshift Athena QuickSight
- 11. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ML
- 12. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ML
- 13. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
- 14. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ML
- 15. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
- 16. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AB
- 17. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
- 18. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon SageMaker
- 19. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon SageMaker ML
- 20. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Jupyter Notebook • • • • VPC ENI VPC
- 21. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. CreateTrainingJob API Docker • • 2 • S3 •
- 22. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. CreateEndpoint API Docker • AB • • • S3
- 23. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon SageMaker ML
- 24. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon SageMaker ML
- 25. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Kinesis Firehose S3 SageMaker Glue Athena QuickSight App Servers
- 26. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. EC Kinesis Firehose S3 SageMaker Glue Athena QuickSight App Servers DynamoDB Lambda
- 27. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Camera Device Detection App GPU Greengrass Core Camera AWS IoT Glue Athena QuickSight S3 SageMaker Greengrass
- 28. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ML
- 29. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. StepFunctions ML Start CodeBuild StartBuild SageMaker CreateTrainingJob CodeBuild StartBuild SageMaker CreateModel SageMaker CreateEndpointConfig SageMaker CreateEndpoint UndateEndpoint wait wait wait wait SNS SNS Greengrass CreateDeployment
- 30. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. StepFunctions RDS S3 RedShift Glue S3 Glue RDS SageMaker
- 31. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. SageMaker SageMaker Glue Dev Endpoint Dev Endpoint boto3 Glue Glue SageMake r S3 Athena SageMaker Athena CTAS Athena S3 SageMaker
- 32. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ML Start SageMaker CreateTrainingJob CloudWatchLogs Lambda DB QuickSight SageMaker CreateTrainingJob SageMaker CreateTrainingJob ...... RDS
- 33. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. S3 RDS Start CreateTrainingJob CreateModel JSON ID Dockerfile CodeCommit S3 SageMaker ECR DVCGit LFS
- 34. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AB Elasticsearch Service Lambda SNS S3 Athena & QuickSight SageMaker AB App Server s Kinesis
- 35. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. / ML AWS S3 AWS ML SageMaker S3 AWS SageMaker S3CodeBuild SageMakerS3 Direct Connect Direct Connect
- 36. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ML
- 37. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ML ML • / • at rest / in transit encryption • AWS IAM AWS KMS Amazon VPC AWS Glue Amazon CloudWatch AWS CloudTrail
- 38. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. UI REST API ETL ETL REST API ML ML SQL SQL Notebook REST API
- 39. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Glue / Amazon SageMaker AWS Glue • Glue S3, RDS, DynamoDB, Redshift / • Athena / EMR Amazon SageMaker API • ML •
- 40. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. SageMaker : SageMaker Notebook Instance • IP • IAM • VPC VPC • VPC S3 S3 EFSRedshift
- 41. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. SageMaker : SageMaker • API Private Link • S3 KMS • S3 VPC S3
- 42. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. SageMaker : SageMaker • API Private Link • S3 KMS • S3 VPC S3 App Servers ELB
- 43. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
- 44. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ML DevOps Amazon SageMaker Amazon SageMaker AWS ML