AWS IoT, AWS Greengrass For Time Series Analysis [English Ver.]
1 like178 views
This document discusses using AWS IoT, AWS Greengrass, and various AWS analytics services like QuickSight, IoT Analytics on SageMaker, and CloudWatch to perform time series analysis on sensor data measuring fine dust (PM2.5) levels. The architecture involves collecting dust sensor data using IoT devices and Greengrass, storing it in S3, and analyzing it using services like QuickSight, IoT Analytics on SageMaker, and Twitter and Line bots to predict dust levels and the best times for outdoor activities. The goal is to help people avoid high dust levels and find optimal exercise times for their health.
AWS IoT, AWS Greengrass For Time Series Analysis [English Ver.]
- 1. AWS IoT, AWS Greengrass For Time Series Analysis “Measuring & Predicting the fine dust(PM2.5) data.” @aniotmaker 정보통신설비 고급감리/Consulting/Firmware Engineer/Back-end Developer • Available Programming: Node.JS, JavaScript, SQL, Node-RED, Python, C&C++ • Seoul Weather Twitter Leader (https://twitter.com/aniotmaker) • AWS-IoT User Group Leader (https://www.facebook.com/groups/awsiot) • Node-RED User Group Leader (https://www.facebook.com/pg/nodereduser) Haesung Lee 2019. 5. 14.
- 2. Contents • Introduction • Stacks For Seoul Weather Twitter • Seoul Weather Architecture • IoT Sensors with JSON format • AWS IoT Core • AWS QuickSight • AWS IoT Analytics on SageMaker • Tableau Analytics • Spark Zeppline Analytics • Twitter-bot • Line-bot • AWS CloudWatch • PM2.5 dust sensor with Prediction • Twitter-bot for Seoul weather • Conclusion • References • Q & A @aniotmaker
- 4. AWS IoT Core AWS Greengrass Amazon QuickSight Open Weather Map APIs AWS SageMaker @aniotmakerStacks For Seoul Weather Twitter https://twitter.com/aniotmaker !4 Amazon EMR Amazon Athena Amazon CloudWatch Amazon EC2 Amazon S3 AWS Glue Amazon Athena Amazon Kinesis Firehose Amazon Redshift* Lambda function
- 5. Seoul Weather Architecture AWS IoT Core AWS Greengrass Door Sensor Amazon CloudWatch Amazon QuickSight AWS IoT Analytics Dust Sensor Open Weather Map APIs Amazon EC2 Time Series Analysis, Weather & Dust Information !5 + Train Model https://twitter.com/aniotmaker AWS SageMaker @aniotmaker Amazon S3 + Amazon Kinesis Firehose AWS Glue ETL (Data Lake) Sensors Unit IoT (ML, Lamda / MQTT, TLS) Unit Big Data Processing Unit BI Visualisations Unit Social Networks Unit Amazon Redshift* (DataWarehouse) Amazon EMR (Hadoop/ Spark) MQTT Lambda function
- 6. IoT Sensors with JSON format !6 A Dust Sensor A Door Sensor Open Weather API @aniotmaker
- 7. AWS IoT Core !7 AWS IoT Core AWS Greengrass Dust Sensor
- 9. AWS IoT Analytics on SageMaker !9 AWS IoT Analytics + Train Model AWS SageMakerAWS IoT Core AWS Greengrass Dust Sensor Amazon S3 +
- 10. Apache Zeppline Analytics AWS IoT Core AWS Greengrass Dust Sensor Amazon S3 !10
- 12. Twitter-bot • AWS-IoT + Twitter API • How To Predict the time of outside exercising? !12 @aniotmaker
- 13. Line-bot !13 • AWS-IoT + Line API @aniotmaker
- 15. PM2.5 dust sensor with Prediction !15 @aniotmaker
- 16. Twitter-bot for Seoul weather @aniotmaker !16
- 17. Conclusion @aniotmaker As you can see, my purpose is to implement time series analysis with AWS IoT. As much as possible to avoid fine dust and time to find the best outdoor exercise timing. Twitter would give you the best outdoor walking and exercise timing for your health Red-Coloured line Is prediction data !17
- 18. References • Designing MQTT Topics for AWS IoT Core (White Paper): https://d1.awsstatic.com/whitepapers/Designing_MQTT_Topics_for_AWS_IoT_Core.pdf • Using Node-RED Library To Wire Telemetry Data From IoT Devices To The Cloud: https://labs.eleks.com/2019/01/node-red-library-iot-cloud.html • Using AWS IoT Analytics to Prepare Data for QuickSight Time-Series Visualizations: https://aws.amazon.com/blogs/iot/using-aws-iot-analytics-to- prepare-data-for-quicksight-time-series-visualizations/ • AWS IoT Analytics 기반 시계열 데이터 QuickSight 시각화 방법: https://aws.amazon.com/ko/blogs/korea/using-aws-iot-analytics-to-prepare-data-for- quicksight-time-series-visualizations/ • AWS Docs: https://docs.aws.amazon.com/index.html • What Is AWS IoT Greengrass? https://docs.aws.amazon.com/greengrass/latest/developerguide/what-is-gg.html • 서버리스 실시간 데이터 처리 애플리케이션 구축: https://aws.amazon.com/getting-started/projects/build-serverless-real-time-data-processing-app- lambda-kinesis-s3-dynamodb-cognito-athena/?trk=gs_card • AWS 기반 지속 가능한 데이터 분석 플랫폼 구축하기 - 소성운, 지그재그 :: AWS Summit Seoul 2019 : https://www.slideshare.net/awskorea/aws-aws- summit-seoul-2019-141290115 • AWS 기반 지속 가능한 데이터 분석하기 GitHub: https://github.com/awskrug/datascience-group/tree/master/workshop-sustainable_data_analysis • Spark Streaming: IoT with Amazon Kinesis and Visualizing with Qubole Notebooks: https://www.qubole.com/blog/spark-streaming-iot-amazon- kinesis-visualizing-qubole-notebooks/ @aniotmaker !18
- 19. Q & A @aniotmaker 서울날씨 @aniotmaker @aniotmaker !19