Analyze Data in MongoDB with AWS
2 likes224 views
The document discusses using Amazon Athena and AWS Glue to analyze over 1TB of documents. It provides SQL code snippets to extract user data from a table and write the results to Parquet format using Glue ETL jobs. Tips are provided for running Glue ETL jobs, including specifying the data source, transforming the data, and writing the output.
Related topics:1 of 72
Download to read offline
![[리빙 포인트] Glue ETL Job을 돌릴땐](https://image.slidesharecdn.com/analyzemongodbwithaws-180809072904/85/Analyze-Data-in-MongoDB-with-AWS-62-320.jpg)
![[리빙 포인트] Glue ETL Job을 돌릴땐](https://image.slidesharecdn.com/analyzemongodbwithaws-180809072904/85/Analyze-Data-in-MongoDB-with-AWS-63-320.jpg)
More Related Content
Ad
Similar to Analyze Data in MongoDB with AWS (20)
Ad
Recently uploaded (20)
Analyze Data in MongoDB with AWS
- 12. 1TB > 1b+ documents wow much documents
- 24. Amazon Athena
- 36. ! "
- 38. AWS Glue
- 49. WITH latest_summaries AS (SELECT user_id AS uid, max(base_datetime) AS mdt FROM user_asset_summaries GROUP BY (user_id)) SELECT user_id FROM user_asset_summaries s JOIN latest_summaries ls ON s.user_id = ls.uid AND s.base_datetime = ls.mdt WHERE cardinality(s.cards) > 0
- 50. WITH latest_summaries AS (SELECT user_id AS uid, max(base_datetime) AS mdt FROM user_asset_summaries GROUP BY (user_id)) SELECT user_id FROM user_asset_summaries s JOIN latest_summaries ls ON s.user_id = ls.uid AND s.base_datetime = ls.mdt WHERE cardinality(s.cards) > 0
- 57. glueContext .getCatalogSource( database = "mart", tableName = "raw_user_asset_summaries", redshiftTmpDir = "", transformationContext = "ds" ) .getDynamicFrame()
- 62. [리빙 포인트] Glue ETL Job을 돌릴땐
- 63. [리빙 포인트] Glue ETL Job을 돌릴땐
- 67. !" #
- 68. !" #
- 69. !
- 70. ❤