SlideShare a Scribd company logo
AWS




      @ryu_kobayashi
•
•
     •                       ...

•                              ...

•   Twitter @ryu_kobayashi
2010

  •
  •
  •
AWSを使ったトラッキングログ収集
•
•
AWSを使ったトラッキングログ収集
AWSを使ったトラッキングログ収集
AWSを使ったトラッキングログ収集
Elastic Load
Balancing
Elastic Load   Amazon Elastic
Balancing      Compute Cloud
Elastic Load   Amazon Elastic
Balancing      Compute Cloud




Auto Scaling
Elastic Load   Amazon Elastic
Balancing      Compute Cloud




Auto Scaling
               {
Elastic Load   Amazon Elastic
Balancing      Compute Cloud




Auto Scaling
               {
Elastic Load   Amazon Elastic
Balancing      Compute Cloud




Auto Scaling
               {
}
Amazon Elastic
Compute Cloud
}
Amazon Elastic
Compute Cloud
}
Amazon Elastic       Amazon
Compute Cloud        SimpleDB
SimpleDB
Domain size                                      10 GB per domain
Domain size                                      1 billion attributes per domain
Domain name                                      3-255 characters    a-z, A-Z, 0-9, ‘_’, ‘-’, and ‘.’
Domain per account                               100
Attribute name-value paris per item              256
Attribute name length                            1024 bytes
Attribute values length                          1024 bytes
Item name Length                                 1024 bytes
Attribute name, attribute value, and             All UTF-8 characters that are valid in XML documents.
item allow characters
Attribute per PutAttributes operation            256
Attribute requested Per Select operation         256
Item per BatchPutAttributes operation            256
Maximum items in Select response                 2500
Maximum query execution time                     5 seconds
Maximum number of unique attributes per select   20
exception
Maximum number of comparisons per                20
Select exception
Maximum response size for Select                 1MB
SimpleDB
Domain size                                      10 GB per domain
Domain size                                      1 billion attributes per domain
Domain name                                      3-255 characters    a-z, A-Z, 0-9, ‘_’, ‘-’, and ‘.’
Domain per account                               100
Attribute name-value paris per item              256
Attribute name length                            1024 bytes
Attribute values length                          1024 bytes
Item name Length                                 1024 bytes
Attribute name, attribute value, and             All UTF-8 characters that are valid in XML documents.
item allow characters
Attribute per PutAttributes operation            256
Attribute requested Per Select operation         256
Item per BatchPutAttributes operation            256
Maximum items in Select response                 2500
Maximum query execution time                     5 seconds
Maximum number of unique attributes per select   20
exception
Maximum number of comparisons per                20
Select exception
Maximum response size for Select                 1MB
AWSを使ったトラッキングログ収集
•   PV/
•   PV/

•
•   PV/

•
•         100
•   PV/

•
•               100


•   1                 1
          ...
•   PV/

•
•                100


•   1                  1
           ...
•                8
          10
•   PV/

•
•                100


•   1                         1
           ...
•                8
          10
•   Item               Attribute
}
Amazon Elastic
Compute Cloud
}
Amazon Elastic
Compute Cloud
}
Amazon Elastic       Amazon
Compute Cloud        SimpleDB
}
Amazon Elastic            Amazon
Compute Cloud             SimpleDB




                     DC
}
Amazon Elastic            Amazon
Compute Cloud             SimpleDB




                     DC
}
Amazon Elastic            Amazon
Compute Cloud             SimpleDB




                     DC
AWSを使ったトラッキングログ収集
Amazon
SimpleDB
Amazon
SimpleDB
Amazon
SimpleDB

           DC
Amazon
SimpleDB

           DC
Amazon
SimpleDB

           DC
DC
DC
DC
DC
DC
AWSを使ったトラッキングログ収集
•   Cassandra   MapReduce
•     Cassandra   MapReduce

•
    HDFS
•     Cassandra   MapReduce

•
    HDFS
•             Map
•     Cassandra        MapReduce

•
    HDFS
•                    Map



•          .......
•     Cassandra        MapReduce

•
    HDFS
•                    Map



•          .......
...
AWSを使ったトラッキングログ収集

More Related Content

PDF
AWS EBS
PPTX
Aws cost strategies
PDF
Introduction to EC2 (AWS)
PDF
Serverless Microservices w/ AWS Lambda and node.js
PPTX
High Performance MongoDB on Storage-Optimized AWS EC2
PDF
CloudFork
PPTX
EC2 Performance, Spot Instance ROI and EMR Scalability
PDF
Aws Elastic Block Storage
AWS EBS
Aws cost strategies
Introduction to EC2 (AWS)
Serverless Microservices w/ AWS Lambda and node.js
High Performance MongoDB on Storage-Optimized AWS EC2
CloudFork
EC2 Performance, Spot Instance ROI and EMR Scalability
Aws Elastic Block Storage

Similar to AWSを使ったトラッキングログ収集 (20)

KEY
SimpleDBを使った ソーシャルアプリ構築事例
PPTX
Amazon Web Services OverView
PDF
Jeff Barr Amazon Services Cloud Computing
PDF
Masterworks talk on Big Data and the implications of petascale science
PDF
Seize The Cloud
PDF
Scaling Out With Hadoop And HBase
PPT
Data Science Day New York: The Platform for Big Data
PDF
AWS reInvent 2023 recaps from Chicago AWS user group
PDF
hadoop事例紹介
PDF
Games + Amazon = Love - Presentation quo vadis 2011
PDF
Cloud Architectures - Jinesh Varia - GrepTheWeb
PPTX
Aws introduction tocloudstorage
PDF
Amazon Ec2
PDF
FAST Search for SharePoint
PDF
Common MongoDB Use Cases
PDF
Tweaking perfomance on high-load projects_Думанский Дмитрий
PDF
Searching conversations with hadoop
PDF
Talk at Microsoft Cloud Futures 2010
PDF
Hadoop and Hive Development at Facebook
 
PDF
Hadoop and Hive Development at Facebook
SimpleDBを使った ソーシャルアプリ構築事例
Amazon Web Services OverView
Jeff Barr Amazon Services Cloud Computing
Masterworks talk on Big Data and the implications of petascale science
Seize The Cloud
Scaling Out With Hadoop And HBase
Data Science Day New York: The Platform for Big Data
AWS reInvent 2023 recaps from Chicago AWS user group
hadoop事例紹介
Games + Amazon = Love - Presentation quo vadis 2011
Cloud Architectures - Jinesh Varia - GrepTheWeb
Aws introduction tocloudstorage
Amazon Ec2
FAST Search for SharePoint
Common MongoDB Use Cases
Tweaking perfomance on high-load projects_Думанский Дмитрий
Searching conversations with hadoop
Talk at Microsoft Cloud Futures 2010
Hadoop and Hive Development at Facebook
 
Hadoop and Hive Development at Facebook
Ad

More from Ryu Kobayashi (7)

PDF
PLAZMA TD Tech Talk 2018 at Shibuya: Hive2 as a new td hadoop core engine
PDF
Treasure Data on The YARN - Hadoop Conference Japan 2014
PDF
Huahin Framework for Hadoop, Hadoop Conference Japan 2013 Winter
KEY
Hadoop Conference Japan 2011 Fall
KEY
Developers summit cassandraで見るNoSQL
KEY
Hadoopソースコードリーディング第3回 Hadopo MR + Cassandra
KEY
Hadoopソースコードリーディング MapReduce障害時のフロー
PLAZMA TD Tech Talk 2018 at Shibuya: Hive2 as a new td hadoop core engine
Treasure Data on The YARN - Hadoop Conference Japan 2014
Huahin Framework for Hadoop, Hadoop Conference Japan 2013 Winter
Hadoop Conference Japan 2011 Fall
Developers summit cassandraで見るNoSQL
Hadoopソースコードリーディング第3回 Hadopo MR + Cassandra
Hadoopソースコードリーディング MapReduce障害時のフロー
Ad

Recently uploaded (20)

PDF
Review of recent advances in non-invasive hemoglobin estimation
PPTX
PA Analog/Digital System: The Backbone of Modern Surveillance and Communication
PDF
The Rise and Fall of 3GPP – Time for a Sabbatical?
PPTX
Effective Security Operations Center (SOC) A Modern, Strategic, and Threat-In...
PDF
TokAI - TikTok AI Agent : The First AI Application That Analyzes 10,000+ Vira...
PPTX
VMware vSphere Foundation How to Sell Presentation-Ver1.4-2-14-2024.pptx
PPTX
Detection-First SIEM: Rule Types, Dashboards, and Threat-Informed Strategy
PDF
Encapsulation theory and applications.pdf
PDF
CIFDAQ's Market Insight: SEC Turns Pro Crypto
PDF
Building Integrated photovoltaic BIPV_UPV.pdf
PDF
Approach and Philosophy of On baking technology
PDF
Per capita expenditure prediction using model stacking based on satellite ima...
PPTX
Digital-Transformation-Roadmap-for-Companies.pptx
PDF
Modernizing your data center with Dell and AMD
PPTX
KOM of Painting work and Equipment Insulation REV00 update 25-dec.pptx
PDF
KodekX | Application Modernization Development
PPTX
Cloud computing and distributed systems.
PPTX
20250228 LYD VKU AI Blended-Learning.pptx
PDF
Agricultural_Statistics_at_a_Glance_2022_0.pdf
PDF
Machine learning based COVID-19 study performance prediction
Review of recent advances in non-invasive hemoglobin estimation
PA Analog/Digital System: The Backbone of Modern Surveillance and Communication
The Rise and Fall of 3GPP – Time for a Sabbatical?
Effective Security Operations Center (SOC) A Modern, Strategic, and Threat-In...
TokAI - TikTok AI Agent : The First AI Application That Analyzes 10,000+ Vira...
VMware vSphere Foundation How to Sell Presentation-Ver1.4-2-14-2024.pptx
Detection-First SIEM: Rule Types, Dashboards, and Threat-Informed Strategy
Encapsulation theory and applications.pdf
CIFDAQ's Market Insight: SEC Turns Pro Crypto
Building Integrated photovoltaic BIPV_UPV.pdf
Approach and Philosophy of On baking technology
Per capita expenditure prediction using model stacking based on satellite ima...
Digital-Transformation-Roadmap-for-Companies.pptx
Modernizing your data center with Dell and AMD
KOM of Painting work and Equipment Insulation REV00 update 25-dec.pptx
KodekX | Application Modernization Development
Cloud computing and distributed systems.
20250228 LYD VKU AI Blended-Learning.pptx
Agricultural_Statistics_at_a_Glance_2022_0.pdf
Machine learning based COVID-19 study performance prediction

AWSを使ったトラッキングログ収集

  • 1. AWS @ryu_kobayashi
  • 2. • • • ... • ... • Twitter @ryu_kobayashi
  • 3. 2010 • • •
  • 5.
  • 6.
  • 11. Elastic Load Amazon Elastic Balancing Compute Cloud
  • 12. Elastic Load Amazon Elastic Balancing Compute Cloud Auto Scaling
  • 13. Elastic Load Amazon Elastic Balancing Compute Cloud Auto Scaling {
  • 14. Elastic Load Amazon Elastic Balancing Compute Cloud Auto Scaling {
  • 15. Elastic Load Amazon Elastic Balancing Compute Cloud Auto Scaling {
  • 18. } Amazon Elastic Amazon Compute Cloud SimpleDB
  • 19. SimpleDB Domain size 10 GB per domain Domain size 1 billion attributes per domain Domain name 3-255 characters a-z, A-Z, 0-9, ‘_’, ‘-’, and ‘.’ Domain per account 100 Attribute name-value paris per item 256 Attribute name length 1024 bytes Attribute values length 1024 bytes Item name Length 1024 bytes Attribute name, attribute value, and All UTF-8 characters that are valid in XML documents. item allow characters Attribute per PutAttributes operation 256 Attribute requested Per Select operation 256 Item per BatchPutAttributes operation 256 Maximum items in Select response 2500 Maximum query execution time 5 seconds Maximum number of unique attributes per select 20 exception Maximum number of comparisons per 20 Select exception Maximum response size for Select 1MB
  • 20. SimpleDB Domain size 10 GB per domain Domain size 1 billion attributes per domain Domain name 3-255 characters a-z, A-Z, 0-9, ‘_’, ‘-’, and ‘.’ Domain per account 100 Attribute name-value paris per item 256 Attribute name length 1024 bytes Attribute values length 1024 bytes Item name Length 1024 bytes Attribute name, attribute value, and All UTF-8 characters that are valid in XML documents. item allow characters Attribute per PutAttributes operation 256 Attribute requested Per Select operation 256 Item per BatchPutAttributes operation 256 Maximum items in Select response 2500 Maximum query execution time 5 seconds Maximum number of unique attributes per select 20 exception Maximum number of comparisons per 20 Select exception Maximum response size for Select 1MB
  • 22. PV/
  • 23. PV/ •
  • 24. PV/ • • 100
  • 25. PV/ • • 100 • 1 1 ...
  • 26. PV/ • • 100 • 1 1 ... • 8 10
  • 27. PV/ • • 100 • 1 1 ... • 8 10 • Item Attribute
  • 30. } Amazon Elastic Amazon Compute Cloud SimpleDB
  • 31. } Amazon Elastic Amazon Compute Cloud SimpleDB DC
  • 32. } Amazon Elastic Amazon Compute Cloud SimpleDB DC
  • 33. } Amazon Elastic Amazon Compute Cloud SimpleDB DC
  • 40. DC
  • 41. DC
  • 42. DC
  • 43. DC
  • 44. DC
  • 46. Cassandra MapReduce
  • 47. Cassandra MapReduce • HDFS
  • 48. Cassandra MapReduce • HDFS • Map
  • 49. Cassandra MapReduce • HDFS • Map • .......
  • 50. Cassandra MapReduce • HDFS • Map • .......
  • 51. ...