SlideShare a Scribd company logo
THE TRILLION-ROW SPREADSHEET™
    SPEAKER: Robert Lefkowitz
                        Director of Web
                        Development
                        1010data

Friday, July 27, 2012
The Trillion Row Spreadsheet
                        r0ml@1010data.com




                                                 2
Friday, July 27, 2012
Friday, July 27, 2012
4
Friday, July 27, 2012
Trillion Row Spreadsheet




                                                   5
Friday, July 27, 2012
Trillion Row Spreadsheet
                 Trillion Row Data Warehouse




                                               6
Friday, July 27, 2012
7
Friday, July 27, 2012
8
Friday, July 27, 2012
9
Friday, July 27, 2012
10
Friday, July 27, 2012
11
Friday, July 27, 2012
Data Warehouse
                        •   Operational -> Analytical
                            -   Design/Model
                            -   Data (cleaning)
                            -   Multiple data sources
                            -   Date (timeseries) analytics
                            -   Reporting (business intelligence)




                                                                    12
Friday, July 27, 2012
13
Friday, July 27, 2012
Operational Data

                        Operational Data           Operational Data



                                      Data Warehouse




                                                                      14
Friday, July 27, 2012
Operational Data

                        Operational Data               Operational Data



                                      Data Warehouse



                         Data Mart         Data Mart      Data Mart




                                                                          15
Friday, July 27, 2012
Operational Data

                        Operational Data                 Operational Data



                                      Data Warehouse



                         Data Mart         Data Mart        Data Mart




                                           Spreadsheet

                                                                            16
Friday, July 27, 2012
Operational Data

                        Operational Data                 Operational Data



                                      Data Warehouse



                         Data Mart         Data Mart        Data Mart




                                           Spreadsheet

                                                                            17
Friday, July 27, 2012
Operational Data

                        Operational Data                 Operational Data



                                      Data Warehouse



                         Data Mart         Data Mart        Data Mart




                                           Spreadsheet

                                                                            18
Friday, July 27, 2012
Operational Data

                        Operational Data                 Operational Data



                                      Data Warehouse




                                           Spreadsheet

                                                                            19
Friday, July 27, 2012
Operational Data

                        Operational Data                 Operational Data



                                      Data Warehouse




                                           Web based
                                           Spreadsheet

                                                                            20
Friday, July 27, 2012
Spreadsheets lack ... ?
                        •   Content management
                        •   Business process management
                        •   Automated scheduling
                        •   Authentication
                        •   Audit
                        •   Collaboration
                        •   Enterprise level roll-up



                                                          21
Friday, July 27, 2012
Spreadsheets lack ... ?
                        •   Scale




                                                  22
Friday, July 27, 2012
Operational Data

                        Operational Data                 Operational Data



                                      Data Warehouse




                                           Web-based
                                           Spreadsheet

                                                                            23
Friday, July 27, 2012
Operational Data

                        Operational Data                 Operational Data



                                      Data Warehouse
                                       Spreadsheet




                                           Web-based
                                           Spreadsheet

                                                                            24
Friday, July 27, 2012
Operational Data

                        Operational Data                 Operational Data



                                      Data Warehouse
                                       Spreadsheet

                                                   If this is fast enough, you can
                                                   recalculate the summaries
                                                   from the detail on the fly....


                                           Web-based
                                           Spreadsheet

                                                                               25
Friday, July 27, 2012
Operational Data

                        Operational Data                 Operational Data



                                      Data Warehouse



                         Data Mart         Data Mart        Data Mart



                                           Web-based
                                           Spreadsheet

                                                                            26
Friday, July 27, 2012
Operational Data

                        Operational Data                 Operational Data



                                      Data Warehouse
                                       Spreadsheet

                                                   If this is fast enough, you can
                                                   recalculate the summaries
                                                   from the detail on the fly....


                                           Web-based
                                           Spreadsheet

                                                                               27
Friday, July 27, 2012
Spreadsheets lack ... ?
                        •   Professional programming practices
                            -   Requirements analysis
                            -   Formal regression testing
                            -   Model/View separation
                            -   Object-Orientation
                            -   Static typing
                            -   Naming conventions




                                                                 28
Friday, July 27, 2012
29
Friday, July 27, 2012
The Trillion Row Spreadsheet
                        r0ml@1010data.com




                                                 30
Friday, July 27, 2012
Friday, July 27, 2012

More Related Content

PPTX
Adam and tennis and ping pong
PPTX
Growing Intelligence by Properly Storing and Mining Call Center Data
KEY
Datacamp @ Transparency Camp 2010
PDF
Data fluency for the 21st century
PDF
Investigative Analytics- What's in a Data Scientists Toolbox
PPSX
Data warehouse
PPSX
Data warehouse
PDF
Flying Blind On A Rocket Cycle: Customer-centered Product Strategy for Machin...
Adam and tennis and ping pong
Growing Intelligence by Properly Storing and Mining Call Center Data
Datacamp @ Transparency Camp 2010
Data fluency for the 21st century
Investigative Analytics- What's in a Data Scientists Toolbox
Data warehouse
Data warehouse
Flying Blind On A Rocket Cycle: Customer-centered Product Strategy for Machin...

Similar to THE TRILLION ROW SPREADSHEET(tm) from Structure:Data 2012 (20)

PPTX
Investigating data scientists
PPT
20100119 mis
PPTX
Google refine from a business perspective
PPTX
Google refine from a business perspective
PPTX
Google refine tutotial
PDF
BIG DATA: AN AUGMENTED INTELLIGENCE FOR STRATEGIC DECISION MAKING from Struct...
PDF
Analytics 101 - How to build a data-driven organisation? - Rafał Małanij, Get...
PPTX
Google refine tutotial
PPTX
Google refine tutotial
PPTX
Google refine tutotial
PPTX
Google refine tutotial
PPTX
Information Management and Analytics
PDF
Data Analyst Roadmap – Key Steps to Master Data Science!
PDF
Bend WebCAM - Making Smarter Decisions with Google Analytics
PDF
ETIS09 - Data Quality: Common Problems & Checks - Presentation
PDF
Understanding Big Data Analytics - solutions for growing businesses - Rafał M...
PDF
How to Build a Successful Data Team - Florian Douetteau @ PAPIs Connect
PPTX
How to Build a Successful Data Team - Florian Douetteau (@Dataiku)
KEY
UK - Agile Data Applications on Hadoop
PPT
Data warehousing
Investigating data scientists
20100119 mis
Google refine from a business perspective
Google refine from a business perspective
Google refine tutotial
BIG DATA: AN AUGMENTED INTELLIGENCE FOR STRATEGIC DECISION MAKING from Struct...
Analytics 101 - How to build a data-driven organisation? - Rafał Małanij, Get...
Google refine tutotial
Google refine tutotial
Google refine tutotial
Google refine tutotial
Information Management and Analytics
Data Analyst Roadmap – Key Steps to Master Data Science!
Bend WebCAM - Making Smarter Decisions with Google Analytics
ETIS09 - Data Quality: Common Problems & Checks - Presentation
Understanding Big Data Analytics - solutions for growing businesses - Rafał M...
How to Build a Successful Data Team - Florian Douetteau @ PAPIs Connect
How to Build a Successful Data Team - Florian Douetteau (@Dataiku)
UK - Agile Data Applications on Hadoop
Data warehousing
Ad

More from Gigaom (20)

PPTX
Structure 2014 - The strategic value of the cloud - Joe Weinman
PPTX
Structure 2014 - The right and wrong way to scale - Rackspace
PPTX
Structure 2014 - The future of cloud computing survey results
PPTX
Structure 2014 - Launchpad Competition
PPTX
Structure 2014 - Disrupting the data center - Intel sponsor workshop
PPTX
Structure 2014 - Cloud trends - Battery
PDF
Structure Data 2014: HOW MICRODATA CAN SAY A LOT ABOUT MACROECONOMICS, David ...
PDF
Structure Data 2014: QLIK SPONSOR WORKSHOP: ANALYTICS THE WAY NATURE INTENDED...
PDF
Structure Data 2014: FIVE MYTHS ABOUT BIG DATA, Amit Bendov
PDF
Structure Data 2014: AMID BILLIONS OF METRICS, YOUR SOFTWARE IS TRYING TO TEL...
PDF
Structure Data 2014: SISENSE SPONSOR WORKSHOP: ON BEER, CHIPS AND DATA,
PDF
Structure Data 2014: INVERTING 80/20: BEYOND BESPOKE BIG DATA, Ari Gesher
PDF
Structure Data 2014: TRACKING A SOCCER GAME WITH BIG DATA, Chris Haddad
PDF
Structure Data 2014: TECH AGAINST HUMAN TRAFFICKING AND ILLICIT NETWORKS, Jus...
PDF
Structure Data 2014: DATA DRIVEN DESIGN AT FORMULA ONE SPEED, Geoff McGrath
PDF
Structure Data 2014: IS VIDEO BIG DATA?, Steve Russell
PDF
Structure Data 2014: BIG DATA ANALYTICS RE-INVENTED, Ryan Waite
PDF
How Data is Remaking E-commerce - from Roadmap 2013
PDF
25 Favorite Experiences in Tech - from Roadmap 2013
PDF
How Moore’s Law is Influencing Design - from Roadmap 2013
Structure 2014 - The strategic value of the cloud - Joe Weinman
Structure 2014 - The right and wrong way to scale - Rackspace
Structure 2014 - The future of cloud computing survey results
Structure 2014 - Launchpad Competition
Structure 2014 - Disrupting the data center - Intel sponsor workshop
Structure 2014 - Cloud trends - Battery
Structure Data 2014: HOW MICRODATA CAN SAY A LOT ABOUT MACROECONOMICS, David ...
Structure Data 2014: QLIK SPONSOR WORKSHOP: ANALYTICS THE WAY NATURE INTENDED...
Structure Data 2014: FIVE MYTHS ABOUT BIG DATA, Amit Bendov
Structure Data 2014: AMID BILLIONS OF METRICS, YOUR SOFTWARE IS TRYING TO TEL...
Structure Data 2014: SISENSE SPONSOR WORKSHOP: ON BEER, CHIPS AND DATA,
Structure Data 2014: INVERTING 80/20: BEYOND BESPOKE BIG DATA, Ari Gesher
Structure Data 2014: TRACKING A SOCCER GAME WITH BIG DATA, Chris Haddad
Structure Data 2014: TECH AGAINST HUMAN TRAFFICKING AND ILLICIT NETWORKS, Jus...
Structure Data 2014: DATA DRIVEN DESIGN AT FORMULA ONE SPEED, Geoff McGrath
Structure Data 2014: IS VIDEO BIG DATA?, Steve Russell
Structure Data 2014: BIG DATA ANALYTICS RE-INVENTED, Ryan Waite
How Data is Remaking E-commerce - from Roadmap 2013
25 Favorite Experiences in Tech - from Roadmap 2013
How Moore’s Law is Influencing Design - from Roadmap 2013
Ad

Recently uploaded (20)

PDF
NewMind AI Weekly Chronicles - August'25-Week II
PPTX
A Presentation on Artificial Intelligence
PDF
A comparative study of natural language inference in Swahili using monolingua...
PPTX
KOM of Painting work and Equipment Insulation REV00 update 25-dec.pptx
PPTX
TLE Review Electricity (Electricity).pptx
PDF
A novel scalable deep ensemble learning framework for big data classification...
PPTX
cloud_computing_Infrastucture_as_cloud_p
PDF
WOOl fibre morphology and structure.pdf for textiles
PDF
Heart disease approach using modified random forest and particle swarm optimi...
PDF
Mushroom cultivation and it's methods.pdf
PPTX
A Presentation on Touch Screen Technology
PDF
ENT215_Completing-a-large-scale-migration-and-modernization-with-AWS.pdf
PDF
Building Integrated photovoltaic BIPV_UPV.pdf
PDF
7 ChatGPT Prompts to Help You Define Your Ideal Customer Profile.pdf
PDF
Web App vs Mobile App What Should You Build First.pdf
PDF
Univ-Connecticut-ChatGPT-Presentaion.pdf
PDF
Video forgery: An extensive analysis of inter-and intra-frame manipulation al...
PDF
Hybrid model detection and classification of lung cancer
PDF
From MVP to Full-Scale Product A Startup’s Software Journey.pdf
PDF
gpt5_lecture_notes_comprehensive_20250812015547.pdf
NewMind AI Weekly Chronicles - August'25-Week II
A Presentation on Artificial Intelligence
A comparative study of natural language inference in Swahili using monolingua...
KOM of Painting work and Equipment Insulation REV00 update 25-dec.pptx
TLE Review Electricity (Electricity).pptx
A novel scalable deep ensemble learning framework for big data classification...
cloud_computing_Infrastucture_as_cloud_p
WOOl fibre morphology and structure.pdf for textiles
Heart disease approach using modified random forest and particle swarm optimi...
Mushroom cultivation and it's methods.pdf
A Presentation on Touch Screen Technology
ENT215_Completing-a-large-scale-migration-and-modernization-with-AWS.pdf
Building Integrated photovoltaic BIPV_UPV.pdf
7 ChatGPT Prompts to Help You Define Your Ideal Customer Profile.pdf
Web App vs Mobile App What Should You Build First.pdf
Univ-Connecticut-ChatGPT-Presentaion.pdf
Video forgery: An extensive analysis of inter-and intra-frame manipulation al...
Hybrid model detection and classification of lung cancer
From MVP to Full-Scale Product A Startup’s Software Journey.pdf
gpt5_lecture_notes_comprehensive_20250812015547.pdf

THE TRILLION ROW SPREADSHEET(tm) from Structure:Data 2012

  • 1. THE TRILLION-ROW SPREADSHEET™ SPEAKER: Robert Lefkowitz Director of Web Development 1010data Friday, July 27, 2012
  • 2. The Trillion Row Spreadsheet r0ml@1010data.com 2 Friday, July 27, 2012
  • 5. Trillion Row Spreadsheet 5 Friday, July 27, 2012
  • 6. Trillion Row Spreadsheet Trillion Row Data Warehouse 6 Friday, July 27, 2012
  • 12. Data Warehouse • Operational -> Analytical - Design/Model - Data (cleaning) - Multiple data sources - Date (timeseries) analytics - Reporting (business intelligence) 12 Friday, July 27, 2012
  • 14. Operational Data Operational Data Operational Data Data Warehouse 14 Friday, July 27, 2012
  • 15. Operational Data Operational Data Operational Data Data Warehouse Data Mart Data Mart Data Mart 15 Friday, July 27, 2012
  • 16. Operational Data Operational Data Operational Data Data Warehouse Data Mart Data Mart Data Mart Spreadsheet 16 Friday, July 27, 2012
  • 17. Operational Data Operational Data Operational Data Data Warehouse Data Mart Data Mart Data Mart Spreadsheet 17 Friday, July 27, 2012
  • 18. Operational Data Operational Data Operational Data Data Warehouse Data Mart Data Mart Data Mart Spreadsheet 18 Friday, July 27, 2012
  • 19. Operational Data Operational Data Operational Data Data Warehouse Spreadsheet 19 Friday, July 27, 2012
  • 20. Operational Data Operational Data Operational Data Data Warehouse Web based Spreadsheet 20 Friday, July 27, 2012
  • 21. Spreadsheets lack ... ? • Content management • Business process management • Automated scheduling • Authentication • Audit • Collaboration • Enterprise level roll-up 21 Friday, July 27, 2012
  • 22. Spreadsheets lack ... ? • Scale 22 Friday, July 27, 2012
  • 23. Operational Data Operational Data Operational Data Data Warehouse Web-based Spreadsheet 23 Friday, July 27, 2012
  • 24. Operational Data Operational Data Operational Data Data Warehouse Spreadsheet Web-based Spreadsheet 24 Friday, July 27, 2012
  • 25. Operational Data Operational Data Operational Data Data Warehouse Spreadsheet If this is fast enough, you can recalculate the summaries from the detail on the fly.... Web-based Spreadsheet 25 Friday, July 27, 2012
  • 26. Operational Data Operational Data Operational Data Data Warehouse Data Mart Data Mart Data Mart Web-based Spreadsheet 26 Friday, July 27, 2012
  • 27. Operational Data Operational Data Operational Data Data Warehouse Spreadsheet If this is fast enough, you can recalculate the summaries from the detail on the fly.... Web-based Spreadsheet 27 Friday, July 27, 2012
  • 28. Spreadsheets lack ... ? • Professional programming practices - Requirements analysis - Formal regression testing - Model/View separation - Object-Orientation - Static typing - Naming conventions 28 Friday, July 27, 2012
  • 30. The Trillion Row Spreadsheet r0ml@1010data.com 30 Friday, July 27, 2012