Designing a Better Planet with Big Data and Sensor Networks (for Intelligent Sensor Networks Conference 2014, Philips High Tech Campus, Eindhoven)
28 likes4,206 views
The document discusses the advancements and challenges associated with big data and sensor networks, emphasizing the growing significance of sensor data over social data. It highlights the evolution of data management techniques, the integration of various data sources, and the importance of real-time decision-making in automated industries. Additionally, it addresses the security concerns related to big data, suggesting that trust management is crucial for protecting digital content and user information.
Designing a Better Planet with Big Data and Sensor Networks (for Intelligent Sensor Networks Conference 2014, Philips High Tech Campus, Eindhoven)
- 1. Designing a Better Planet with Big Data and Sensor Networks ISN Conference 2014, Eindhoven Rainer Sternfeld @rsternfeld Rainer Sternfeld, CE September 2014 November 4, 2014
- 2. ABB OneCampus (2009-2011) 2 November 2014 ABB Baltic States 2006-2011 Business Development Manager, Baltics (1,300 people, $300M revenue) Business Development • Fast-charging network for electric cars (first nation-wide in the world) • Regional + Local BU strategies • New product rollout and ramp-up • Production management software dev. Corporate Development • OneCampus production facilities • SAP implemented in 3 countries • 5S + COPQ implementation • New operations, processes and policies • Restructuring of operations Statue of Liberty of Estonia (2007-2009) eMobility Estonia (2011) Data buoy for phytoplankton (2008-2011) UGV+manipulator for DoD (2005-2006) About the author | Rainer Sternfeld
- 3. Swimming in sensors, drowning in data 3 November 2014 Data management will not scale as we know it Sensor data will outgrow social data in 3 years Sensor data is too big to move
- 4. Trends in industrial machine data 4 November 2014 Current Market Connecting Devices Higher velocities Larger volumes Wider varieties Future Market Automated Industries Real-time Decisions Data & Insight Markets VERACITY VELOCITY
- 5. 2004 2014 5 November 2014 • Nobody talks about Big Data (really) • No easy and cheap way to scale business • New growth driven by social media • Smartphones “don’t exist” in the public eye • MapReduce was becoming a household name (although Google had already abandoned it) • Data is manageable • “One size fits all” horizontal solutions • Companies invest heavily in Big Data • Amazon S3 has changed how new software is built • Growth driven by connected devices • Hadoop is the de facto data processing engine • HDFS is the de facto storage layer • The semantic web dream is crushed • Companies don’t know what data they have • Domain-specific integrated engines Source: Bryan Cantrill, Joyent CTO, http://www.slideshare.net/bcantrill/velocity2014 A decade of Big Data
- 6. We love our conscious devices 6 November 2014
- 7. Map of all devices on the Internet August 2, 2014 7 November 2014
- 8. What about the real world? Air. Land. Oceans. Space. 8 November 2014
- 9. Sensor Data Value Chain HARDWARE SENSORS RAW DATA SOFTWARE ANALYTICS 9 November 2014 SATELLITES DRONES CARS BUOYS OIL PLATFORMS SHIPS TRACTORS LIGHTING PHYSICAL CHEMICAL BIOLOGICAL OPTICAL NON-OPTICAL (RF) TIME-SERIES RASTER ARRAYS VECTORS SEISMIC VIDEO ACOUSTIC QA/QA OUTLIER DETECTION MODELING DATA FUSION DATA LOGISTICS VISUALIZATION ENHANCEMENT DERIVATIVES CUSTOMER APPLICATIONS ENERGY WEATHER AGRICULTURE TRANSPORTATION INSURANCE TELECOMMS LIGHTING HEALTHCARE EXCLUSIVITY AND UNIQUE SOLUTIONS ARE KEY
- 10. Luckily, sensor data is easy. Right… • Poor interoperability, metadata and standards • Legacy formats and protocols • More data, more noise • High barrier of entry (know your market and workflows) • High barrier of entry 2 (know your hardware) • Miscalibration due exposure to the environment • Diverse data acquisition patterns and sampling rates • Irregular updates, whitelists, lost data, lost devices 10 November 2014
- 11. Robotic ocean-borne sensor platforms increase productivity LIQUID ROBOTICS WAVE GLIDER 11 November 2014
- 12. 2004 2014 2K 400K n < 10 n > 1000 2D 4D 6 weeks 6 months 12 November 2014 A decade in marine acoustics sensor data summer year-round sample rate # of sensors dimensions time span activity PGS SURVEY VESSEL, 12KM STREAMERS
- 13. Tomorrow’s offshore oil exploration is unthinkable without integrated data systems USE CASES • MetOcean planning, expertise and monitoring • Fast crisis response • Common Operating Picture • Data vendor delivery, reporting, logs • Helps reduce non-productive time • Full software integration for centralized access to data THE MOST NORTHERN OIL PLATFORM IN THE ARCTIC, LUKOIL (RUSSIA) 13 November 2014
- 14. More data needs powerful tools. Increase the data fluency of your operations. USE CASES • Contextual data streams across sources • SCADA • Weather and environmental data • Drones and other robotic measurement platforms • Satellite imagery • 3D Common Operating Picture • Full software integration for centralized access to data • Event notifications 14 November 2014
- 15. 15 November 2014 Not so fast… Using data, Maersk proves smart routing pays off MAERSK LINE
- 16. Image Credit: NASA Statistical weather models don’t work any more. Data-driven forecasting is the only viable way. 16 November 2014
- 17. 17 November 2014 Prediction models are applied to local sensors and are domain-specific
- 18. SPIRE, A SAN FRANCISCO STARTUP BUILDING NON-IMAGING LOW-ORBIT NANOSATELLITES USING RF SENSORS Satellites are getting smaller and cheaper. 150 launched since 2011 (3x of the market estimate) 18 November 2014
- 19. 19 November 2014 Unmanned vehicles are estimated to grow 10x in 10 years Image Credit: Northrop Grumman
- 20. Image Credit: http://iphonedroneimagery.com/ Precision Agriculture is a $3.7 trillion market 20 November 2014
- 21. 21 November 2014 Connected cars are the next frontier in consumer markets GOOGLE’S SELF-DRIVING CAR CONCEPT 2014
- 22. DisSceonvseorry DEantgai ne 22 November 2014
- 23. From a small buoy to Big Data 2008 2012 2014 Ocean Data Management Analytical Sensor Data Platform 23 November 2014 X-Buoy 450 Market: $2 billion Competitors: 100+ producers Scalability: poor to limited Market: $5 billion Competitors: 25+ Scalability: good but slow Market: $100+ billion Competitors: 15+ Scalability: very scalable & fast if P/M fit
- 24. One interface to discover local, remote, open and vendor data 24 November 2014
- 25. Explore, visualize, monitor and build datasets Advanced Data Discovery Raster data / heat map overlays Access with third party applications Raster data / quiver plots Graph Monitor Build custom datasets 25 November 2014
- 26. Integrate, curate, control access and monitor dataflows Rich browsing experience of existing data Integrate and configure new data channels Roles, authorizations and logs Dataflow Dashboard 26 November 2014
- 27. Access to vendor and open data Image Credit: NASA OPEN DATA • 42,000+ data streams • 33 organizations • Up to 100 years of historic data • 7,400+ professionals VENDOR DATA • Discover data you didn’t have • Buy single datasets • Instantly subscribe to vendor deliveries • All your vendors in one place 27 November 2014
- 28. Keep the data where it is. Let us discover and index it. < Dynamic Datasets 28 November 2014 DATA PIPELINE Acquisition Transformation Indexing Storage Configuration Dashboard DATA CURATION DATA DISCOVERY DATA STUDIO Access Control Data Curation Asset Directory Metadata Edit References Audit Logs Query Browse Contextualize Visualize Grouping Bookmark Filter Visualization Data Fusion Monitoring Annotation Collaborate Marketplace Analytics APIs EMBEDDED FEATURES Crawlers LOCAL SOURCES REMOTE SOURCES VENDORS DATA OPEN DATA Crawlers Integration Integration
- 29. Supported data types and formats 29 29 November 2014 DATA TYPES ARRAYS MODELS VIDEO SONARS (ADCP) SPATIO-TEMPORAL INDEXING TIME-SERIES VECTORS RASTERS INSTRUMENTS SATELLITES IN-SITU DEVICES HF RADARS SEISMIC M M M Metadata-level
- 30. Image Credit: General Electric Sources http://www.wired.com/images_blogs/beyond_the_beyond/2012/11/ge-industrial.jpg 30 November 2014
- 31. Image Credit: General Electric Sources http://www.environmentalleader.com/wp-content/uploads/2012/11/GE-industrial-internet.jpg 31 November 2014
- 32. Image Credit: CISCO 32 November 2014
- 33. What about security? 33 November 2014 CONFIDENTIALITY INTEGRITY AVAILABILITY STANDARDS
- 34. There are always the bad guys 34 November 2014
- 35. Big Data clouds need trust management • Trusted digital certificates to authenticate the identity of devices • Tools to protect software code integrity from being compromised • Protecting a user's private information • Protecting and managing digital content rights 35 November 2014
- 36. Index the real world 36