Data warehouse modernization programme by TOBY WOOLFE at Big Data Spain 2014
1 like789 views
IBM implemented a significant data warehouse modernization program for General Motors, focusing on integrating big data technologies such as Hadoop to optimize data management and analytics. The initiative aims to reduce operating expenses by 25-30%, increase measurable business value, and improve operational efficiency through a centralized enterprise architecture. The project responds to GM's need for a robust data infrastructure amidst challenges like extensive vehicle recalls, while also enhancing real-time data analytics capabilities.
Data warehouse modernization programme by TOBY WOOLFE at Big Data Spain 2014
- 1. DATA WAREHOUSE MODERNIZATION PROGRAMME TOBY WOOLFE BIG DATA SOLUTIONS LEADER IBM
- 2. © 2014 IBM Corporation Case studies from around the world help explain what is working in the world of Hadoop, real time analytics and internet connected devices. IBM delivers a significant data warehouse modernization programme to General Motors Toby Woolfe, Big Data Industrial and Automotive Solutions Leader, IBM Europe Toby.woolfe@uk.ibm.com +44 7795 328 742
- 3. A New Era of Smart We are on the precipice of awareness of, and adoption of, internet connected devices far beyond the Smartphone... Devices: 145 actuators, 70 on-board computers, 4700 relays and 70 sensors, including radar, sonar, accelerometer, camera, rain sensors. 433MB per minute 1 car year = 1TB Plus existing enterprise data, social media data, 3rd party suppliers,...etc 2 © 2014 IBM Corporation
- 4. A New Era of Smart But how fast does information currently get delivered? Monthly reporting? Based originally on the lunar cycle, the Julian Calendar was introduced by Julius Caesar in 46BC Quarterly reporting? Celebrated by Druids for centuries, the four seasons result from the yearly revolution of the earth around the sun and the tilt of the earth’s axis relative to the plane of revolution. Annual reporting? In temperate and polar regions, the seasons are marked by changes in the intensity of sunlight that reaches the Earth's surface, variations of which may cause animals to go into hibernation or to migrate, and plants to be dormant Weekly reporting? Recorded in Babylonian carvings dated 6th century, the origin of the 7 day week was probably based on a quarter of the lunar cycle (though inaccurate) 3 © 2014 IBM Corporation
- 5. A New Era of Smart “GM Opens New Data Center Modeled on Google, Facebook” CIO Randy Mott launched a new $130 million data center Chief Executive Dan Akerson said ”If we are going to win, we must turn IT into a competitive advantage and not treat it as something that is just a utility,” The IT Operations and Command Center in the $130-million Enterprise Data Center at the General Motors Technical Center in Warren, Michigan. 4 © 2014 IBM Corporation
- 6. A New Era of Smart GM’s people, process and technology evolution GM has hired about 1,500 software developers and engineers, up from “close to zero” just one year ago, according to Tim Cox, CIO of GM global development services About 70% of the 10,000 IT staff will be focused on innovation Until recently, GM was using 23 different data centers. Now the company is moving to two The site in Warren has the capacity to hold more than 10,000 ‘pizza box’ size servers, as well as smaller-size servers and larger mainframes The data warehouse incorporates both the cutting edge IBM BigInsights for Hadoop technology as well as more traditional Massively Parallel Processing technologies historically used for data warehousing Source/thanks to 5 © 2014 IBM Corporation
- 7. A New Era of Smart General Motors wiki Business profile Global vehicle sales leader for 77 consecutive years from 1931 – 2007 212,000 staff in 157 countries Business problem GM has issued 45 recalls in 2014 involving 28 million cars worldwide The cost of one recall due to faulty ignition switches, which has been linked to at least 13 deaths, is $1.3 billion IT transformation General Motors (GM) is in the process of constructing a single global information warehouse that will become the foundation for all business analytics and decision support across the enterprise. The business objectives are – to reduce operating expenses for data warehouses by 25-30% – deliver 10x the measurable business value – double IT project capability – and integrate existing data warehouse systems with new technology to offload data, reduce latency, reduce costs and improve performance. 6 © 2014 IBM Corporation
- 8. A New Era of Smart Abstract: IBM delivers a significant data warehouse modernization programme to General Motors High performance and continuously available data management environment 329,000 users with a 10% concurrency rate Grow to approximately one petabyte in size over three years IBM proposed a centralized enterprise architecture leveraging a traditional data warehouse matched with IBM Big Insights Hadoop technology for big data analytics Data types: • Inventory control of parts • Manufacturing equipment and assembly line data • Warranty and services data from dealers • Telemetry data from vehicles • Customer services and social media data 7 © 2014 IBM Corporation
- 9. A New Era of Smart Big data = disparate data! 8 © 2014 IBM Corporation
- 10. A New Era of Smart Probabilistic matching 9 © 2014 IBM Corporation
- 11. A New Era of Smart Analyst validation of key IBM technologies.... ....and Analyst validation of IBM potential future phase technologies.... 10 © 2014 IBM Corporation
- 12. A New Era of Smart Capabilities required 1. Efficient data protocols 2. Real time analytics 3. Capture all data to a landing zone 4. High performance analytics platform 5. Application Platform 6. Data governance & integration 11 © 2014 IBM Corporation
- 13. A New Era of Smart IBM conclusions applicable to enterprise data warehouse modernisation learned from work with GM GM’s objectives published in WSJ: –Reduce operating expenses for data warehouses by 25-30% –Deliver 10x the measurable business value –Double IT project capability –Integrate existing data warehouse systems with new technology to offload data, reduce latency, reduce costs and improve performance Educate enterprise architecture staff on the new disruptive technology capabilities delivered from Hadoop, and its integration into existing data architectures Engage in a revue of data warehouse architecture to harness the cost savings, speed advantages and business focus that new technologies such as IBM BigInsights for Hadoop brings to market Experiment with new data asset/business objective combinations with IBM’s Big Data experts 12 © 2014 IBM Corporation
- 14. A New Era of Smart Case study illustrates the hidden value of machine data For a volume production car fleet using telematics data, IBM correctly predicted warranty codes/claims 43 days out with 86% accuracy and only 1% false positives For a new electric car model, IBM predicted warranty claims 60 days out with 96% accuracy and 1% false positives on 619 vehicles using Big Data and analytics technology 13 © 2014 IBM Corporation
- 15. A New Era of Smart Broad range of role based visualisation and analytical capabilities Data scientist Business person R Script support SPSS Predictive analytics Integration with Cognos BI Spreadsheet visualisation of data 360 degree view of machine/client/location Delivered using the BigInsights wizard to accelerate the use of Hadoop data 14 © 2014 IBM Corporation
- 16. A New Era of Smart BigInsights visualisation examples 1 –Hadoop raw data in spreadsheet format 2 - Sort, filter etc. 15 © 2014 IBM Corporation
- 17. A New Era of Smart ...and onto many kinds of analytical capability using Cognos 16 © 2014 IBM Corporation
- 18. A New Era of Smart How is Big Data Being Used in Industry? Product Development & Engineering Patent Analytics Social Media Analytics Quality & Warranty Analytics Manufacturing & Quality Predictive Maintenance Quality Early Warning Operational Efficiency Supply Chain Analytics Connected Vehicle V2V, V2C, V2I, V2X Automotive Telematics Real-time Alerts Geospatial Analytics Text Analytics Predictive Failsafe Analytics Driver Specific Predictive Analytics Condition Based Monitoring Marketing & Sales Social Media Analytics Text Analytics Customer 360 Actionable Customer Intelligence Next Best Action After Sales Fraud Analytics Regulatory Analytics Social Media Analytics Warranty Analytics Customer 360 Next Best Action 17 © 2014 IBM Corporation
- 19. A New Era of Smart Experiment with Hadoop 18 © 2014 IBM Corporation
- 20. A New Era of Smart Suggested next action: Create an inventory of your structured and unstructured data and assess where business value will be found Internal External Complaints Telematic Data Workflow Data Geolocation Txn History Data Machine failure data Financial Data Shipping schedule data Warranty History Payment History Machine Performance Call Data Recordings Google Alerts Social Media Maintenance records Activities Data Catastrophic Emails Web Traffic Data Structured Unstructured MDM Customer Data Relatively easy to acquire Difficult to acquire 19 © 2014 IBM Corporation
- 21. A New Era of Smart 20 20 © 2014 IBM Corporation
- 22. 17TH ~ 18th NOV 2014 MADRID (SPAIN)