Data Virtualization Journey: How to Grow from Single Project and to Enterprise Adoption
0 likes605 views
Intel's IT transformation leverages data virtualization to enhance data integration and accelerate service delivery. The initiative has resulted in significant efficiency gains, including time savings in developing web services and the ability to manage diverse data sources more effectively. Establishing governance and communication channels has further supported agile development and engagement within the organization.
Data Virtualization Journey: How to Grow from Single Project and to Enterprise Adoption
- 1. Joshua Wise, IT Enterprise Architect, Intel Data Virtualization Intel’s Journey to Enterprise Adoption Fast Data Strategy Virtual Summit
- 2. 2 Legal Notices This presentation is for informational purposes only. INTEL MAKES NO WARRANTIES, EXPRESS OR IMPLIED, IN THIS SUMMARY. Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Performance tests, such as SYSmark and MobileMark, are measured using specific computer systems, components, software, operations and functions. Any change to any of those factors may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your contemplated purchases, including the performance of that product when combined with other products. For more complete information about performance and benchmark results, visit www.intel.com/benchmarks Intel and the Intel logo are trademarks of Intel Corporation in the U.S. and/or other countries. * Other names and brands may be claimed as the property of others. Copyright © 2016, Intel Corporation. All rights reserved.
- 3. 33 >6,320 IT employees 71 global IT sites >104,820 Intel employees1 153 Intel sites in 72 Countries 61 Data Centers (91 Data Centers in 2010) 80% of servers virtualized (42% virtualized in 2010) >220,000+ Client Devices 100% of laptops encrypted 100% of laptops with SSD’s >50,100 handheld devices 238 mobile applications developed Source: 2015 summary information provided by Intel IT as of Jan 2016 1Total employee count does not include wholly owned subsidiaries that Intel IT does not directly support Intel IT Vital Statistics
- 4. 4 Recognize the Challenge • Lack of consistent capability to integrate data from disparate data sources and deliver using agile standardized methods. • Intel’s data is globally distributed across heterogeneous tools & technologies. • New data sources (ex: big data) & consumers (ex: emergence of SaaS). • New information exchange channels (ex: mobility). Ad-hoc data requests Multiple service protocols Ex: SOAP & REST REST SOA P SOAP SOAP Point-to-point interfaces Ex: UOM, LOC…etc. . . MDM . . Enterprise Application . . ROO
- 5. 5 See the Opportunity Data Virtualization - An agile data integration method that simplifies information access Data Consumers Data Sources TTM Agility Manageability Reuse view web service web service Cloud SaaS web service
- 6. 6 Start Small, Think Big TTM New data service: >50-80% time savings Multiple protocol (REST, SOAP,…): 100% time savings No need for highly skilled programmers (except for complex web services) Ex: Supplier service was developed in 8 hrs. vs. 180hrs. Agility Decouple data consumers from data sources /providers Merge Structured /Unstructured data Ease of external (Cloud/SaaS) data integration Ex: Supplier service changed data source w/o impacting consumers End-To-End Manageability Ability to track Consumers, Data lineage, Consumption Simplified Architecture & Capability Stack Ex: impact analysis Accelerated time-to- information • Accelerate Services strategy • Support Ad-hoc data requests • Facilitate Data explore/discovery
- 7. 7 • Supplier Master Data is highly shared data about companies that Intel purchases from, pays, outsource manufactures with, etc. • Choosing a Supplier is the point of entry to many business process. If it fails or is slow, it impacts all 70+ downstream consumers • Prior to DV: Development resources were extremely constrained & development time was months • After DV: Able to create web services in an agile manner in 2 weeks through to Production without a highly skilled Developer Data Virtualization for Supplier Master Data Build for failure – Redundant HA Service Pair Supplier Master Data Enterprise Data Warehouse Data Virtualization ETL Supplier Invoicing Supplier Registration Supplier Analytics Backup DB for Failover Purposes Real Time Primary DB Service Calls
- 8. 8 • A recent DB purchase did not come with an Identity Management Solution. The team needed a solution that could source users and roles from a directory for assignment in the DB. • The team worked in Denodo to utilize Active Directory as a data source to provide the DB with an IDMS. • An ETL solution, was used to call the Denodo user service that was created and enable a full and delta role function for scheduled load back into the DB. • This innovative solution was delivered more quickly than any other possible option and offset the need to purchase an IDMS. Data Virtualization for Directory Services Directory Data Virtualization In-memory DB Solution Bulk Load Users and Groups ETL ETL
- 9. 9 The MySamples project needed a way to quickly show the status of samples requests. The team had explored several solutions but all had limitations in the data source connectivity space or performance space. A custom service was their next best option, requiring time and resources. Using Denodo, Samples was able to bring 3 different data sources together, join and filter the data then produce a single service back in real time to serve their analyst UI. • MySamples DB – MSSQL Server containing customer information • ERP – A proprietary system containing the samples request information (if requested) • EM – A proprietary system containing the samples shipment status (if shipped) A Sample Example in MySamples Samples DB MSSQL EM Data Virtualization MySamples Application Shipment Status Customer Samples Service Calls ERP Delivery Note
- 10. Educate and Build Trust • Establish clear governance • Create a developer communication channel • Ratify governance with internal working groups • Communicate an upgrade cadence • Internal training for developers • Training as a gate for development • Require quick code reviews for migration clearance • Create a flexible architecture that lets you scale in small and large units across zones, regionally or globally • Inspire platform confidence – 24/7 support 10 Scaling to the Enterprise
- 11. 11 Guide and Govern Appropriately • Create Web services and Business Views with business terminology to abstract. • Align DV governance with SOA and ETL governance. • Use Caching for Static and Semi-static data • Move to CRUD usage after mastery of Read only • Get specific • e.g. Services: <15 seconds and <50mb payload • Large ETL on a case by case basis • Ensure the health of the platform and set expectations Enterprise Governance 11
- 12. Intel’s DV Release Process 12 • Retain governance, streamline touch points • Focus on Time to Information • Enable Agile development • Use CI/CD technologies to speed agility • Developer self help • Wiki • Social/collaboration site • Video Channel Creating a Path to Speed and Agility Engage 10 min Develop Document 10 min Register 5 min Review 10 min Migrate 5 min Audit Train and Inform
- 13. 13 Create Platform Engagement at All Levels • Executive Messaging Create critical success indicators, measure progress to plan, communicate milestones • Customer Messaging Innovate your platform and capabilities, communicate your wins and showcase your customers • Vendor Messaging Influence through regular engagements, request platform enhancements, report internally and externally on vendor support Influence for Growth -200 0 200 400 600 800 1000 2013 2014 2015 2016 2017 Growth of Data Services Goal Actual Projected
- 14. 14 The Results of the Enterprise Journey The value of Data Virtualization as a technology offering at Intel is strong. • Establishing a framework for data virtualization governance and growth has created a path to speed and agility for our developers. • Early successes as well as demonstrated performance and consistent support have built trust with our customers and management.
- 15. Learn more about Intel IT’s Initiatives at www.intel.com/IT Sharing Intel IT Best Practices With the World