Data Virtualization: An Introduction
1 like119 views
The document discusses a webinar series on data virtualization that addresses challenges in data integration, highlighting its benefits in providing real-time access to diverse data sources without physical movement. Key themes include the evolution of data architectures, the importance of a data abstraction layer, and practical use cases demonstrating the flexibility and agility that data virtualization offers. It emphasizes that data virtualization can significantly reduce delivery times for analytics while hiding the complexity of underlying data infrastructure.
Data Virtualization: An Introduction
- 1. DATA VIRTUALIZATION PACKED LUNCH WEBINAR SERIES Sessions Covering Key Data Integration Challenges Solved with Data Virtualization
- 2. Data Virtualization: An Introduction Paul Shimabukuro Sales Engineer, Denodo Paul Moxon SVP Data Architectures & Chief Evangelist, Denodo
- 3. Agenda 1. Data Virtualization: An Introduction 2. Data Virtualization – How it works 3. Product Demo 4. Q&A 5. Next Steps
- 4. Data Virtualization: An Introduction 4
- 5. Data Integration – A Classic Approach 5 Operational Data Stores Staging Area Data Warehouse Data Marts Analytics and Reporting ETLETLETL
- 6. Data Integration – A Modern Data Ecosystem 6
- 7. The Data Integration Challenge 7 Manually access different systems IT responds with point-to- point data integration Takes too long to get answers to business users MarketingSales ExecutiveSupport Database Apps Warehouse Cloud Big Data Documents AppsNo SQL Businesses are reporting that integrating data from silos to support real-time insights has become a nightmare, especially when supporting large and complex data sets Big Data Fabric 2.0 Drives Data Democratization, May 9, 2019
- 8. 8 The Data Integration Challenge It is difficult to integrate numerous on-premises and cloud data sources. Traditional tools cannot integrate streaming data and data-at-rest in real time. It is difficult to maintain consistent data access and governance policies across data siloes. Traditional data integration is extremely resource intensive.
- 9. The Solution – A Data Abstraction Layer 9 Abstracts access to disparate data sources Acts as a single repository (virtual) Makes data available in real-time to consumers DATA ABSTRACTION LAYER “Enterprise architects are finding that traditional data architectures are failing to meet new business requirements, especially around data integration for streaming analytics and real-time analytics.” The Forrester Wave: Enterprise Data Virtualization, Jan 12, 2018 DATA VIRTUALIZATION PLATFORM
- 10. Source: “Gartner Market Guide for Data Virtualization, November 16, 2018” Data virtualization can be used to create virtualized and integrated views of data in-memory rather than executing data movement and physically storing integrated views in a target data structure. It provides a layer of abstraction above the physical implementation of data, to simplify query logic.
- 11. Data Virtualization 11 Consume in business applications Combine related data into views Connect to disparate data sources 2 3 1 DATA CONSUMERS DISPARATE DATA SOURCES Enterprise Applications, Reporting, BI, Portals, ESB, Mobile, Web, Users Databases & Warehouses, Cloud/Saas Applications, Big Data, NoSQL, Web, XML, Excel, PDF, Word... Analytical Operational Less StructuredMore Structured CONNECT COMBINE PUBLISH Multiple Protocols, Formats Query, Search, Browse Request/Reply, Event Driven Secure Delivery SQL, MDX Web Services Big Data APIs Web Automation and Indexing CONNECT COMBINE CONSUME Share, Deliver, Publish, Govern, Collaborate Discover, Transform, Prepare, Improve Quality, Integrate Normalized views of disparate data “Data virtualization integrates disparate data sources in real time or near-real time to meet demands for analytics and transactional data.” – Create a Road Map For A Real-time, Agile, Self- Service Data Platform, Forrester Research, Dec 16, 2015
- 12. 12 How Does It Work? Development Lifecycle Mgmt Monitoring & Audit Governance Security Development Tools and SDK Scheduled Tasks Data Caching Query Optimizer JDBC/ODBC/ADO.Net SOAP / REST WS U Business View Data Mart View J Application Layer Business Layer Unified View Unified View Unified View Unified View A J J Derived View Derived View J JS Transformation & Cleansing Data Source Layer Base View Base View Base View Base View Base View Base View Base View Abstraction
- 13. Data Virtualization Connects the Users to the Data That They Need 1. Data Virtualization allows you to connect to (almost) any data source 2. You can combine and transform that data into the format needed by the consumer 3. The data can be exposed to the consumers in a format and interface that is usable by them • Typically consumers use the tools that they already use – they don’t have to learn new tools and skills to access the data 4. All of this can be done without copying or moving the data • The data stays in the original sources (databases, applications, files, etc.) and is retrieved, in real-time, on demand 13 Cliffs Notes version (TL;DR)
- 14. Gartner – The Evolution of Analytical Environments 14 Operational Application Operational Application Operational Application IoT Data Other NewData Operational Application Operational Application Cube Operational Application Cube ? Operational Application Operational Application Operational Application IoT Data Other NewData 1980s Pre EDW 1990s EDW 2010s2000s Post EDW Time LDW Operational Application Operational Application Operational Application Data Warehouse Data Warehouse Data Lake ? LDW Data Warehouse Data Lake Marts ODS Staging/Ingest Unified analysis › Consolidated data › "Collect the data" › Single server, multiple nodes › More analysis than any one server can provide ©2018 Gartner, Inc. Unified analysis › Logically consolidated view of all data › "Connect and collect" › Multiple servers, of multiple nodes › More analysis than any one system can provide ID: 342254 Fragmented/ nonexistent analysis › Multiple sources › Multiple structured sources Fragmented analysis › "Collect the data" (Into › different repositories) › New data types, › processing, requirements › Uncoordinated views
- 15. 15 Gartner – Logical Data Warehouse “Adopt the Logical Data Warehouse Architecture to Meet Your Modern Analytical Needs”. Henry Cook, Gartner April 2018 DATA VIRTUALIZATION
- 16. − Market Guide for Data Virtualization, Gartner, November, 16, 2018 “Through 2022, 60% of all organizations will implement data virtualization as one key delivery style in their data integration architecture.” 16
- 17. 17 Denodo ‘Use Case’ Categories Customer Centricity APIs/Services ✓ Data as a Service ✓ Microservices/containers ✓ API Data Services ✓ Application Migration Machine Learning ✓ Data Catalog ✓ Metadata management ✓ Universal data access ✓ Governance, Risk, Compliance (GRC) ✓ Data Masking/Data Privacy ✓ Auditing/data lineage ✓ Hybrid/Multicloud Integration ✓ Cloud Data Analytics ✓ Cloud IT Modernization
- 18. Product Demonstration Data Virtualization – An Introduction 18 Sales Engineer, Denodo Paul Shimabukuro
- 19. 19 Demo Architecture What’s the impact of a new marketing campaign for each country? ▪ Historical sales data offloaded to Hadoop cluster for cheaper storage ▪ Marketing campaigns managed in an external cloud app ▪ Country is part of the customer details table, stored in the DW Sources Combine, Transform & Integrate Consume Base View Source Abstraction join group by country join Sales Campaign Customer
- 20. Demo 20
- 21. Key Takeaways 21
- 22. Key Takeaways 22 FIRST Takeaway Data Virtualization is a key technology when building a modern data architecture SECOND Takeaway It provides flexibility and agility and reduces the time to deliver data to the business by up to 10X THIRD Takeaway Data Virtualization hides the complexity of a constantly changing data infrastructure from the users FOURTH Takeaway In doing so, it allows you to introduce new technologies, formats, protocols, etc. without causing user disruption
- 23. Q&A
- 24. 24 Next Steps Access Denodo Platform in the Cloud! Take a Test Drive today! www.denodo.com/TestDrive GET STARTED TODAY
- 25. Thank you! © Copyright Denodo Technologies. All rights reserved Unless otherwise specified, no part of this PDF file may be reproduced or utilized in any for or by any means, electronic or mechanical, including photocopying and microfilm, without prior the written authorization from Denodo Technologies.