DataVirtulization
- 1. By Bhaven Chavan bhaven2001@yahoo.com 6/23/2016 Data Virtualization 6/23/2016 Confidential | 2016 DISCLAIMER Note: It is understood that the material in this presentation is intended for general information only and should not be used in relation to any specific application without independent examination and verification of its applicability and suitability by professionally qualified personnel. Those making use thereof or relying thereon assume all risk and liability arising from such use or reliance.
- 2. Agenda • High level walkthrough of the Data Virtualization concepts and its possible utilization: • What is Data Virtualization? • Why use Data Virtualization? • When not to use Data Virtualization? • What functionality it provides? • Data Virtualization Overview • Data Virtualization and Big Data/NoSQL Overview • *Drawbacks* • Q&A 6/23/20166/23/2016 Confidential | 2016
- 3. What is Data Virtualization? • Data Virtualization is an umbrella term used to describe any approach to data management that allows an application to retrieve and manipulate data without requiring technical details about the data, such as how it is formatted or where it is physically located. • Data virtualization is a technique to deliver the data by consuming many desperate data sources (internal/external) with a simplified, integrated view of trusted data within enterprise using real-time or near real time mechanism to achieve the business goals that support business transactions, analytics, predictive analytics, and other workloads and pattern. 6/23/20166/23/2016 Confidential | 2016
- 4. Why use Data Virtualization? • Today’s complex world with so much data, business is looking for instant access to all the complex data irrespective of the location to meet the immediate market needs with an Agile manner. • It helps in reducing the cost in data replication and data consolidation. • It adds value in Data Governance. • Improves the Data Quality. • It reduce data storage required. 6/23/20166/23/2016 Confidential | 2016
- 5. When not to use Data Virtualization? • Data Virtualization is not the solution to every data integration problems. Such as, persisting need of the data in a warehouse (UDL/ODS) or data-mart, along with E-T-L or E-L-T is better solution for specific use case. Sometimes a hybrid solution is the right answer. 6/23/20166/23/2016 Confidential | 2016
- 6. What functionality it provides? • Virtualized Data Access • It connects to the different data sources and make them accessible from a common data access point. • Data Transformation • It transforms improved data quality and it reformats the source data the way consumer needs. • Data Federation • It combines results set from across the multiple heterogeneous source systems. • Data Delivery • It publishes result sets as views and/or data services executed by client application or users when requested. 6/23/20166/23/2016 Confidential | 2016
- 7. Data Virtualization Overview Data Virtualization Server OLTP Databases Data Warehouse & Data Marts Applications ASSET JMS SQL Unstructured Data XSLT ESB SOAP EXCEL Big Data Store Social Media Data HIVE JSON Private Data External Data Prop. OLTP Application Analytics & Reporting ODBC/ JDBC/SQL JDBC/SQL Service API XML/SOAP REST/JSON Mobile App Website XQuery DAX/MDX Dashboard 6/23/2016 Confidential | 2016
- 8. Denodo: Data Virtualization Overview 6/23/2016 Confidential | 2016
- 9. Data Virtualization and Big Data/NoSQL Overview 6/23/20166/23/2016 Confidential | 2016
- 10. Data Virtualization and Big Data/NoSQL • It unleashes the full value of Big Data for analytics • It speeds up development on Big Data sources • It offers an evolutionary adoption of Big Data • It makes Big Data available to everyone • Higher Big Data ROI 6/23/20166/23/2016 Confidential | 2016
- 11. NoSQL as Sand Box 6/23/2016 OLTP Databases Reporting & Analytics SQL SQL SQL SQL NoSQL Data Staging Area Data Warehouse Data Marts Data Virtualization Server 6/23/2016 Confidential | 2016
- 12. NoSQL for Storing Cold Data 6/23/2016 OLTP Databases SQL SQL SQL SQL NoSQL Data Staging Area Data Warehouse Data Marts Reporting & Analytics Data Virtualization Server 6/23/2016 Confidential | 2016
- 13. NoSQL as Staging Area 6/23/2016 Data Virtualization Server OLTP Databases SQL SQL SQL NoSQL Data Staging Area Data Warehouse Data Marts Reporting & Analytics 6/23/2016 Confidential | 2016
- 14. NoSQL as Extra Data Warehouse Database 6/23/2016 OLTP Databases SQL SQL SQL SQL NoSQL Data Staging Area Data Warehouse Data Marts Reporting & Analytics Data Virtualization Server 6/23/2016 Confidential | 2016
- 15. NoSQL ETL Processing 6/23/2016 Data Virtualization Server Data Warehouse Reporting & Analytics OLTP Databases SQL SQL SQL SQL NoSQL Data Staging Area Data Marts 6/23/2016 Confidential | 2016
- 16. Drawbacks • Another/A new DataStore in production to take care of. • May impact Operational systems response time, particularly if under- scaled to cope with unanticipated user queries or not tuned early on. • Does not impose heterogeneous data model, meaning the user has to interpret the data, unless combined with Data Federation and business understanding of the data. • Requires a defined Governance approach to avoid budgeting issues with the shared services. • Not suitable for recording the historic snapshots of data. Data Warehouse is better for this. • Change management “ is a huge overhead, as any changes need to be accepted by all applications and users sharing the same virtualization kit. 6/23/20166/23/2016 Confidential | 2016