Govern and Protect Your End User Information
0 likes232 views
The document discusses the role of data virtualization in data governance and protection, particularly in the context of GDPR compliance and cloud adoption challenges. It highlights how Denodo's solutions can facilitate real-time data access, enhance data governance, and simplify integration between on-premise and cloud data sources. Additionally, a case study illustrates how a financial company utilizes data virtualization to manage compliance and improve data management processes.
Govern and Protect Your End User Information
- 2. Govern and Protect Your End User Information Nageswar Cherukupalli Vice President & Group Manager, Infosys Clinton Cohagan Chief Enterprise Data Architect, Lawrence Livermore National Lab
- 3. LLNL-PRES-933672 This work was performed under the auspices of the U.S. Department of Energy by Lawrence Livermore National Laboratory under contract DE-AC52-07NA27344. Lawrence Livermore National Security, LLC Data Governance & Protection with Data Virtualization Denodo Fast Data Strategy Conference 2018 Clinton Cohagan Chief Enterprise Information Architect, WTE Program Group Lead Manager, ASQ Division Computation Directorate April 12, 2018
- 4. Data Governance with Data Virtualization
- 5. 5 Business can now make faster & better decisions with all data accessible by any tool of choice Data Virtualization Reference Architecture IT Flexible Source Architecture Business Flexible Tool Choice IT can now be more strategic & move at a more intentional pace without affecting business agility
- 6. All NNSA Sites Required Sharing / Integrating Data Services NSC NNSS Y-12 Pantex SRS LANL SNL LLNL NSC NNSS Y-12 Pantex SRS LANL SNL LLNL NSC Pantex LLNL vs
- 7. Tackle GDPR (General Data Protection Regulation) with Data Virtualization
- 8. GDPR Principles and Impacts Comes into effect May 2018 Aims to give control back to all individuals within the European Union over their personal data Extends the scope of the EU data protection law to all foreign companies processing data of EU residents Affects how companies collect, use and transfer personal data Requires Accountability ▪ GDPR requires controllers to show HOW they comply with principles Promotes Protection and Accuracy ▪ Personal information must be accurate and able to be corrected on request ▪ On-line access (360 degree view)
- 9. Data Services can Provide 360 Views Across the Enterprise LDAP XML API SQL Server DB Oracle DB
- 10. Catalog of Data Services ▪ Browse and Search ▪ Descriptions, Categories, and Tags ▪ Relationships and Data Lineage Data Discovery & Self Service Access ▪ Search Content of Data Services ▪ Customize and Save Queries ▪ Export to Local File Data Governance & Data Quality ▪ Certified Data Services ▪ Web-based and Secure ▪ Does not require SQL expertise Denodo’s Dynamic Data Catalog Enhances Data Service Usability
- 12. Data Virtualization – Enabling Cloud Adoption and Data Governance 12
- 13. Cloud Adoption – Trends and Challenges 13 Cloud Adoption Trends • More and more Enterprises are adopting Cloud platforms and solutions due to many benefits that such solutions provide: - Cost reduction - Agility and Time to Market - Elasticity and Scalability - Drive Innovation • New data stores and applications are being created or migrated to the Cloud. Cloud native data stores (e.g. Azure SQL DW, AWS Redshift, etc) are being leveraged. Customers are also adopting PaaS and SaaS offerings and sometimes IaaS offerings from Cloud providers such as AWS, Microsoft, Google, etc. • Most customers are not adopting a big-bang approach to Cloud adoption. Existing On-premise data assets would continue leading to an hybrid ecosystem of data assets deployed On-premise and in the Cloud. Challenges Cloud adoption presents new challenges: • Integration across On-premise and Cloud data stores - Data might need to be replicated in order to present a combined view. • Real Time data access - Data from both On-premise and Cloud data assets may be required by consuming applications/BI reports in real time. Traditional ETL processes would introduce latency. • Disruption to Business - During Cloud migrations (re-architecture, lift and shift, etc.) existing business processes could get impacted • Security and Governance - Data Governance is still of paramount importance and a hybrid ecosystem makes it more complex to manage. Metadata, lineage, quality and security of the data needs to be ensured. • Compliance - It may not be feasible to move data from existing Data Centers to the Cloud and vice-versa due to compliance requirements, more so for Financial Services companies.
- 14. How Data Virtualization can help with Cloud Adoption and Data Governance 14 Denodo is helping solve and mitigate some of the key challenges that enterprises are facing with Cloud adoption and associated Data Governance challenges • Hybrid Data Hubs - Denodo can help create a hybrid data hub by combining On-premise and Cloud data stores. Deploy Denodo instance closer to the major data sources whether On-premise or on the Cloud. Both Cloud native use cases and hybrid use cases can be supported by this solution pattern. • Avoid Data Replication - With virtualization, data does not need to be replicated. Caching and MPP fabric capabilities can address any performance bottlenecks. Avoiding data replication also results in many other add on benefits with faster time to market, avoiding data copies, reducing storage costs, etc. • Real Time data access - There is no movement of the data, unlike a tradition ETL processes, enabling real time access to data. • Avoid Disruption to Business Process during Cloud adoption Journey - Denodo can abstract and hide the complexity and changes to the data sources while changes to the data store and data pipelines happen making the process less disruptive. • Security and Governance - Denodo can provide single point of access and control while managing fine grained data access. This ensures better governance and auditability in the consumption layer across various consumption patterns. It also enables lesser complexity by managing security in a single access layer as opposed to managing security at multiple layers. PII data can be encrypted as well. • Compliance - Using Data virtualization can address many compliance needs since data movement across data center and cloud can be avoided. • Information discovery and Self Service - Denodo provides self-service capabilities wherein data consumers are able to search metadata , explore data relationship and lineage , create their own views using a UI based interface which can subsequently be shared and published.
- 15. Denodo can help realize better Data Governance & Security with Cloud Adoption 15 • Centralized Security & Governance - Data can be secured in the Virtualization layer, which can act as single point for securing and monitoring data access. - Abstraction of Security model. Denodo hides the complexity of managing security in the underlying data sources (e.g. relation sources, Hadoop data lake, Files, SaaS) - Flexibility to integrate with LDAP or Active Directory. - Can support very fine grained controls based on RBAC models at row and column level. Masking and encryption can also be done to as per the requirement. • Better Auditing - History of Data access/queries are stored and can be used for any auditing. • Data Lineage, Impact Analysis - Denodo stores metadata of sources ,transformations etc. implemented as part of virtualization layer and uses this to provide data governance capabilities like Data lineage and Impact analysis. - Denodo metadata is exposed through APIs and can be integrated with external tools. - Denodo can also integrate with Reference Data management tools such as Collibra and IBM IGC, to further strengthen enterprise wide data governance initiatives.
- 16. Denodo Cloud Data Virtualization – Solution Overview 16 SOAP/REST Web Service0 Azure SQL DW/ RedShift Data LakeSQL Server / Oracle/Netezza/Teradata Security & Governance Self Service Connect Combine Publish Monitoring& Auditing On – Prem. Cloud Security & Governance Self Service Connect Combine Publish Monitoring& Auditing Reports Data Consumers Apps • Create a data virtualization layer across On- premise and Cloud data sources with a bridge between both. • Suitable for use cases where data virtualization already plays a key role in On-premise setup. • As new data sources emerge in cloud, Denodo instance in the cloud can be used for data virtualization for cloud sources. • This setup simplifies the integration and data access across cloud and On-premise data sources. Both instances see each other as another JDBC source and data is aggregated and processed at the right place, reducing data movement over the network. • For cloud native use cases, where all or majority of sources are present in cloud, Denodo instance in cloud would be sufficient.
- 17. It is amongst the leading financial holding company, has provided financing, leasing and advisory services to small and middle market businesses for over a century Client • As a SIFI, the bank is subject to additional scrutiny and faces tougher stress- testing expectations from regulators • Acquisition of another bank necessitated multiple data integration initiatives. • Ability to aggregate and identify risk exposures quickly and accurately at the enterprise-wide level, across business lines and legal entities • Accuracy, Integrity, Completeness, Timeliness and Adaptability • Old habits die hard. Culture is a difficult thing to change. Problem • Company has to deal with its legacy systems and data, along with those of acquisition of the new bank. • Different technologies are used to host contracts, positions and balances for different businesses and regions • Supporting firm-wide risk analysis, CCAR and other comprehensive regulatory reporting across regions/products requires a single comprehensive view of the contracts, positions and balances across all lines of businesses / regions for various products • Applications are built and deployed in isolation • Challenges with data integration due to data is incorrect, missing, uses the wrong format, incomplete, and so on • Missing of master data and Governance rules • Show improved schedule compliance , Predictable project budgets, Enhanced data consistency, lower project and operational costs Challenges One Source of the Truth: Authoritative sources of data by domains, Access all data via common provisioning point Build for Reuse, Leverage Investment: Build once, use many – CCAR, Liquidity, Risk, Avoid point-to-point integration, simplify Minimize Data Replication: Eliminate data redundancy and proliferation, Eliminate redundant data reconciliation efforts. Enable Effective Data Governance: Enforcing policies, standards and procedures, Defining authoritative source of data, Efficient data lineage and metadata management, Monitoring data quality, Empowering data stewards Simplified Architecture, Agile Data Integration: Faster time-to-market delivery, Incremental information delivery via standard APIs, Simplified architecture, data makes minimal hops ArchitectureConsideration • Created an virtual Canonical logical layer integrating the desperate source systems for data needs from New Business into Customer • Underlying changes should not affect end user’s view of the data • Early identification of Gaps from source system and requirement perspective saving cost and time • Involving Business from initial phase to avoid last minute surprises • Analyzing data quality upfront and certifying • Quickly consolidation from various systems to see the requirement feasibility and quality issues • Changes to source systems can be easily sustainable • Regulation and policy change rules can be applied quickly and viewed for refinement of compliance parameters throughout the organization Solution Case Study – How a Leading Financial Company is using Data Virtualization to address regulatory aspects 17
- 18. Case Study - Logical view of Solution Architecture DW (SQL Server ,Oracle ) Data Mart (SQL Server ,Oracle) Source Systems & External Data P H Y S I C A L L A Y E R F O R M A T L A Y E R B U S I N E S S L A Y E R Caching/ Reference Data( SQL Server) Data Quality Source Systems (SQL Server ,Oracle) Files) Meta Data External Feeds (DnB, Chatham) Virtual Marts Reference Data Virtual Views Pre-Processed Views A B S T R A C T L A Y E R canonical Models RealTimeAPI’s (WebServices) Actimize ReportingLayer (OBIEE,QLikView, SSRS) FormattedDataOutput Data Down streams 18 Denodo Data Virtualization Denodo Data Virtualization
- 19. M O D E R A T E D B Y 19 Clinton Cohagan Chief Enterprise Data Architect, Lawrence Livermore National Lab Nageswar Cherukupalli Vice President & Group Manager, Infosys Saptarshi Sengupta Principal Product Marketing Manager Governing and Securing Data with Data Virtualization
- 20. Q&AQ&A
- 21. 21 DOWNLOAD DENODO EXPRESS DENODO FOR AWS DENODO FOR AZURE Download Denodo Express Next Steps Access Denodo Platform in the cloud! 30 day free trial available!
- 22. Thank you! © Copyright Denodo Technologies. All rights reserved Unless otherwise specified, no part of this PDF file may be reproduced or utilized in any for or by any means, electronic or mechanical, including photocopying and microfilm, without prior the written authorization from Denodo Technologies. #FastDataStrategy