Product Keynote: Advancing Denodo’s Logical Data Fabric with AI and Advanced Semantics
0 likes134 views
The document discusses Denodo's advancements in logical data fabric, emphasizing the transition from monolithic architectures to distributed and logical models that enhance data access and reduce replication. It outlines Denodo's vision for modern data architectures, including features like a data marketplace, an AI and recommendations engine, and collaborations for improved data discovery. Key takeaways stress the importance of data virtualization and the expansion of Denodo's capabilities in augmented data catalog and advanced semantics.
Product Keynote: Advancing Denodo’s Logical Data Fabric with AI and Advanced Semantics
- 1. Advancing Denodo’s Logical Data Fabric with AI and Advanced Semantics #DenodoDataFest
- 2. #DenodoDataFest with AI Advancing Denodo’s Logical Data Fabric with AI and Advanced Semantics and Advanced Semantics Advancing Denodo’s Logical Data Fabric with AI and Advanced Semantics CTO & Executive VP ALBERTO PAN
- 3. Agenda 1. Denodo Vision for Modern Data Architectures 2. Logical Data Fabric: Completing the Vision
- 4. Denodo Vision for Modern Data Architectures
- 5. #DenodoDataFest Monolithic Architectures ▪ Centralized: All/Most data in a single system ▪ All data needs to be copied to the target system ▪ Data needs to be replicated to adapt it for each new use case (data marts) ▪ Managed by a central IT data team ▪ Physical: Consumers need to know: ▪ Data Location ▪ How data is represented in that system ▪ What access methods are supported in that system ▪ Examples: Data Warehouse, Data Lake, Data LakeHouse
- 6. #DenodoDataFest Logical and Distributed Architectures ▪ Distributed: Data resides at multiple systems / locations ▪ Modern analytics needs are too diverse: one size never fits all ▪ Hybrid and multicloud ▪ Logical: Consumers access data through semantic models, decoupled from data location and physical schemas ▪ Semantic models adapt to consumer needs and enforce common policies ▪ Minimize data replication ▪ Allows for technology evolution and infrastructure changes (e.g. cloud transition)
- 7. #DenodoDataFest Monolithic Architectures: One Size Never Fits All “Inherent in the LDW architecture is the recognition that a single data persistence tier and type of processing is inadequate to meet the full scope of modern data and analytics demands” The Practical Logical Data Warehouse (Dec 2020) by Henry Cook, Rick Greenwald and Adam Ronthal
- 8. #DenodoDataFest Monolithic Architectures: Slow and Rigid ▪ Need to ingest all data in a new system ▪ Existing analytics systems cannot be reused ▪ Data is replicated for every different purpose / use case ▪ Changes require modifying pipelines and datasets at multiple stages
- 9. #DenodoDataFest Monolithic Architectures: Lock-in ▪ Applications and Reports are dependent of the central system ▪ Cannot easily move workloads to other systems
- 10. #DenodoDataFest The Logical Data Warehouse Unified data access layer to multiple data sources
- 11. #DenodoDataFest Data Fabric: Supported by the Major Analysts Source: Forrester Enterprise Data Fabric Wave, June 2020 Source: Demystifying the Data Fabric Gartner,, September 2020
- 12. #DenodoDataFest Data Mesh: Decentralized Organization SQL Operational EDW Data Lakes Files SaaS APIs REST GraphQL OData Event Product Customer Location Employee Customer Management Event Management Human Resources 1. Domains create virtual models in separate schemas. Execution servers can be scaled independently 2. Domains can choose and autonomously evolve their data sources 3. Domains can share standardized definitions. Products can be used to define other products 4. A central team can set guidelines and enforce global security, quality and governance policies in the virtual models
- 13. #DenodoDataFest Denodo’s Logical Data Fabric ▪ Unified Data Integration and Delivery ▪ Allows reusing existing analytics systems ▪ Allows using the best system for each need ▪ Abstraction: no lock-in ▪ Evolve / optimize infrastructure without affecting data consumers ▪ Dramatically Increased Productivity ▪ Minimize data replication: virtual or smart, selective data replication
- 14. 14 In 2020, organizations utilizing data virtualization will spend 45% less on building and managing data integration processes.” Source: Gartner 2018 Data Virtualization Market Guide 83% reduction in revenue-driving project ramp time 65% Decreased data delivery times by over ETL” Source: Forrester Research - The Total Economic Impact Of Denodo’s Data Virtualization Software
- 15. Logical Data Fabric: Completing the Vision
- 16. #DenodoDataFest Denodo Logical Data Fabric: Pillars Pillars of the Logical Data Fabric: 1. Agile Data Delivery using multiple integration styles 2. Data Catalog 3. AI and Recommendations engine 4. Advanced Semantic Layer and Extended Metadata 5. Solution Manager: DataOps and Management
- 17. #DenodoDataFest Data Catalog: A Data Marketplace for the Business Data Marketplace where business users and data scientists can discover, understand and get Access to the data available in the Logical Data Fabric Leverages active metadata to guide data discovery
- 18. #DenodoDataFest Data Catalog: Guided Data Discovery Personalized recommendations and shortcuts to most used datasets. Endorsements and ranking MY RECOMENDATIONS
- 19. #DenodoDataFest Data Catalog: Collaboration Collaborative Features: • Endorsements • Deprecation Notices • Warning Messages • Tags and others
- 20. #DenodoDataFest ML/AI Based Automation ▪ Privileged position to gather key information about data access and data delivery ▪ Semantic information and active metadata used to feed ML automation processes
- 21. #DenodoDataFest Automatic Recommendations for Smart Query Acceleration Denodo uses AI to analyze past queries in the system and: 1. Automatically detects patterns to classify similar queries 2. Identifies frequent, common intermediate result sets for each group 3. Precomputes the ones that maximize performance in the workload At query time, the optimizer can automatically use them to save processing in other queries following similar patterns Cache or Data Source
- 22. #DenodoDataFest Advanced Semantic Layer: Tags ▪ Expand Denodo Semantic Layer beyond data delivery ▪ Tags, endorsements, comments, activity usage,… ▪ Tags add additional semantics that can be used for: ▪ Data Discovery ▪ Search and classification ▪ Security and Governance
- 23. #DenodoDataFest Policy description Roles affected by the policy Columns affected by the policy Masking expression
- 24. #DenodoDataFest Key Takeaways 1. Modern Data Architectures should be Logical an Distributed 2. Denodo and Data Virtualization are key enablers of modern data architectures 3. Denodo capabilities keep expanding in areas such as Augmented Data Catalog, Artificial Intelligence and Advanced Semantics
- 25. © Copyright Denodo Technologies. All rights reserved Unless otherwise specified, no part of this PDF file may be reproduced or utilized in any for or by any means, electronic or mechanical, including photocopying and microfilm, without prior the written authorization from Denodo Technologies. Thank You!