Solution Centric Architectural Presentation - Implementing a Logical Data Warehouse
0 likes125 views
The document outlines the implementation of a logical data warehouse using the Denodo platform, spearheaded by Holly Burch and Jon McManus since 2018. It discusses the transformation from a slow data processing system to an agile framework that integrates multiple data sources, creating a data marketplace for easier access and enhanced data blending. The summary highlights the benefits of using Denodo in facilitating backend migrations and supporting agile workflows while combining on-premise and cloud-based data.
Solution Centric Architectural Presentation - Implementing a Logical Data Warehouse
- 1. Logical Data Fabric: The Future of Data Management and Analytics
- 2. #DenodoDataFest Implementing a Logical Data Warehouse Data Architect, Sharp Healthcare Holly Burch
- 4. Introduction: Holly Burch • Primary Denodo architect and platform administrator Denodo Journey • Jon McManus, new VP Enterprise Analytics, arrives Oct 2018 with the vision of implementing a Logical Data Warehouse • Began implementing Denodo platform Summer 2019 • Currently at more than 100 data sources from more than 40 source systems • Denodo 8.0 platform currently on-premise and planning to move it to the cloud
- 5. Vision: Logical Data Warehouse • Apply an architectural approach that enables access to multiple, diverse data sources (both on premise and multi-cloud) while appearing as one “logical” data source to users. Data Marketplace • Provide a single place to browse, consume, and create consumable content.
- 7. Frustrations: • 30 projects/year • Process = Slow grind • New Source • 2 – 3 months • Enhancement • 2 – 3 weeks • Limited access for end-users • Hard to blend data AGILE? NO
- 9. Transformation: • Centers of Excellence • Process = Agile teams • New source: 2 – 3 weeks • Enhancement: 2 – 3 days • Easy to blend/enrich data • Navigate EMR and ERP system changes • Back-end migrations with no impact to end-users • Marketplace access • Many ways to deliver AGILE? YES Logical Data Warehouse
- 10. Use Case: Migration off Cloudera Impala to Snowflake Cloudera Impala • Real-Time streaming of HL7 messages – Lab orders/results, Scheduling • Real-Time streaming of Electronic Medical Record data • Denodo views of Lab data feeding to critical covid dashboards • Denodo views of Scheduling data exported hourly for patient feedback Snowflake Migration • Migrated storage of data from Impala to Snowflake • Brought in new Snowflake base views to Denodo • Updated existing integration layer views to source from Snowflake base views instead of Impala base views. • 3 - 4 weeks to fully test and implement in Denodo
- 11. Summary: Logical Data Warehouse with Denodo as a key component allows us to: • Create a Data Marketplace for client consumption • Bring on-prem and cloud-based data together to appear as one “logical” source • Blend/enrich data between sources • Navigate major EMR and ERP platform changes • Migrate back-end sources without impacting end users • Support agile workflows
- 12. © Copyright Denodo Technologies. All rights reserved Unless otherwise specified, no part of this PDF file may be reproduced or utilized in any for or by any means, electronic or mechanical, including photocopying and microfilm, without prior the written authorization from Denodo Technologies. Thank You!