Presentation by Bart Gielen (DataSense) at the Data Vault Modelling and Data Governance conference on Oct. 17, 2019: Data Warehouse Automation and Data Vault implementation in Open Source technology at Bank Degroof Petercam
1 like280 views
The document discusses the Data Vault 2.0 automation project at Bank Degroof Petercam, outlining the challenges with their current data architecture and the need for improved data governance. It highlights the role of Talend as a leading data integration platform and describes a hands-on workshop for implementing Data Vault 2.0 automation. The goal is to enhance data delivery and meet various reporting requirements through a structured methodology and tool integration.
Presentation by Bart Gielen (DataSense) at the Data Vault Modelling and Data Governance conference on Oct. 17, 2019: Data Warehouse Automation and Data Vault implementation in Open Source technology at Bank Degroof Petercam
- 1. The Data Vault Modelling & Data Governance Conference 17/10/2019 Data Vault 2.0 Automation for EDW
- 2. Bank Degroof Petercam • 1871 • Independent Group • Activities • Private Banking • Institutional Asset Management • Investment Banking • Asset Services • Headquarter in Brussels
- 3. DataSense • Part of Cronos Group • Focus on Data Vault 2.0 automation projects • Proven reference architecture • (Enterprise) Open Source • Headquarter in Leuven • New company : NXTDATA
- 4. Talend - a leader in Data Integration & Integrity 3,000+ Customers 1,200+ Employees Recognized as a Leader 100% Cloud Growth Revenue $198M +33%
- 5. Talend - a global community who provides guidance and insights 3M+ Downloads of Talend Open Studio 50.000+ Posts in Community Forums 900+ connectors
- 7. Data Vault 2.0 Automation Dan Linstedt “Businesses are looking to do things better, faster and cheaper, and you really can’t do that without automating or generating your solutions in your data warehousing system. In order for your information delivery team to be agile and provide the data downstream quicker, you need a metadata- driven automation tool.”
- 8. Bank Degroof Petercam Case • Current Data Warehouse • contains mainly Belgian data • data is stored & loaded in different ways in data marts for certain purposes and are highly modified • time to market problem • Luxembourg is not having a proper data warehouse • Many departments need data • To respond to regulatory, legal, accounting, customer, analytic and marketing reports. • To provide consolidated dashboard for decisional management. • To provision system and application with common standardized data • Lots of projects are asking for raw to near-raw data in order to perform process of their own . • Data & Architecture principles are defined and need to be implemented into the landscape.
- 9. THE CURRENT DATA ARCHITECTURE CAN’T KEEP UP
- 10. WHAT’S NEEDED IS A FORCE MULTIPLIER
- 11. TO PUT MORE DATA TO WORK WITH LESS IT BUDGET
- 12. Approach Hands-on Workshop Assessment Study Data Vault 2.0 Automation Pilot Data Vault 2.0 Automation Project
- 13. Approach Hands-on Workshop Data Vault 2.0 Automation Pilot Data Vault 2.0 Automation Project Assessment Study • As-is, to-be & specific budget estimates • Governance • Team skills / Profiles • Methodology / Standards • Solution Architecture • Data Modeling • Data Quality Strategy • Metadata Approach • Testing Strategy • Technical Architecture • Tools such as ELT, CDC, database, deployment, reporting, …. • Documentation • Data Roadmap à Give advice + a proposal
- 14. Approach Data Vault 2.0 Automation Pilot Data Vault 2.0 Automation Project Assessment Study • Data Vault 2.0 session • Intro in Vaultspeed (Data Vault 2.0 automation tool) • Hands-on session and exercise in Vaultspeed • Generate via the Vaultspeed the Talend jobscripts and import them into Talend Data Integration • See the result of the custom made logging • Q&A Hands-on Workshop
- 15. Approach Data Vault 2.0 Automation Project Assessment Study Hands-on Workshop • 4 sprints of 3 weeks • Setup the environment • Data Vault 2.0 and Vaultspeed training • Integrate a first set of tables of the contract system of LU • Knowledge sharing sessions • Deliver an information mart for Accounting LU Data Vault 2.0 Automation Pilot
- 16. Approach Hands-on Workshop Assessment Study Data Vault 2.0 Automation Pilot Data Vault 2.0 Automation Project
- 17. BDP version of the EDW – Functional view
- 18. NO Source Driven Data Vault GIPERSF GISOCIF Natural Persons Additional Information Legal Persons Additional Information RELATIONS GIRELAF PARTY HUB_PERSONS SAT_PERSONS_GIRELAF SAT_PERSONS_CRM RAW Data Vault SAT_PERSONS_GIPERSF SAT_PERSONS_GISOCIF Data Sources CRM CONTRACT SYSTEM Solution Logical Data Model (=SLDM)
- 19. More than a data warehouse … CRM CONTRACT SYSTEM BE NEW CONTRACT SYSTEM BIDATAHUB CONTRACT SYSTEM LU
- 20. BDP current version of the EDW – Technical view Landing Zone Raw Data Vault layer Talend extraction jobs Data Sources ELT Business Data Vault layer Generated DDL & ELT JSON queueing MVIEWS Virtualized Information layer ELT
- 22. Thank you for your attention! Bart Gielen Managing Partner +32 479 81 68 83 bart.gielen@datasense.be www.datasense.be www.linkedin.com/in/bartgielen-datasense