Data Staging Strategy
1 like824 views
This document outlines different strategies for data staging when loading data into a data warehouse. It discusses extracting data in either pull or push mode, using a staging layer with either an ELT or direct load approach, and loading either full data or just changes. The staging layer can store data in files or tables before it is transformed and loaded into the data warehouse.
Data Staging Strategy
- 1. Data Staging Strategy By: Milind Zodge
- 2. Points to consider Define data extraction mode Define Staging layer design Define Types of Data Load Define Types of data stores in staging
- 3. Data Extraction Pull Mode
- 4. Data Extraction Push Mode
- 5. Staging Layer - Store and Forward ELT mode Load into staging layer first then transform Data is transformed and the records in stage are updated with the transformed values Data then can be loaded into data warehouse or data mart
- 6. Staging Layer - Direct load In this method you avoid the data staging layer Data can be extracted and directly loaded into data warehouse layer In can be used where no transformation is needed
- 7. Data Load – Full Data All the data from the source is loaded into staging tables Then compare and load into data warehouse layer When no change data capture technique is available on the source, most of the time you end up using this method
- 8. Data Load – Changes only Only newly added records or changed records are processed in this method This can be used when changed data capture is defined or can be defined on the source
- 9. Data Stores There are two ways one can store data in staging layer ◦ File ◦ Table