ETL Validator Usecase - Data Profiling and Comparison
Download as PPTX, PDF0 likes266 views
The document outlines the use cases for an ETL validator focused on data profiling, emphasizing comprehensive testing automation with benefits like 100% test coverage and faster time to market. It details steps for creating and executing test plans, including SQL query configuration and data comparison between source and target tables. Key functionalities include automatic query generation, field selection, and exporting results to Excel or web formats for reporting.
ETL Validator Usecase - Data Profiling and Comparison
- 1. ETL Validator Usecases: Data Profiling (Counts, Missing Values, Averages etc) www.datagaps.com
- 2. ETL Validator Comprehensive Testing Automation Extract. Load. Validate (Patented)
- 3. 100% Test Coverage Repeatability Cost Reduction Faster Time to Market End to End Testing ETL Validator Key Benefits
- 4. Usecase : Create a test case: Profile data in source and target tables. Start with creating a new Data Profile Test Plan Data Profile
- 5. Usecase: Name the test plan. Select the Target and Source Database Connection. Navigate to the next screen. Data Profile
- 6. Usecase: SQL for query can be typed into Target Query and Source Query areas. OR Use Query Builder. Data Profile
- 7. Usecase: Query Builder opens up. Select the Target and Source Data Sources. Drag and drop the tables in their respective panes. Data Profile
- 8. Usecase: When the tables are dragged into the Query Columns panes, all the columns appear. Click on ‘Save’. Drag & Drop Duplicates Data Profile
- 9. Usecase: The Target and Source SQL queries are auto generated. They can be modified if needed. Click on ‘Execute Query’ to check if the data is being fetched properly. Navigate to the next screen. Data Profile
- 10. Usecase: In order to run test cases of only the ‘Customers’ table – • Click on settings icon • Unselect other tables. Save the settings. Click on ‘Run’ Data Profile
- 11. Usecase: The list of fields from both tables is displayed. In the first column, select/de- select fields as per requirement. Source table fields should correspond with that of target fields. In case they are off, select the right one from the dop-down. Select the required aggregate functions and variance. Navigate to next screen. Data Profile
- 12. Usecase: Click on ‘Run’. Data Profile
- 13. Usecase: ‘Overall Status’ = ‘Failure’ indicates that at least one aggregator didn’t match for each of those data fields. Individual aggregators like ‘Count’, ‘Missing Fields’ etc will show ‘Failure’ in case they didn’t match. This data can be exported in Excel. The report can also be viewed in browser. Data Profile Export to Excel Report in Browser
- 14. Usecase: Same info is displayed in web layout. The link can be shared with others. Data Profile
- 15. More with ETL Validator…. • Validating Field and Data Format • Data counts validation with allowed variance • Check Data Quality using Data Rules Test Plan • Advanced ETL Testing using a Component Test Case • Avoiding inline views on your queries in ETL Validator • Checking for Mandatory Fields • Data Profiling of Source and Target www.datagaps.com