ETL Validator Usecase - Transformation logic in input data source
Download as PPTX, PDF1 like274 views
This document discusses using ETL Validator to test derived fields in target data by using transformation logic defined in source data. It provides step-by-step instructions to create a test case validating a 'cust_level' field derived in target based on logic in source. The test case executes the queries, identifies differences between target and transformed source data, and provides results that can be exported or viewed as a report. ETL Validator allows comprehensive testing of ETL processes through automation, repeatability, and validation of data across sources and targets.
ETL Validator Usecase - Transformation logic in input data source
- 1. ETL Validator Usecases: Test derived fields in output with transformation logic in input www.datagaps.com
- 2. ETL Validator Comprehensive Testing Automation Extract. Load. Validate (Patented)
- 3. 100% Test Coverage Repeatability Cost Reduction Faster Time to Market End to End Testing ETL Validator Key Benefits
- 4. Usecase : Transformation Logic Create a test case: Verify derived field ‘cust_level’ in Target data source using transformation logic in Source data source doesn’t have that field. Start with creating a new Query Compare Test Case
- 5. Usecase: Name the Test Case. Select the Target and Source Database Connection. Navigate to the next screen. Transformation Logic
- 6. Usecase: To add Target and Source tables, click on ‘Query Builder’. Select the Target table in left most pane and drag it to adjacent screen to add all the columns. Similarly, select the Source table in right most pane. ‘Save’ and ‘Close’ the ‘Query Builder’. Transformation Logic
- 7. Usecase: The SQL of Target and Source query is generated automatically. ‘cust_level’ in Target Query is a derived field. Transformation Logic
- 8. Usecase: Click on ‘Execute Query’ in Target Query pane to see the data. Based on the cust_income_level, the cust_level has been derived. Transformation Logic
- 9. Usecase: In Source Query, add the transformation logic to simulate the derived field. Click on ‘Execute Query’ in Source Query pane to see the data. cust_level has been added in the source. ‘Save’ the queries. Navigate to next screen. Transformation Logic
- 10. Usecase: Choose the columns that have to be displayed in the test results. By default, all columns are displayed. Select the field that joins the two data sources. In this case it is ‘cust_id’ Notice that the ‘cust_level’ is not listed in the Source columns. ‘full_name’ and ‘cust_level’ are extra fields in target. Transformation Logic
- 11. Usecase: To display the ‘cust_level’ in list of fields, click on ‘Remap Columns by Name’. When the ‘Confirm’ window pops up, click ‘OK’. Transformation Logic
- 12. Usecase: Now ‘cust_level’ in Target has a corresponding field in ‘Source. We have left out ‘full_name’ as it does not have a corresponding field in Source. Navigate to next screen. Transformation Logic
- 14. Usecase: Results of the execution are displayed. Click on downward arrows to see the details. Click on ‘View Report in Browser’ to see the Web layout of the report. (Refer to last slide) Note: ‘Run’ button at the bottom of the screen can be used for re-running the test case. Transformation Logic
- 15. Usecase: Click on arrow next to ‘Data differences…’ option to see the details. Data differences in ‘Full Name’ are displayed. Also the other fields that have data differences are displayed. Transformation Logic
- 16. Usecase: Matched data is displayed here. In the left pane, there is an option to Export the differences into Excel. Also there is information available on: Queries Run Durations Data Sources Transformation Logic
- 17. Usecase: Report in Browser: Same info is displayed in web layout. The link can be shared with others. Click on the upward arrow to see the records ‘Expand Source’ to see the source query etc Transformation Logic
- 18. More with ETL Validator…. • Validating Field and Data Format • Data counts validation with allowed variance • Check Data Quality using Data Rules Test Plan • Advanced ETL Testing using a Component Test Case • Avoiding inline views on your queries in ETL Validator • Checking for Mandatory Fields • List of Values Conformance Check www.datagaps.com