ETL Validator Usecase - Testing Transformations or Derived fields
Download as PPTX, PDF0 likes230 views
The document outlines the ETL Validator's use cases for testing derived fields, emphasizing the process of creating component test cases for validation. It describes steps to set up data sources, build SQL queries, establish mappings, and verify data consistency between source and target tables. Additionally, it highlights the benefits of ETL validation such as 100% test coverage, repeatability, and cost reduction.
ETL Validator Usecase - Testing Transformations or Derived fields
- 1. ETL Validator Usecases: Testing Derived Fields (Using Component Test Case) www.datagaps.com
- 2. ETL Validator Comprehensive Testing Automation Extract. Load. Validate (Patented)
- 3. 100% Test Coverage Repeatability Cost Reduction Faster Time to Market End to End Testing ETL Validator Key Benefits
- 4. Usecase : Testing Derived Fields Create a test case: Src_Customers table has ‘First Name’ and ‘Last Name’ fields. Customers_Derived has ‘Full Name’ field. Identify records where ‘First Name’ + ‘Last Name’ from former doesn’t match ‘Full Name’ of the latter. Start with creating a new Component Test Case
- 5. Usecase: Name the test case. All the other values remain same. Navigate to Next Screen. Testing Derived Fields
- 6. Usecase: In the Design Area, drag and drop the ‘Data Source Component’ . Name the Data Source. Select the DB Schema it belongs to. Click on ‘Apply’ Testing Derived Fields Data Source Component
- 7. Usecase: SQL Query of the DB Component opens. Click on ‘Build Query’ in the far right corner. ‘Rule Builder’ window opens the list of tables. Select ‘src_customers’ Testing Derived Fields
- 8. Usecase: Drag and drop the table ‘src_customers’ into ‘Query Columns’ window. All the fields are displayed. To look at the data, click on ‘Build Query’. ‘Results’ window displays the data in grid below. SQL window displays the sql generated. Save Query. Testing Derived Fields
- 9. Usecase: We come back to the ‘SQL Query’ window. Add the transformation – cust_first_name || ' ' || cust_last_name as full_name Click on ‘Save’. If a message box comes up asking for confirmation, click ‘OK’. Click on ‘BACK TO DESIGN’. Testing Derived Fields
- 10. Usecase: We come back to design area. Similar to adding Source Table, drag and drop another Data Source component and that will be the target table. Configure it to ‘Customers_Derived’ table. (Refer to earlier slides for the steps) Connect the arrows to Data Sources from ‘Start’ button. Testing Derived Fields
- 11. Usecase: After building the query in ‘Rule Builder’ window, we come back to the SQL Query window. Please move the ‘customers_derived.full_na me’ field to be the 4th field in the list. Please note that the order of the fields in Source and Target tables needs to be same. If there are extra fields in Target tables, they are ignored. Testing Derived Fields Extra field is ignored while matching
- 12. Usecase: We come back to design area. Drag and drop the Mapping component. Give it a name. Click on Apply. Testing Derived Fields
- 13. Usecase: Complete the arrows and double click on the Mapping component. Testing Derived Fields
- 14. Usecase: Mapping Component window opens. Click on ‘+’ next to ‘Table Validation. Dataset A and Dataset B are filled in with the tables from Workschema. These tables contain the data corresponding to the data sources chosen. Navigate to Next Screen to set the mapping. Testing Derived Fields
- 15. Usecase: Dataset A and Dataset B field list opens. Select the ‘Cust_id’ field and click in the small square to specify this as ‘Join’ condition. Notice that the ‘full_name’ fields are in the same order. Click on ‘Save’ Click on ‘BACK TO DESIGN’ in the top right corner. Testing Derived Fields
- 16. Usecase: Click on ‘Save’. Status changes from ‘Not Verified’ in red to ‘Verified’ in green. Navigate to Next Screen. Testing Derived Fields
- 17. Usecase: Click on the ‘Run’ Testing Derived Fields
- 18. Usecase: Results of the execution are displayed. Click on ‘Table Validation’ to see the details. Click on ‘View Report in Browser’ to see the Web layout of the report. (Refer to last slide) Note: ‘Run’ button at the bottom of the screen can be used for re-running the test case. Testing Derived Fields
- 19. Usecase: Click on arrow next to ‘Data differences…’ option to see the details. Testing Derived Fields
- 20. Usecase: Mismatched data in ‘Full Name’ is displayed. Also the other fields that have data differences are displayed. Testing Derived Fields
- 21. Usecase: Report in Browser: Same info is displayed in web layout. The link can be shared with others. Click on the upward arrow to see the records Testing Derived Fields
- 22. More with ETL Validator…. • Validating Field and Data Format • Data counts validation with allowed variance • Check Data Quality using Data Rules Test Plan • Advanced ETL Testing using a Component Test Case • Avoiding inline views on your queries in ETL Validator • Checking for Mandatory Fields www.datagaps.com