Creating Data Warehouse Using Power Query & Power Pivot
- 1. CREATING DATA WAREHOUSE USING POWER QUERY & POWER PIVOT DATA WAREHOUSE OF MEANINGFUL INFORMATION ON ROAD ACCIDENTS Author: Anthony Mok Date: 18 Nov 2023 Email: xxiaohao@yahoo.com
- 2. WHAT IS POWER QUERY & POWER PIVOT Excel Power Query A data transformation and preparation tool that simplifies the process of importing, shaping, and analysing data from various sources Excel Power Pivot A data modeling add-in for Excel that enables us to create sophisticated data models, analyse large datasets, and perform complex calculations
- 3. PROJECT’S CONTEXT, OBJECTIVE & STRATEGIES Context Social Enterprise, from a neighboring country which provides ambulatory services, has collected data on road accidents and is keen to use the data to inform on its resource deployment. It has stored the data into three files: ‘Accidents.xlsx’, ‘Casualties.xlsx’ and ‘Vehicles.txt’ Objective Create a data warehouse containing meaningful information on road accidents Strategies Import file and transform data Create queries as a new table Merge these tables Summary table Power Pivot and create a data model
- 4. IMPORT 1ST FILE, TRANSFORM DATA & CREATE 1ST POWER QUERY The file: ‘data warehouse on road accidents.xlsx’, was created in MS Excel, and the ‘Accidents0515 sheet’ from the ‘Accidents.xlsx’ file was imported into this data warehouse.xlsx file to conduct data transformation In the ‘Day_of_Week’ column, in the ‘Accidents0515 sheet’, records with value of -1 were excluded since this is an impossible value Format of the ‘Date’ column was changed to text since this is not a numerical data In the ‘Speed_limit’ column, only records that are 40 and above were selected When loading the ‘Accidents0515’ sheet into Power Pivot, ‘Only Create Connection’ option was chosen since this would be combined with the 2nd Power Query to form a new query
- 5. IMPORT 2ND FILE, TRANSFORM DATA & CREATE 2ND POWER QUERY ▪ The ‘Vehicles.xlsx’ file was imported into the MS Excel file: ‘data warehouse on road accidents.xlsx’ ▪ Records with value of -1 in the ‘Age_of_Vehicle’ and ‘Age_of_Driver’ columns were excluded since this is an impossible value When loading the ‘Vehicles0515’ sheet into Power Pivot, ‘Only Create Connection’ option was chosen since this would be combined with the 1st Power Query already created to form a new combined query
- 6. MERGE TWO QUERIES TO CREATE NEW COMBINED POWER QUERY ▪ The ‘Accident0515’ and ‘Vehicle0515’ queries were merged to create the new ‘Vehicles with Accident History’ query ▪ The ‘Accident_index’ was used as the key to merge these two queries ▪ When loading the ‘Vehicles with Accident History’ table, ‘Only Create Connection’ option was chosen since there is one more Power Query to create before loading these 3 queries into Power Pivot as a data model
- 7. PREPARE SUMMARY TABLE TO CREATE FINAL POWER QUERY ▪ The ‘Casualties.xlsx’ file was imported into the MS Excel file: ‘data warehouse on road accidents.xlsx’ ▪ The ‘Summary Table of Total Pedestrian Cases By Casualty Severity Types’ was prepared, through Power Pivot, with the following columns: ▪ Total Number of Accidents ▪ Average Age of Casualty (using the variable: ‘Age_of_Casualty’) ▪ Total Pedestrian Cases (using the variable: ‘Pedestrian Indicator’) for each ‘Casualty_Severity’ type (using the variable: ‘Casualty_Severity’) ▪ When loading the “Casualties0515” sheet, ‘Only Create Connection’ option was chosen since this is the final Power Query to create before loading this and two previously created queries into Power Pivot as a data model
- 8. CREATE DATA MODEL FROM 3 POWER QUERIES Loading Power Queries into Power Pivot as Data Model The ‘Accidents 0515’, ‘vehicles05151 and ‘causalities0515’ Power Queries were loaded into Power Pivot as a data model Excel Power Pivot To complete the data modelling process, at the Diagram View module in the Power Pivot workspace, the four tables were linked, using the: ‘Accident_index’ key for the ‘Accidents 0515’, ‘vehicles0515’ and ‘causalities0515’ tables, ‘Vehicle_Type’ key for the ‘vehicles0515’ and ‘Vehicle_Description’ tables (the “F1” key in the ‘Vehicle_Description’ table is the ‘Vehicle_Type’ key)
- 9. CREATING DATA WAREHOUSE USING POWER QUERY & POWER PIVOT DATA WAREHOUSE OF MEANINGFUL INFORMATION ON ROAD ACCIDENTS Author: Anthony Mok Date: 18 Nov 2023 Email: xxiaohao@yahoo.com