![THANK YOU Q & A [email_address]](https://image.slidesharecdn.com/phdprogress14092008-124343275047-phpapp02/85/Ph-D-Progress-14-09-2008-27-320.jpg)
Ph D Progress 14 09 2008
- 1. GOAL-ORIENTED AND ONTOLOGY-DRIVEN REQUIREMENT ANALYSIS METHOD FOR EXTRACTION-TRANSFORMATION-LOADING (ETL) PROCESSES IN DATA WAREHOUSE SYSTEM By AZMAN TA’A (91161) SUPERVISORS Assoc. Prof. Dr. Norita Md. Norwawi Dr. Mohd Syazwan Abdullah COLLEGE OF ART AND SCIENCES September 14, 2008
- 2. OUTLINE INTRODUCTION PROBLEM STATEMENT REQUIREMENT ANALYSIS FOR DW REQUIREMENT ANALYSIS FOR ETL PROCESSES REQUIREMENT ANALYSIS APPROACH TRANSFORMATIONAL ANALYSIS ONTOLOGY MODELING EXAMPLE: CASE STUDY AT UUM CONCLUSION
- 3. INTRODUCTION ETL consumes 70 – 80 % of DW development resources Success of ETL is depending on data integration and transformation that deal with semantic reconciliation
- 4. INTRODUCTION Problem in design and development of ETL processes: Complexity and hugeness of DW (Vassiliadis, 2000; Kimball & Caserta, 2004) Inefficiency of data loading (Chaudhuri & Dayal, 1997; Kimball & Ross, 2002) Data integration and transformation process (Rahm & Do, 2000; Halevy, 2005) Generating the data transformation mechanism (Kimball & Caserta, 2004; Alexiev et al., 2005)
- 5. PROBLEM STATEMENT Problems in ETL processes can be classified into: 1) define and maintain the ETL specifications 2) handle semantic heterogeneity problems ETL specification is an application-driven rather than data-driven – cause error-prone to maintain and platform dependent fashion Semantic heterogeneity is the problems of data conflicts in integration and transformation However, the design of DW system should be based on proper requirement engineering process (Kimball & Caserta, 2004; Simitsis, 2004; Lujan-Mora, 2005; Rizzi et al., 2006)
- 6. PROBLEM STATEMENT Several efforts on ETL modeling and data integration approach have been suggested. However, these efforts not focus in resolving the heterogeneity problems at modeling level, which related to the user requirements. User requirements does not properly guiding the developer to design the ETL processes due to unmanageable the user interpretation in business requirements The ETL designer need a proper method to design the ETL processes with consideration of data heterogeneity problems, and generation of ETL specifications from the early phase of DW development
- 7. REQUIREMENT ANALYSIS FOR DW Requirement Perspective of DW
- 8. REQUIREMENT ANALYSIS FOR DW DW requirements aim to identify the decisional information Generally, DW requirements approach: 1) process-driven (Kimball, 1996) 2) supply/data-driven (Inmon, 2002) 3) demand/user-driven (Winter & Strauch, 2003) DW requirements focus only on the data or information-centric – both supply and demand are relevant to adopt Moreover, demand/supply-driven compliment each other to support complex requirements
- 9. REQUIREMENT ANALYSIS FOR ETL PROCESSES Transformation of informal statements of user requirements into formal expression of ETL specification. User requirements elicited and analyzed from the organization and decision-maker perspective. These requirements will be mapped with the available data sources through ETL processes, which is should be derived from transformation analysis Thus, proper and systematic transformation analysis is required in the early phase of DW development
- 10. REQUIREMENT ANALYSIS FOR ETL PROCESSES General Requirement Analysis Model (Adapted from Prakash and Gosain (2003))
- 11. REQUIREMENT ANALYSIS FOR ETL PROCESSES Requirement analysis approach centered on the organizational and decisional modeling, and focus on transformation analysis. Organization modeling – to identify goal that related to DW components such as facts and attributes of organization – as is analysis Decision modeling – to identify goal that related to decision maker to DW components such as facts, dimension, measures, and transformation – to be analysis Requirement analysis method is based on Goal-Oriented methodology
- 12. REQUIREMENT ANALYSIS FOR ETL PROCESSES Detail Requirement Analysis Model (Adapted from Giorgini et al . (2008))
- 13. REQUIREMENT ANALYSIS APPROACH The approach using Tropos methodology, which based on i* conceptual framework of software development (Yu, 1995; Bresciani et al ., 2003) Founded on the agent-oriented software development methodogy, which using agent and related mentalistic notion in all phases of software development Importantly, Tropos support an early requirement analysis to the implementation, which essentially explain how the intended system (i.e. DW system) will meet organization goals.
- 14. REQUIREMENT ANALYSIS APPROACH The Tropos methodology is consists of five main phases: early requirements, late requirements, architectural design, detailed design, and implementation. Introduce the concepts of Actor, Goal, Plan, Resource, Dependency, Capability, Belief. Modeling activities are Actor modeling, Dependency modeling, Goal modeling, Plan modeling, Capability modeling Apply three basic reasoning techniques: means-end analysis, contribution analysis, and AND/OR decomposition
- 15. TRANSFORMATIONAL ANALYSIS Is not supported in previous requirement analysis approach. However, the analysis should perform from an early phase of requirement engineering to ensure meeting the organization goal. In Tropos, transformation analysis deal with Plan modeling , which is compliment to the decision-goal modeling. Plan modeling will determine set of transformations and constraint activities as required.
- 16. ONTOLOGY MODELING Ontology approach is used to model two sources of information prior to the conceptual design of ETL processes. First source: modeling the glossaries of DW terms (i.e. facts, attributes, dimensions, measures, actions, constraints) produced by requirement analysis process Second source: modeling the data sources that related to subject area or application of DW system. The tasks of ontology construction, mapping, ETL specification construction will establish before and during the conceptual design of ETL processes.
- 17. EXAMPLE: CASE STUDY IN UUM University Goals
- 18. EXAMPLE: CASE STUDY IN UUM Actor Diagram for UUM example
- 19. EXAMPLE: CASE STUDY IN UUM Rationale Diagram for UUM actor from organizational perspective
- 20. EXAMPLE: CASE STUDY IN UUM Extended Rationale Diagram for UUM actor from organizational perspective
- 21. EXAMPLE: CASE STUDY IN UUM Rationale Diagram for AAD Director from decisional perspective
- 22. EXAMPLE: CASE STUDY IN UUM Extended Rationale Diagram for AAD Director
- 23. CONCEPTUAL MODEL OF ETL PROCESSES
- 24. LOGICAL MODEL OF ETL PROCESSES
- 25. CONCLUSION The aim of this study is to model and design the ETL processes in DW system from requirement analysis tasks to ETL processes implementation. The use of Goal-Oriented approach to analyst user requirements and Ontology-based approach to model requirement glossaries and data sources hoping to resolve these problems mentioned. Works still in progress - concentrate on UUM case study to pre-confirm the model and methods proposed. Next step is to implement the solutions in different domain and bigger scale.
- 26. PAPERS SAS Forum Malaysia – Modeling BI in Academic Information Portal, SAS Kuala Lumpur, 5 S eptember 2007. SUGI08 – Academic Business Intelligence Development Using SAS Tools , San Antonio, Texas, USA. 15-19 March 2008 – Paper accepted and selected as SAS Student Ambassador. Camp08 – Ontology-Based Extraction-Transformation-Loading (ETL) Processes Model in Data Warehouse Environments, Kuala Lumpur. 18 March, 2008 – Paper submitted.
- 27. THANK YOU Q & A [email_address]