Planning Data Warehouse
Download as PPT, PDF6 likes7,765 views
The document discusses requirements gathering for data warehousing projects. It emphasizes that requirements for data warehousing are different than for operational systems, as data warehousing is meant to provide strategic information rather than capture data. While users may have trouble defining their exact needs, they can identify important business dimensions and measurements. Gathering requirements involves open-ended interviews with various stakeholders to understand objectives, issues, anticipated usage, and success metrics. Proper requirements form the basis for all subsequent development phases of the data warehouse.
![Planning & Project Management Fahri Firdausillah [M031010012]](https://image.slidesharecdn.com/presentdatawarehouse-110216105944-phpapp02/85/Planning-Data-Warehouse-1-320.jpg)
Planning Data Warehouse
- 1. Planning & Project Management Fahri Firdausillah [M031010012]
- 2. Joke First, Serious Later Consultant: So, your company is into data warehousing? How many data marts do you have? Project Manager: Eleven. Consultant: That’s great. But why so many? Project Manager: Ten mistakes.
- 3. Defining the Business Requirements Chapter 5
- 4. Preamble OLTP and DW planning is different in term of requirements clarity Planning DW is about solving users’ problems and providing strategic information to the user. OLTP systems are primarily data capture systems. On the other hand, data warehouse systems are information delivery systems. Unlike an OLTP system, which is needed to run the day-to-day business, no immediate payout is seen in a decision support system.
- 5. Dimensional Analysis the users are generally unable to define their requirements clearly. For most of the users, this could be the very first data warehouse. How can you build something the users are unable to define clearly and precisely? Need different approach of requirements gathering.
- 6. Dimensional Analysis (cont'd) They can tell you what measurement units are important for them, how they combine the various pieces of information for strategic decision making. Although the actual proposed usage of a data warehouse could be unclear, the business dimensions used by the managers for decision making are not nebulous at all
- 7. Dimensional Analysis “in Action”
- 8. More Complex Dimensional Model
- 9. Information Packages The business dimensions and their hierarchical levels form the basis for all further development phases. The dimension hierarchies are the paths for drilling down or rolling up in our analysis
- 10. Requirements Gathering Methods Types of Questions Open Ended Question These open up options for interviewees to respond Closed Question These allow limited responses to interviewees Probes These are really follow-up questions. Probes may be used after open-ended or closed questions Questionnaires Group Session Interview
- 11. Sample Expectation from Interviews Senior Executives Dept. Managers IT Dept. Professionals Organization objectives Criteria for measuring success Key business issues, current & future Problem identification Vision and direction for the organization Anticipated usage of the DW Departmental objectives Success metrics Factors limiting success Key business issues Products & Services Useful business dimensions for analysis Anticipated usage of the DW Key operational source systems Current information delivery processes Types of routine analysis Known quality issues Current IT support for information requests Concerns about proposed DW
- 12. Adapting JAD Identify project objectives and limitations Identify critical success factors Define project deliverables Define the schedule of workshop activities Select the participants Prepare the workshop material Organize workshop activities and exercises Prepare, inform, educate the workshop participants Coordinate workshop logistics 1. Project Definition 2. Research 3. Preparation 4. JAD Sessions 5. Final Document
- 13. Requirement Definition: Scope & Content Requirements definition document is the basis for the next phases. Formal documentation will also validate your findings when reviewed with the users Data Sources Data Transformation Data Storage Information Delivery Information Package Diagrams
- 14. Requirements Definition Document Outline Introduction General Requirements Descriptions Specific Requirements Information Packages Other Requirements User Expectations User Participation and Sign-Off General Implementation Plan
- 15. Requirements as the Driving Force for Data Warehousing Chapter 6
- 16. Preamble If accurate requirements definition is important for any operational system, it is many times more important for a data warehouse extremely important that your datawarehouse contains the right elements of information in the most optimal formats Every task that is performed in every phase in the development of the data warehouse is determined by the requirements Every decision made during the design phase is totally influenced by the requirements.
- 17. Data Design
- 18. Data Design (cont'd) Structure for Business Dimensions Importance of having the appropriate dimensions and the right contents in the information package diagrams . Structure for Key Measurements Users measure performance by using and comparing key measurements In order to review using proper key measurements, DW has to guarantee the information package diagrams contain all the relevant keys. Levels of Detail DW needs to provide drill-down and roll-up facilities for analysis How deep detail of data is needed in DW
- 19. Data Design “in Action” Structure for Business Dimensions Structure for Key Measurements Levels of Detail
- 21. Source Data Production Data: Data get from operational system. Normally include financial system, customer relationship system, manufacturing system, etc. Internal Data: Private data keep by internal organization. Could be spreadsheets, documents, even departmental database Archived Data: Old data that is already not to be used in operational system. External Data: Data from outside systems, it can also from outside company. This type of data usually do not conform internal format
- 22. Data Staging Bad data lead to bad decision, data quality in data warehouse is sacrosanct ETL process ensure data to be ready stored and processed in DW. In many cases data need to be extracted from sources in different scheme, different vendor, even in different format of flat files. If data extraction for a DW poses great challenges, data transformation presents even greater challenges. Data need to be cleaned from misspelling, resolution conflict, duplication, setting default missing values, etc. Initial load moves very large volumes of data. After that data staging will continuously extract the changes from sources. Extract Transform Load
- 24. Data Storage Specifications DBMS Selection User requirements affect the selection of the proper DBMS. Choice of the DBMS may be conditioned by its tool kit component. Features to be considered: Level of User Experience, Types of Queries, Need for Openness, Data Loads, Metadata Management, Data Repository Locations, Data Warehouse Growth. Storage Sizing Determined by how many data source and how much the data will grows continuously. If DW is expected to support Online Analytical Processing OLAP, then how much OLAP is necessary.
- 26. Metadata Operational Metadata: When deliver information to the end-users, you must be able to tie that back to the original source data sets. Operational metadata contain all of this information about theoperational data sources. Extraction and Transformation Metadata: Storing information of extraction frequencies, extraction methods, and business rules for the data extraction. End-User Metadata: Navigational map of the data warehouse, allows the end-users to use their own business terminology and look for information in those ways.
- 27. Management & Control Sits on top of all the other components. Controls the data transformation and the data transfer into the data warehouse storage. Interacts with the metadata component to perform the management and control functions. Metadata is the source of information for the management module.
- 28. End of Presentation & Thank You Very Much