bi notes.docx
- 1. Agenda O What Is The Need For BI? O What Is Data Warehousing? O Key Terminologies Related To DWH Architecture Q OLTP VS OLAP O ETL 0 Data Mart Metadata O DWH Architecture O Demo: Creating A DWH Why Business Intelligence? Business Intelligence is the activity which contributes to the growth of any company. What Is Business Intelligence? BI is the act of transforming raw/ operational data into useful information for business analysis. How Does It Work? BI based on Data Warehouse technology extracts information from a company's operational systems. The data is transformed (cleaned and integrated), and loaded into Data Warehouses. Since this data is credible, it is used for business insights. Why Data Warehouse? Data collected from various sources & stored in various databases cannot be directly visualized. The data first needs to be integrated and then processed before visualization takes place.
- 2. What Is A Data Warehouse? A central location where consolidated data from multiple locations (databases) are stored. DWH is maintained separately from an organization's operational database. End users access it whenever any information is needed. Note:- Data Warehouse is not loaded every time new data is added to database. What Are The Advantages Of A Data Warehouse? > Strategic questions can be answered by studying trends. Data Warehousing is faster and more accurate. Note:- Data Warehouse is not a product that a company can go and purchase, it needs to be designed & depends entirely on the company's requirement. PropertiesOfADataWarehouse "A Data Warehouse is a subject-oriented, integrated, time-variant and nonvolatile collection of data in support of management's decision-making process." -Bill Inmon, Father of Data Warehousing Subject-oriented Data is categorized and stored by business subject rather than by application. Data on a given subject is collected from disparate sources and stored in a single place. Time-variant Data isstored as a series of snapshots, each representing a period of time. Non-volatile Typically data in the data warehouse is not updated or deleted. Information Systems:- OLTP (DB) vs. OLAP (DWH)
- 3. Relational Database (OLTP) Analytical Data Warehouse (OLAP) Contains current data Contains historical data Useful in running the business Useful in analyzing the business Based on Entity Relationship Model Based on Star, Snowflake and Fact Constellation Schema Provides primitive and highly detailed data Provides summarized and consolidated data Used for writing data into the database Used for reading data from the data warehouse Database size ranges from 100 MB to 1 GB ta Warehouse size ranges from 100 GB to 1 TB Fast; provides high performance Highly flexible; but not fast Number of records accessed is in tens Number of records accessed is in millions Ex: All bank transactions made by a customer Ex: Bank transactions made by a customer at a particular time. InformationSystems:- OLTP(DB) vs. OLAP (DWH) OLTP Examples: A supermarket server which records every single product purchased at that market. A bank server which records every time a transaction is made for a particular account. A railway reservation server which records the transactions of a passenger. OLAP Examples: Bank Manager wants to know how many customers are utilizing the ATM of his branch. Based on this he may take a call whether to continue with the ATM or relocate it. An insurance company wants to know the number of policies each agent has sold. This will help in better performance management of agents. ETL is the process of extracting the data from various sources, transforming this data to meet your requirement and then loading it into a target data warehouse.
- 4. Data Mart > Data mart is a smaller version of the Data Warehouse which deals with a single subject > Data marts are focused on one area. Hence, they draw data from a limited number of sources > Time taken to build Data Marts is very less compared to the time taken to build a Data Warehouse Data Mart Data mart is a smaller version of the Data Warehouse which deals with a single subject Data marts are focused on one area. Hence, they draw data from a limited number of sources Time taken to build Data Marts is very less compared to the time taken to build a Data Warehouse 2 3 Data Warehouse Data Mart 1 Data Mart Data Mart Data Warehouse Data Maro Enterprise wide data Department wide data Sales Data Multiple subject areas Single subject area Multiple data sources Limited data sources Occupies large memory Occupies limited memory Longer time to implement Shorter time to implement Operations Data Information Systems:- OLTP (DB) vs.OLAP (DWH) OLTP Examples: A supermarket server which records every single product purchased at that market. A bank server which records every time a transaction is made for a particular account. A railway reservation server which records the transactions of a passenger. OLAP Examples: Bank Manager wants to know how many customers are utilizing the ATM of his branch. Based on this he may take a call whether to continue with the ATM or relocate it.
- 5. An insurance company wants to know the number of policies each agent has sold. This will help in better performance management of agents.