Data mining
Download as PPT, PDF1 like581 views
Data mining is the process of analyzing large databases to discover useful patterns. It involves applying computer-based methods to derive knowledge from large amounts of data. The main components of data mining are knowledge discovery, where concrete information is gleaned from known data, and knowledge prediction, which uses known data to forecast future trends. Data is collected and stored in a centralized data warehouse to allow for easier querying. Common data mining techniques include classification, clustering, regression, and association rule mining. Data mining has various applications in areas such as business, science, medicine, and more to gain useful insights from data. However, effective data mining requires linking multiple data sources which can raise privacy concerns if a person's entire data history is assembled.
Data mining
- 1. DATA MINING
- 2. What is Data Mining? •New buzzword, old idea. •“The process of semi automatically analyzing large databases to find useful patterns” (Silberschatz) •KDD – “Knowledge Discovery in Databases” •Inferring new information from already collected data. •Areas of Use : Internet – Discover needs of customers Economics – Predict stock prices Science – Predict environmental change Medicine – Match patients with similar problems cure
- 3. Data Mining –Main Components Wikipedia definition : “Data mining is the entire process of applying computer-based methodology, including new techniques for knowledge discovery, from data.” Knowledge Discovery Concrete information gleaned from known data. Data you may not have known, but which is supported by recorded facts. Knowledge Prediction Uses known data to forecast future trends, events, etc Wikipedia note: "some data mining systems such as neural networks are inherently geared towards prediction and pattern recognition, rather than knowledge discovery.“ These include applications in AI and Symbol analysis
- 4. Data Warehouse: “is a repository (or archive) of information gathered from multiple sources, stored under a unified schema, at a single site.” (Silberschatz) Collect data Store in single repository Allows for easier query development as a single repository can be queried. Data Mining: Analyzing databases or Data Warehouses to discover patterns about the data to gain knowledge. Data Mining & Data Warehousing
- 6. Classification •Classification: Given a set of items that have several classes, and given the past instances (training instances) with their associated class, Classification is the process of predicting the class of a new item. •Therefore to classify the new item and identify to which class it belongs •Example: A bank wants to classify its Home Loan Customers into groups according to their response to bank advertisements. The bank might use the classifications “Responds Rarely, Responds Sometimes, Responds Frequently”. The bank will then attempt to find rules about the customers that respond Frequently and Sometimes. The rules could be used to predict needs of potential customers.
- 7. Clustering “Clustering algorithms find groups of items that are similar. … It divides a data set so that records with similar content are in the same group, and groups are as different as possible from each other. ” Example: Insurance company could use clustering to group clients by their age, location and types of insurance purchased. The categories are unspecified and this is referred to as ‘unsupervised learning’
- 8. Regression “Regression deals with the prediction of a value, rather than a class Example: Find out if there is a relationship between smoking patients and cancer related illness. Given values: X1, X2... Xn Objective predict variable Y One way is to predict coefficients a0, a1, a2 Y = a0 + a1X1 + a2X2 + … anXn Linear Regression .
- 9. Regression Example graph: Line of Best Fit Curve Fitting .
- 10. Association Rules An association algorithm creates rules that describe how often events have occurred together.” Example: When a customer buys a hammer, then 90% of the time they will buy nails.
- 11. Uses of Data Mining AI/Machine Learning Combinatorial/Game Data Mining Good for analyzing winning strategies to games, and thus developing intelligent AI opponents. (ie: Chess) Business Strategies Market Basket Analysis Identify customer demographics, preferences, and purchasing patterns. Risk Analysis Product Defect Analysis Analyze product defect rates for given plants and predict possible complications (read: lawsuits) down the line.
- 12. Uses of Data Mining (Cont.) Sales/ Marketing Diversify target market Identify clients needs to increase response rates Fraud Detection Identify people misusing the system. E.g. People who have two Social Security Numbers Customer Care Identify customers likely to change providers Identify customer needs
- 13. Sources of Data for Mining •Databases •Text Documents •Computer Simulations •Social Networks
- 14. Privacy Concerns •Effective Data Mining requires large sources of data •To achieve a wide spectrum of data, link multiple data sources •Linking sources leads can be problematic for privacy as follows: If the following histories of a customer were linked: •Shopping History •Credit History •Bank History •Employment History •The users life story can be painted from the collected data
- 15. THANK YOU