datamining-introduction.pdf

Data Mining : Lecture One Dr. Ahmed Alnasheri 1
DATA MINING
INTRODUCTION
What is data mining?
Applications and techniques

• This course has been designed to give students an introduction to
data mining and hands on experience with all phases of the data
mining process using real data and modern tools. It covers many
topics such as data formats, and cleaning; make prediction using
supervised and unsupervised learning using Python and other
tools, and sound evaluation methods; and data/knowledge
visualization.
Course Description

• Providing the fundamental understanding of data mining in order to extract
hidden knowledge.
• Exploring the different data mining tasks to extract knowledge:
• Classification,
• Clustering,
• Association Rules extraction, and
• Outlier detection.
• Practicing the data mining project phases
• Presenting the data in the early stage of data mining projects as well as the
extracted knowledge.
• Provide the students the latest hot topics in data mining field.
• Strengthen the team work
Course Objectives

• Why Data Mining?
• What is Data Mining ?
• Knowledge Data Discovery (KDD) Process
• Data Mining Task
• Data Mining Function.
• Data Creates Values
Outline

“Data is the new oil. It’s valuable, but
if unrefined it cannot really be used.
It has to be changed into gas, plastic,
chemicals, etc to create a valuable
entity that drives profitable activity;
so must data be broken down,
analyzed for it to have value.”

• The Explosive Growth of Data: from terabytes to petabytes
• Data collection and data availability
• Automated data collection tools, database systems, Web, computerized society.
• Major sources of abundant data
• Business: Web, e-commerce, transactions, stocks, …
• Science: Remote sensing, bioinformatics, scientific simulation, …
• Society and everyone: news, digital cameras, YouTube
• We are drowning in data, but starving for knowledge!
• “Necessity is the mother of invention”
• Data mining → Automated analysis of massive data sets
Why Data mining?

Why data mining? “Benefits of Data Mining”
• Scientific point of view
• Scientists are at an unprecedented position where they can collect TB (tuberculosis ) of information
• Examples: Sensor data, astronomy data, social network data, gene data
• We need the tools to analyze such data to get a better understanding of the world and
advance science and help people
• Commercial point of view
• Data has become the key competitive advantage of companies
• Examples: Facebook, Google, Amazon
• Being able to extract useful information out of the data is key for exploiting them
commercially.
• Scale (in data size and feature dimension)
• Why not use traditional analytic methods?
• Enormity of data, curse of dimensionality
• The amount and the complexity of data does not allow for manual processing of the
data. We need automated techniques.

Why Data mining?
• Every human, physical, or machine activity
generates data.
• Transaction data in stores, credit cards
• Scientific measurements
• DNA sequences, gene co-expression
• Health records, brain images, daily measurements
• The Web, Wikipedia, Facebook posts, Tweets, Online
Reviews
• Queries to Google, Clicks, Browsing behavior, Ads
• Facebook likes and comments, Twitter retweets
• The Web graph, Facebook friends, Twitter followers
• Movement data, Trajectories,
• Mobile use, telephone calls
• Wearable devices
• Machine and workflow monitoring
• Everybody collects data!

• Data mining is an interdisciplinary field
Why Data mining?
Data mining is an
interdisciplinary subfield
of computer science and
statistics with an overall
goal to extract information
(with intelligent methods)
from a data set and
transform the information
into a comprehensible
structure for further use.
What are the
origins of data
mining?

Data Mining: Confluence of Multiple Disciplines
Data Mining
Database
Technology Statistics
Machine
Learning
Pattern
Recognition
Algorithms
Other
Disciplines
Visualization

Data Mining
Database
Machine
Learning
Pattern
Recognition
Algorithms
Other
Disciplines
Visualization

Data Mining
Database
Machine
Learning
Pattern
Recognition
Algorithms
Distributed
Computing
Visualization

• After years of data mining there is still no unique answer to this question
• Data Mining Is a knowledge discovery from data
• Extraction of interest (Non-trivial, Implicit, Previously unknown, and Potentially useful) pattern
or knowledge from huge amount of data.
• A tentative definition:
• Data mining is the use of efficient techniques for the analysis of very large collections of data and the extraction
of useful and possibly unexpected patterns in data.
What is data mining ?

• Data Mining is:
• (1) The efficient discovery of previously unknown, valid, potentially useful,
understandable patterns in large datasets.
• (2) The analysis of (often large) observational data sets to find unsuspected relationships
and to summarize the data in novel ways that are both understandable and useful to the
data owner

• What is the difference between data mining and Database Query?
We know what exactly we want We vaguely know what we are looking for

Data Mining
• In simple terms:
Data Data Mining Value

Knowledge Data Discovery (KDD) Process
• Data mining plays an essential role in the knowledge discovery process and
highly dependent on data

The Data Mining Process
1. Understand the domain
2. Create a dataset:
• Select the interesting attributes
• Data cleaning and preprocessing
3. Choose the data mining task and the specific algorithm
4. Interpret the results, and possibly return to 2

Why Data Preprocessing?
• Data in the real world is dirty
• incomplete: lacking attribute values, lacking certain attributes of interest .
• noisy: containing errors or outliers.
• inconsistent: containing discrepancies in codes or names
• No quality data, no quality mining results!
• Quality decisions must be based on quality data.
• Data warehouse needs consistent integration of quality data.
• Required for both OLAP and Data Mining!

Why can Data be Incomplete?
• Attributes of interest are not available (e.g., customer information
for sales transaction data)
• Data were not considered important at the time of transactions, so
they were not recorded!
• Data not recorder because of misunderstanding or malfunctions
• Data may have been recorded and later deleted!
• Missing/unknown values for some data

Data Cleaning
• Data cleaning tasks
• Fill in missing values
• Identify outliers and smooth out noisy data
• Correct inconsistent data

Types of data
• Numeric data: Each object is a point in a multidimensional space
• Categorical data: Each object is a vector of categorical values
• Set data: Each object is a set of values (with or without counts)
• Sets can also be represented as binary vectors, or vectors of counts
• Ordered sequences: Each object is an ordered sequence of values.
• Graph data

What can we do with data mining?
• Some examples:
• Frequent item sets and Association Rules extraction
• Clustering
• Classification
• Ranking
• Exploratory analysis

• You are the owner of a social network, and you have full access to
the social graph, what kind of information do you want to get out of
your graph?

• Suppose that you are the owner of a supermarket and you have
collected billions of market basket data. What information would
you extract from it and how would you use it?
• What if this was an online store?

• Suppose you are a search engine and you have a toolbar log consisting of
• Pages browsed,
• Queries,
• Pages clicked,
• Ads clicked
• Each with a user id and a timestamp. What information would you like to
get our of the data?

Data Set : High School Students’ Grade

Data Mining Task
• Two main types of data mining task
• Descriptive: Characterize properties of the data in a target data set.
• Predictive : Perform induction on the current data in order to predict values
of a new data.

Data Mining Task
1. Classification: learning a function that maps an item into one of a set of
predefined classes
2. Regression: learning a function that maps an item to a real value
3. Clustering: identify a set of groups of similar items
4. Dependencies and associations: identify significant dependencies between
data attributes
5. Summarization: find a compact description of the dataset or a subset of the
dataset

Data Mining Function.
• Data mining functionalities are used to specify the kind of patterns
to be found in data mining tasks.

The data is complex and interconnected
• Multiple types of data: database tables, text, time series, images,
videos, graphs, etc
• Spatial and temporal aspect
• Interconnected data of different types:
• From the mobile phone we can collect, location of the user, friendship
information, check-ins to venues, opinions through twitter, status updates in
FB(FaceBook), images though cameras, queries to search engines

Data creates value
Natural language
understanding is
driven by data

Data creates value
Precision/Personalized medicine:
Find the best treatment for patients
using their genotype and all data
that are related to them
Also: understanding drug side-
effects through google queries

Data creates value
Self-Driving Cars:
Car is the next computer. A
future of smart cars that can
drive themselves and learn
from data
Also: smart cities – urban
computing

Data creates value
Computers learn to play
games by observing data

Data creates value
Use of data for crisis management

Data creates value
• All major soccer and basketball teams use data mining to make
decisions.
The national team of Germany
had a special software for the
analysis of video.
They concluded that the
possession time per player
should be reduced.
Germany won the 2014 word cup

Data creates value
James Harden defense

Putting it all together: The Data Mining Pipeline (LinkedIn)
feature extraction
feature transformation
user modeling
Data Pipeline
data tracking & logging candidates generation
multi-pass rankers
Online Serving System
real-time feedback
Model Fitting Pipeline (Hadoop)
offline modeling fitting
(cold-start model)
nearline modeling fitting
(warm-start model)
daily/weekly minutes/hourly
online A/B test
model evaluation

Data Mining Example
• Suppose that you were creating the Yemeni Facebook.
• What kind of data would you collect and store?
Social network contacts
Interaction with contacts: messages, likes, replies, shares
Posts, content of posts
Interactions with feed: Clicks, Likes, Comments, Shares
Photos
Demographics: Age, City, etc
Ads seen, ads clicked
Products bought
Videos uploaded videos consumed
and many more!
What would you do with this data?

Exploratory Analysis
• Make measurements to understand what the
data looks like
• Example: Posts
• How often do users posts, how many posts per
user, when do they post, is there a correlation
between number of posts and number of friends,
etc
• This is one of the first steps when collecting
data.
• Metrics: Deciding what to measure is important
• The example of the Web graph

Exploiting similarities
• Consider the following data for six users:
• Number of times they have clicked on posts from these pages
• What conclusion can we draw?
NBA ESPN Sports.com MSNBC NY Times Wall Street Politico
A 100 50 73 10 1 1 4
B 500 200 400 20 10 4 1
C 80 100 60 1 3 1 1
D 4 2 1 12 90 100 80
E 9 3 4 9 100 80 70
F 3 4 5 30 300 200 500

• Two types of users and two types of pages
• Sports and politics
• Questions:
• How do we compute similarity?
• How do we group similar users? Clustering
A 100 50 73 10 1 1 4
B 500 200 400 20 10 4 1
C 80 100 60 1 3 1 1
D 4 2 1 12 90 100 80
E 9 3 4 9 100 80 70
F 3 4 5 30 300 200 500

• What if we were missing this entry?
• Can we fill this value?
• Similar users like items similarly: Recommendation systems
A 100 50 73 10 1 1 4
B 500 200 400 20 10 4 1
C 80 100 ??? 1 3 1 1
D 4 2 1 12 90 100 80
E 9 3 4 9 100 80 70
F 3 4 5 30 300 200 500

Amazon Recommendations
• “People who have bought this also bought…”
• A huge breakthrough for amazon
• Took advantage of the long tail
• A big breakthrough for data mining in general

Making predictions
• Filling the missing value can also be viewed as a prediction task
• Types of prediction tasks:
• Predicting a real value (e.g. number of clicks): Regression
• Predicting a YES/NO value (e.g., will the user click?): Binary classification
• Predicting over multiple classes (e.g., what is the topic of a post): Classification
• Can you think of prediction/classification tasks for your social
network?
Ad click prediction
Ad clickthrough prediction
Like prediction
Predict if a user will like a post over another:
Learning to rank
Predict if a post is offensive
Predict if a photo contains nudity

Classification
• Classification process:
• Find features that describe an entity.
• Use examples of the classes you want to predict.
• Learn a model (function) that predicts
• Classification is the engine behind the AI revolution
• Used in all systems that make decisions
• Became very powerful with Deep Learning
• Huge applications in vision
Tid Refund Marital
Status
Taxable
Income Cheat
1 Yes Single 125K No
2 No Married 100K No
3 No Single 70K No
4 Yes Married 120K No
5 No Divorced 95K Yes
6 No Married 60K No
7 Yes Divorced 220K No
8 No Single 85K Yes
9 No Married 75K No
10 No Single 90K Yes
10
Refund
MarSt
TaxInc
YES
NO
NO
NO
Yes No
Married
Single, Divorced
< 80K > 80K

Deep learning
• Machine learning systems that use neural networks with multiple
layers and are trained on very large quantities of data
• Able to learn complex representations and powerful models.
• Applications in recommendations, network analysis, text analysis, image
recognition, car driving, playing games…
• Require less feature engineering

The social graph
• Your Yemeni Facebook also has a social graph. What can you do
with this data?
What is the shortest path between two nodes?
Who is important and influential in the graph?
How does information spread in the network?
What becomes viral?
Will two users become friends in the future?

What is the most important node in
this graph? Degree Centrality and
Closeness Centrality
• The PageRank algorithm: A node is
important if it is pointed to by other
important nodes.
• The idea of the basic PageRank
algorithm is that the importance of a
node depends on the number and
importance of the neighbor nodes
pointing to it.
Node importance

The Web as a graph
• When ranking pages, the
authoritativeness is
factored in the ranking.
• This is the idea that made
Google a success around
2000
• Today a lot more
information is used, like
clicks, browsing behavior,
etc
• Ranking of the pages is a
very complex task that
requires sophisticated
techniques

Friendship suggestions
• LinkedIn, Twitter, Facebook friendship suggestions
• Useful for the users to discover their friends, but also useful for the network in order
to grow, and increase engagement
• LinkedIn success story
• Triadic closure principle: Links are created in a way that usually closes a
triangle
• If both Bob and Charlie know Alice, then they are likely to meet at some point.

What is Data Mining again?
• “Data mining is the analysis of (often large) observational data sets to
find unsuspected relationships and to summarize the data in novel ways
that are both understandable and useful to the data analyst” (Hand,
Mannila, Smyth)
• “Data mining is the discovery of models for data” (Rajaraman, Ullman)
• We can have the following types of models
• Models that explain the data (e.g., a single function)
• Models that predict the future data instances.
• Models that summarize the data
• Models the extract the most prominent features of the data.
• “Data Mining is the study of collecting, processing, analyzing, and
gaining useful insights from data” – Charu Aggarwal

The buzz around data
• Data Science: Data is useful to understand a process and improve it. All
organizations should have a data science team that analyses their data and
proposes improvements
• Focuses on more immediate applications and insights
• Big Data: Data appear everywhere. We should process it collectively and
interconnect them. We need infrastructure (cloud computing, cloud storage) to
do this
• More systems oriented
• AI/Machine Learning/Deep Learning: These have been around for a while but
now we have the data to learn more complex models that are significantly
more powerful
• More emphasis on scientific breakthroughs (Penetration)

New era of data mining
• Boundaries are becoming less clear
• Today data mining, machine learning, and AI are synonymous. It is assumed
that the algorithms should scale. It is clear that statistical inference is used
for building the models.
• Data is the engine for AI
• Data Mining touches everything related to data.

Which also has a dark side
• Are the algorithms making fair
and correct decisions?
• Do algorithms create filter
bubbles, echo chambers, and
promote misinformation? Are
they a threat to democracy?
• Surveillance capitalism
• Is AI a threat?

The Skills of a Data Miner – Data Scientist
It is a hard job

But also a rewarding one
"The success of companies
like Google, Facebook,
Amazon, and Netflix, not to
mention Wall Street firms and
industries from manufacturing
and retail to healthcare, is
increasingly driven by better
tools for extracting meaning
from very large quantities of
data. 'Data Scientist' is now
the hottest job title in Silicon
Valley." – Tim O'Reilly
Sexiest Job but…

Homework 1
• Read chapter one and answer the questions on page 34, and 35. By hand
• Data Scientist: The Sexiest Job of the 21st Century. PowerPoint Presentation.
• http://www.cse.msu.edu/~ptan/dmbook/software/

datamining-introduction.pdf

More Related Content

Similar to datamining-introduction.pdf (20)

Recently uploaded (20)

datamining-introduction.pdf