NUS-ISS Learning Day 2018- Pandas ate my data

#ISSlearn
#ISSlearn
PANDAS ATE MY DATA
13 July 2018 / Mr. Kenneth Phang
© 2018 National University of Singapore. All Rights Reserved

#ISSlearn
Agenda
Visualization
Bar chart, Histogram and
Pie Chart
Comparison with SQL
Introduction to Pandas Library
Data Structure - Series and
DataFrame
Data Analysis & Manipulation
MultiIndex, GroupBy, Merging,
Joining, Concentanation
2© 2018 National University of Singapore. All Rights Reserved

#ISSlearn
INTRODUCTION TO PANDAS LIBRARY

#ISSlearn
Agenda
Visualization
Bar chart, Histogram and
Pie Chart
Comparison with SQL
Introduction to Pandas Library
Data Structure - Series and
DataFrame
Data Analysis & Manipulation
GroupBy, Merging, Joining,
Concentanation

#ISSlearn
Series
Series is the data structure for a single column of a DataFrame,
not only conceptually, but literally i.e. the data in a DataFrame
is actually stored in memory as a collection of Series.

#ISSlearn
DataFrame
pandas.DataFrame(data=None, index=None, columns=None,
dtype=None, copy=False)
Two-dimensional size-mutable, potentially heterogeneous
tabular data structure with labeled axes (rows and columns).
Arithmetic operations align on both row and column labels. Can
be thought of as a dict-like container for Series objects. The
primary pandas data structure

#ISSlearn
SQL Comparison - SELECT
SQL
SELECT total_bill, tip, smoker, time
FROM tips
LIMIT 5;
PD
tips[['total_bill', 'tip', 'smoker', 'time']].head(5)

#ISSlearn
SQL Comparison - WHERE
SQL
SELECT *
FROM tips
WHERE time = 'Dinner'
LIMIT 5;
PD
tips[tips['time'] == 'Dinner'].head(5)

#ISSlearn
SQL Comparison - WHERE
SQL
SELECT *
FROM tips
WHERE size >= 5 OR total_bill > 45;
PD
tips[(tips['size'] >= 5) | (tips['total_bill'] > 45)]

#ISSlearn
SQL Comparison - WHERE NULL
SQL
SELECT *
FROM frame
WHERE col2 IS NULL;
PD
frame[frame['col2'].isna()]

#ISSlearn
SQL Comparison - WHERE NULL
SQL
SELECT *
FROM frame
WHERE col1 IS NOT NULL;
PD
frame[frame['col1'].notna()]

#ISSlearn
SQL Comparison - GROUP BY
SQL
SELECT sex, count(*)
FROM tips
GROUP BY sex;
/*
Female 87
Male 157
*/
PD
tips.groupby('sex').size()

#ISSlearn
SQL Comparison - GROUP BY
PD
In [19]: tips.groupby('sex')['total_bill'].count()
Out[19]:
sex
Female 87
Male 157
Name: total_bill, dtype: int64

#ISSlearn
SQL Comparison - INNER JOIN
SQL
SELECT *
FROM df1
INNER JOIN df2
ON df1.key = df2.key;
PD
pd.merge(df1, df2, on='key')

#ISSlearn
SQL Comparison - LEFT OUTER JOIN
SQL
-- show all records from df1
SELECT *
FROM df1
LEFT OUTER JOIN df2
PD
pd.merge(df1, df2, on='key', how='left')

#ISSlearn
SQL Comparison - RIGHT JOIN
SQL
SELECT *
FROM df1
RIGHT OUTER JOIN df2
PD
pd.merge(df1, df2, on='key', how='right')

#ISSlearn
SQL Comparison - FULL JOIN
SQL
SELECT *
FROM df1
FULL OUTER JOIN df2
PD
pd.merge(df1, df2, on='key', how='outer')

#ISSlearn
SQL Comparison - UNION
SQL
SELECT city, rank
FROM df1
UNION
SELECT city, rank
FROM df2;
PD
pd.concat([df1, df2]).drop_duplicates()

#ISSlearn
Pandas Cheat Sheet

#ISSlearn 23
Pandas Workshop
Workshop
https://github.com/kenken64/PandasAteMyData

#ISSlearn 24
THANK YOU ☺
kenneth.phang@nus.edu.sg

NUS-ISS Learning Day 2018- Pandas ate my data

More Related Content

Similar to NUS-ISS Learning Day 2018- Pandas ate my data (20)

More from NUS-ISS (20)

Recently uploaded (20)

NUS-ISS Learning Day 2018- Pandas ate my data