PyData Ljubljana meetup #1
Download as PPTX, PDF0 likes302 views
The document summarizes the agenda for the first PyData meetup organized by Andrey Vykhodtsev. The meetup will include short introductions to Pandas and NumPy, a comparison of SQL and Pandas, an interactive demo of Pandas features like data loading, indexing, grouping and plotting. Attendees will learn why Pandas and NumPy are core tools in the Python data science toolchain and how numerous libraries build upon them.
PyData Ljubljana meetup #1
- 1. PyData meetup #1 Andrey Vykhodtsev <andrej.vihodcev@gmail.com>
- 2. Welcome to the first PyData meetup! • My name is Andrey Vykhodtsev • I plan to do 1 meetup per 2 months • Meetups can be more frequent and interesting if you also engage • Meetup group fee is paid by NumFocus (NCO) • Attendees get stickers (until out of stickers) • first meetup is pretty basic (sorry hardcore ppl.)
- 3. Agenda • Short intro Pandas / Numpy • SQL vs Pandas • Going through Pandas features in an interactive demo mode • data loading • Indexing, manipulating columns and rows • Grouping, stacking, reshaping, pivoting • Plotting directly from pandas • Time series, categoricals • Gotchas
- 4. Why are we talking about this • Pandas and Numpy are at the core of python datascience toolchain • PyData is another name • Numerous Python data science libraries work with or on top of Pandas / Numpy • scikit-learn • scikit-image • matplotlib • networkx • bokeh • Jupyter (these notebooks are made with Jupyter)
- 5. Intro to Numpy • Numpy is around for long time • Part of SciPy • Library to work with multidimensional arrays • Very fast • Vectorization • Integration with BLAS/Lapack and Intel MKL
- 6. # Pandas • Wrapper on top of Numpy • inspired by R data.frame and data.table • A lot of flexibility • sometimes lost performance • Drawbacks: • Single-threaded • Memory bound
- 7. Pandas vs. SQL • SQL syntax can be easier to read • SQL has mature optimizers • SQL works on top of databases • Windowed aggregates are easier to express in SQL • Pandas is more flexible • Pandas is geared towards datascience • Some things are much easier to express in Pandas than in SQL • how about "give me all columns that start with 'mn_' and aggregate them using mean"?
- 8. Loading data - Available methods • IO documentation • from CSV • from TXT • from SQL database • from Excel • from url, zipped • clipboard • json, xml, html • python pickle files • h5
- 9. reading and writing files • head command displays first 5 or N rows • delimiter parameter • usecols parameter • setting pandas display options • setting types while reading • nrow
- 10. Main data structures • Doc bookmark about data structures • pd.Series - like column, indexed by some values • for ints and floats it is actually numpy ndarray • pd.DataFrame - bunch of pd.Series, sharing same index • pd.Index, pd.MultiIndex • pd.GroupedData • pd.Panel
- 11. Indexing and selecting • by using df.col • by list of names • by list or range of numbers • by boolean vector • advanced slicing • query and np.where • df.loc, df.iloc, df.ix, df.at, df.iat
- 12. Further read • Modern Pandas series • Pandas docs