Elephants Ibises and a more Pythonic way to work with databases
- 1. Presented by: Elephants, ibises and a more Pythonic way to work with databases July 13, 2022 Marlene Mhangami
- 2. 2 ● Harare, Zimbabwe ● Developer Advocate Voltron Data ● Previous Vice-Chair and Director for the Python Software Foundation @marlene_zw marlenemhangami.com Marlene Mhangami
- 3. 3 Art Around Dublin post conference exploration
- 4. 4 MySQL database - Not as familiar - Had to switch to terminal - Working with specific data types (e.g geographic data) - Not as maintainable (tests, readability etc)
- 5. 5 There’s got to be a more Pythonic way?
- 7. 7 Ibis - A Python package that provides us with a more pythonic way to work to databases - A library for analytical SQL - Inspired by dplyr
- 8. 8 Ibis the idea behind ibis is that we want lightweight Python code telling the backend to do the heavy lifting Hey! Can you turn to the right and take us there
- 9. 9 - pandas-like - compiles to sql (executed efficiently) * anything you can write with a SELECT statement you can write in ibis
- 10. t.groupby(‘region’) .aggregate(topk=t.funding_rounds.count()) .sort_by(ibis.desc(‘topk’)) .head() Produces SQLAlchemy Expression <sqlalchemy.sql.select able.Select object> SQL BACKENDS Ibis produces Expressions or Strings for SQL backends SELECT region, count(founding_rounds) AS topk FROM companies GROUP BY region ORDER BY topk SQL STRINGS EXECUTED Ibis expression
- 11. 11 Ibis Elephants are large and your data can be too
- 14. 14 Installing Ibis pip install ibis-framework import ibis
- 15. 15 Connecting to database -switch to jupyter notebook (using vscode with theme) -import ibis -connect to database
- 16. 16 Table columns Add description... ● access sqlite table ● column names ● pandas-like
- 17. 17 Query the dataset Add description... - 420 artists part of the project - 5 Councils took part
- 18. 18 Methods in Ibis Add description... -methods available -table. (hit tab) -visit ibis-project.org for docs
- 19. 19 Displaying the art in a notebook Add description...
- 20. 20 Dublin Convention Center Find the canvas art closest to us
- 21. 21 Switching to PostgreSQL Add description... - GeoJSON file - PostGIS for working with geospatial geometries - Data stored in wkb format
- 22. 22 Working with GeoPandas Add description... - get wkb data - get corresponding art titles - use geopandas to change wkb to longitude and latitude - Create a dataframe containing titles and lats and long - add latitude and longitude of the convention center
- 23. 23 Working with GeoPandas Add description... Use Geopandas explore method on the dataframe to visualise geographic location of the art
- 24. 24 ‘Rabbits’
- 25. 25 ‘Beckett’
- 26. marlene@voltrondata.com marlene_zw Contact READ THE BLOG POST! Twitter SOCIAL MEDIA GENERAL CONTACT marlenemhangami.com
- 27. Thanks