Introduction to basic data analytics tools
2 likes2,664 views
This document introduces basic data analytics tools. It discusses the data analytics pipeline of collecting, refining, storing, analyzing, and presenting data. It describes tools for each step including Requests and BeautifulSoup for data acquisition, Pandas and SQLAlchemy for data processing and storage, R and RStudio for data analysis, and Plotly and Matplotlib for data visualization. Apache Superset is highlighted as a tool for data visualization and exploration. Challenges of data analytics like data quality, privacy, and scaling are also outlined.
Introduction to basic data analytics tools
- 1. Introduction to Basic Data Analytics Tools Sa-ad Mahmud
- 2. What is Data Analytics? Data analytics is the science of analyzing raw data in order to make conclusions about that information. Data Analytics Pipeline Collect Refine Store Analyze Presentation
- 4. How To Collect Data! ★ REST API ★ From end users ★ Web scrape ★ Email and cloud storage ★ Client’s server
- 5. Requests A library for making HTTP requests in Python. Tool for Data Acquisition: Key Features: • Keep-alive & Connection Pooling • Sessions with Cookie Persistence
- 6. BeautifulSoup A library for parsing HTML and XML documents. Tool for Data Acquisition: Key Features: • Multiple parser support (e.g., lxml, html5lib, and others) • Creates parse tree which is easy to navigate
- 7. Flask, Flask-RESTPlus and Swagger UI Flask is a micro web framework written in Python. Flask-RESTPlus is an extension for Flask that adds support for quickly building REST APIs. It automatically documents the APIs which is visible in Swagger UI. Tools for Data Acquisition:
- 9. How To Clean Data! ★ Remove duplicate ★ Validate ★ Handle missing data ★ Fix errors ★ Filter outliers How To Store Data! ★ RDBMS ★ ORM
- 10. Pandas A library for data manipulation and analysis. Tool for Data Manipulation: Key Features: • Loading data into in-memory data objects from different file formats. • Data alignment and integrated handling of missing data.
- 11. SQLAlchemy SQLAlchemy is a popular SQL toolkit and Object Relational Mapper. Tool for Database Operations: Key Features: • Function-based query construction. • Multiple database support (e.g., SQLite, Postgresql, MySQL, Oracle, MS-SQL, Firebird, Sybase and others).
- 12. Data Analysis
- 13. How To Analyze Data! ★ Five number summary (maximum, minimum, median, 1st quartile, 3rd quartile) ★ Average ★ Standard Deviation ★ Ratio ★ Interval ★ Trends ★ Aggregate and group by ★ Regression ★ Clustering
- 14. R and RStudio R is a popular programming language for data analysis. RStudio is an IDE for R. Tools for Data Analysis: Original Classes Clusters by k-means
- 16. How To Visualize Data! ★ Charts ○ Line ○ Bar ○ Pie ○ Scatter ★ Graphs ★ Maps ○ Bubble ○ Polygon ★ Dashboards
- 17. Plotly An interactive graphing library. Tools for Data Visualization: Matplotlib A plotting library for Python.
- 18. Apache Superset A Data Visualization and Data Exploration Platform. Tool for Data Visualization: Key Features: • It supports all the data sources that support SQL Alchemy and supports querying using SQL. • Superset allows sharing dashboards. • It comes with security features like Authentication, User Management and Roles.
- 19. Other Notable Tools ★ Excel ★ Tableau Public ★ Grafana ★ Microsoft Power BI ★ And many more . . .
- 20. Challenges 1. Poor quality data 2. Data privacy and security 3. Weak infrastructure 4. Data from multiple sources 5. Scaling data analysis
- 21. Links Flask: https://flask.palletsprojects.com/en/2.0.x/ Flask-RESTPlus: https://flask-restplus.readthedocs.io/en/stable/ SQLAlchemy: https://www.sqlalchemy.org/ R Programming Language: https://www.r-project.org/ k-means Clustering: https://en.wikipedia.org/wiki/K-means_clustering Plotly: https://plotly.com/ Superset Docs: https://superset.apache.org/docs/intro Presentation GitHub Link: https://github.com/saadrumon/basic-data-analytics-tools-presentation.git
- 22. Thank You Any Questions? “Information is the oil of the 21st century, and analytics is the combustion engine.” - Peter Sondergaard