Unit - I - Lecture-2.pdf
- 1. CS3352 - Foundations of Data Science III Semester CSE © Vignesh Saravanan K, AP/CSE Lecture-2 Data Science Process: Overview UNIT I – INTRODUCTION
- 2. 2 © Vignesh Saravanan K, AP/CSE Foundations of Data Science RAMCO INSTITUTE OF TECHNOLOGY 2 Data Science Process – 6 Steps (RRPEMP) Setting the research goal Retrieving data Data preparation Data exploration Data modeling Presentation and automation
- 3. 3 © Vignesh Saravanan K, AP/CSE Foundations of Data Science RAMCO INSTITUTE OF TECHNOLOGY 3 1. Setting the research goal When the business asks you to perform a data science project, you’ll first prepare a Project Charter. Project charter contains information such as: What you’re going to research? How the company benefits from that? what data and resources you need? A timetable. Expected deliverables.
- 4. 4 © Vignesh Saravanan K, AP/CSE Foundations of Data Science RAMCO INSTITUTE OF TECHNOLOGY 4 2. Retrieving data The second step is to collect data. In this step you ensure that you can use the data in your program, which means: Checking the existence of data Quality and access to the data. Data can also be delivered by third-party companies and takes many forms ranging from Excel spreadsheets to different types of databases.
- 5. 5 © Vignesh Saravanan K, AP/CSE Foundations of Data Science RAMCO INSTITUTE OF TECHNOLOGY 5 3. Data preparation Data collection is an error-prone process In this step phase you enhance the quality of the data and prepare it for use in subsequent steps Consists of three sub-phases: Data Cleansing - removes false values from a data source Data Integration - enriches data sources by combining information from multiple data sources Data Transformation - ensures that the data is in a suitable format for use in your models.
- 6. 6 © Vignesh Saravanan K, AP/CSE Foundations of Data Science RAMCO INSTITUTE OF TECHNOLOGY 6 4. Data exploration Deeper understanding of your data. Try to understand: how variables interact with each other, the distribution of the data, and whether there are outliers. To achieve this you mainly use descriptive statistics, visual techniques, and simple modeling. - Exploratory Data Analysis (EDA)
- 7. 7 © Vignesh Saravanan K, AP/CSE Foundations of Data Science RAMCO INSTITUTE OF TECHNOLOGY 7 5. Data modeling or model building In this phase you use models, domain knowledge, and insights about the data Select a technique from the fields of statistics, machine learning, operations research, and so on Building a model is an iterative process that involves selecting the variables for the model, executing the model, and model diagnostics
- 8. 8 © Vignesh Saravanan K, AP/CSE Foundations of Data Science RAMCO INSTITUTE OF TECHNOLOGY 8 6. Presentation and automation Finally, you present the results to your business. Results can take many forms: ranging from presentations Graphs, pictures, analysis diagrams research reports Sometimes you’ll need to automate the execution of the process, business will want to use the insights you gained in another project.
- 9. CS3352 - Foundations of Data Science III Semester CSE © Vignesh Saravanan K, AP/CSE CS3352 - Foundations of Data Science III Semester CSE © Vignesh Saravanan K, AP/CSE CS3352 - Foundations of Data Science III Semester CSE © Vignesh Saravanan K, AP/CSE End of Lecture • Data Science Process – 6 Steps
- 10. 10 © Vignesh Saravanan K, AP/CSE Foundations of Data Science RAMCO INSTITUTE OF TECHNOLOGY 10