Data Science Project Lifecycle
8 likes5,781 views
The document outlines the typical lifecycle of a data science project, including business requirements, data acquisition, data preparation, hypothesis and modeling, evaluation and interpretation, and deployment. It discusses collecting data from various sources, cleaning and integrating data in the preparation stage, selecting and engineering features, building and validating models, and ultimately deploying results.
Related topics:1 of 11
Downloaded 200 times
Data Science Project Lifecycle
- 1. Data Science Project Lifecycle Jason Geng @Data Application Lab Miya Du @Data Science Association
- 3. Business Requirements u Data scientists need to work with business people and those with expertise in understanding the data, understanding the business u Specify the business requirements u For instance, the healthcare data
- 4. e.g. ‘DISCWT’: ‘This the discharge-level weight on the HCUP nationwide data to produce national estimates’ Understand the data: Understand the Business: Goal: Predict Readmission Rate Database: Healthcare: Readmissions Database Modeling
- 5. Data Collection u Data from product line u Purchase third party data u Social media (Facebook, LinkedIn) u Web crawling u Open source data (Opendata, U.S. Census Data) Challenge Data Storage Data Management
- 6. Legacy data OLTP Web Log Web Crawler Open Source Third Party Data Social Media Data XML CSV LOG SQL … Product Line Business Intelligence Data Science App
- 7. Data Preparation (Data Wrangling) u Cleaning data (semantic errors, missing entries, or inconsistent formatting) u Challenge: data integration u 80% time in project workflow Data Source A Data Source B Data Source B ETL Data Warehouse
- 8. Feature Engineering Select or creating features Research feature relevance Experiment and validation Change the feature set Go back to feature selection step
- 10. Deploy to Product Line
- 11. Thank you! https://www.DataAppLab.com Feb 2017 PPT: Xiaolu Zhao @ Feb 16, 2017