Session 01 designing and scoping a data science project
Download as PPTX, PDF5 likes1,016 views
The document outlines a beginner's course on data science, focusing on project design, communication, and essential tools and concepts. It describes the process of data science, methods for acquiring, cleaning, and exploring data, and highlights the role of a data scientist. Additionally, it addresses ethical considerations in data projects and provides exercises to help participants define their questions and communicate results effectively.
Session 01 designing and scoping a data science project
- 1. Designing and Scoping a Data Science Project Data Science for Beginners, Session 1
- 3. Session Format Session: ā¢ One topic ā¢ Learn 4-6 concepts related to that topic ā¢ Try apps or code related to that topic Before each session: ā¢ Install required tools (see the ātool installsā instructions sheet) ā¢ Do background reading
- 4. Session Topics People ā¢ Designing a data science project ā¢ Communicating results Tools ā¢ Python basics ā¢ Enterprise data tools Getting Data ā¢ Acquiring data ā¢ Cleaning and exploring data Special data types ā¢ Handling text data ā¢ Handling geospatial data ā¢ Handling big data Learning from data ā¢ Predicting values from data ā¢ Learning relationships from data ā¢ Learning classes from data
- 5. Sessions Timeline 1. Scoping a data science project 2. Python basics 3. Acquiring data 4. Communicating results 5. Cleaning and exploring data 6. Predicting values from data 7. Handling text data 8. Handling geospatial data 9. Learning relationships from data 10. Enterprise data tools 11. Learning classes from data 12. Handling big data
- 6. Session 1: your 5-7 things ā¢ What is data science? ā¢ Data science is a process ā¢ Whatās a data scientist? ā¢ Data science competitions ā¢ Writing a problem statement
- 7. What is Data Science?
- 8. Defining Data Science āA data scientistā¦ excels at analyzing data, particularly large amounts of data, to help a business gain a competitive edge.ā āThe analysis of data using the scientific methodā āA data scientist is an individual, organization or application that performs statistical analysis, data mining and retrieval processes on a large amount of data to identify trends, figures and other relevant information.ā
- 10. Data Science is a Process ā¢ Ask an interesting question ā¢ Get the data ā¢ Explore the data ā¢ Model the data ā¢ Communicate and visualize your results
- 11. Ask an interesting question Write hypotheses that can be explored ā Do people have more phones than toilets? ā How is Ebola spreading? ā Is using wood fires sustainable in rural Tanzania? ā Can we feed 9 billion people? Make them simple, actionable, incremental
- 12. Get the data Data files (CSV, Excel, Json, Xml...) ā Databases (sqlite, mysql, oracle, postgresql...) ā APIs ā Report tables (tables on websites, in pdf reports...) ā Text (reports and other documentsā¦) ā Maps and GIS data (openstreetmap, shapefiles, NASA earth images...) ā Images (satellite images, drone footage, pictures, videosā¦)
- 13. Most data is small, butā¦
- 15. Explore the data
- 16. Model the Data
- 19. The Data Science Venn Diagram
- 20. How do you become a data scientist? Learning and Practice ā Kaggle - online datascience competitions ā Driven Data - social good datascience competitions ā Innocentive - some datascience challenges ā CrowdAnalytix - business datascience competitions
- 21. Should you become a data scientist? ā Not necessarily. There are lots of data science students desperate for good problems to work on. ā You might want to become someone who can work with data scientists ā Which means learning how to specify data problems well
- 23. Who Does What ā¢ Ask an interesting question ā¢ Get the data ā¢ Explore the data ā¢ Model the data ā¢ Communicate and visualize your results Problem Owner Competitor ?
- 24. DrivenData
- 25. Kaggle
- 26. DataKind
- 27. Example project: Pump It Up Tanzania wells: āYour goal is to predict the operating condition of a waterpoint for each record in the datasetā
- 28. Example project: Cervical cancer
- 29. DrivenData competition guidelines Impact: āā¦ clear win for the organisation in terms of effective planning, resources saved or people servedā¦ good story around how they generate social impactā¦ā Challenge: āā¦ challenging enough for a rich competitionā¦ā Feasibility: āā¦.the right kind of data to answer the question at handā¦ does it have enough signal to be useful?...ā Privacy: āā¦ can answer this question while protecting the privacy of individuals in the dataset and the operational privacy of an organisationā¦ā
- 31. Design your project Context: who needs this work, and what are they doing it for? Needs: what are you trying to fix Vision: what do you expect your final result to look like? Outcome: how do you get your results to the people who need them? What happens next?
- 32. Design your questions Is the question concrete enough? Can you translate the question into an experiment? Is it actionable? What actions will be taken given the answer? What data is needed to do the analysis?
- 34. Data Risk and Ethics Youāre responsible for your data outputs Could your outputs increase risk to anyone? How will you respect privacy and security?
- 35. Data Risk Risk: āThe probability of something happening multiplied by the resulting cost or benefit if it doesā Risk of: physical, legal, reputational, privacy harm Likelihood (e.g. low, medium, high) Risk to: data subjects, collectors, processors, releasers, users
- 36. PII: Personally Identifiable Information āPersonally identifiable information (PII) is any data that could potentially identify a specific individual. Any information that can be used to distinguish one person from another and can be used for de-anonymizing anonymous data can be considered PII.ā
- 37. PII Red Flags Names, addresses, phone numbers Locations: lat/long, GIS traces, locality (e.g. home + work as an identifier) Members of small populations Untranslated text Codes (e.g. ā41ā) Slang terms
- 38. Exercises
- 39. 3-minute exercise: Ask interesting questions Either your own questions: Questions that data might help with Stories you want to tell with data Datasets youād like to explore Or pick an existing question: ā Competition questions: Kaggle, DrivenData ā A data science project that interested you
- 40. 3-minute exercise: Get the data Pick one of your questions List the ideal data you need to answer it List the data thatās (probably) available Think about what youāll do if the data you need isnāt available What compromises could you make Where would you look for more data Are there proxies (other datasets that tell you something about your question)
- 41. 3-min exercise: design your communications List the types of people youād want to show your results to How do you want them to change the world? Can they take actions, can they change opinions etc Describe the types of outputs that might be persuasive to them - visuals, text, numbers, stories, artā¦ be as wild with this as you want
- 42. Things to do before next week See file Tool Install Instructions ā¢ Make friends with the terminal window ā¢ Install iPython ā¢ Install Git