What data scientists really do, according to 50 data scientists
- 1. What data scientists really do, according to 50 data scientists Hugo Bowne-Anderson @hugobowne
- 2. Illustrations you can use, just copy/pasteA bit about me ➔ Hugo Bowne-Anderson, data scientist at DataCamp ◆ Undergrad in sciences/humanities (double math major) ◆ PhD in Pure Mathematics (UNSW, Sydney) ◆ Applied math research in cell biology (Yale University, Max Planck Institute) ◆ Python curriculum engineer at DataCamp ◆ Host of DataFramed, the DataCamp podcast
- 3. ➔ Data science at a high level ➔ The nitty gritty of data science ➔ Future data science Today’s topics of discussion
- 4. ➔ Data science at a high level ◆ Data science as hype ◆ What data scientists do (take one) ◆ The emergence of modern data science ◆ Data science and the decision function ➔ The nitty gritty of data science ➔ Future data science Today’s topics of discussion
- 5. Where is data science in the Gartner Hype cycle?
- 6. Where is big data in the Gartner Hype cycle?
- 7. Where is data science in the Gartner Hype cycle?
- 8. ➔ Data science at a high level ◆ Data science as hype ◆ What data scientists do (take one) ◆ The emergence of modern data science ◆ Data science and the decision function ➔ The nitty gritty of data science ➔ Future data science Today’s topics of discussion
- 9. What Data Scientist Actually Do ➔ Build Venn Diagrams ➔ Have Imposter Syndrome ➔ Love Logistic Regression Today’s data scientists do three things...
- 10. Venn Diagrams I google image searched “Venn diagram” and look what I found:
- 11. Venn Diagrams
- 12. Venn Diagrams
- 13. Drew Conway on building data science teams ➔ “Data Science” diagram, NOT “Data Scientist” diagram
- 14. Data science is ill-defined ➔ In terms of techniques, career trajectory, job titles ➔ “Do you think that imprecise ethics, no standards of practice, and a lack of consistent vocabulary are not enough challenges for us today?” -- Hilary Mason ➔ Brandon Rohrer (Facebook), in an article called “Imposter Syndrome”:
- 15. Imposter syndrome ➔ Fernando Perez, co-founder & co-lead of Project Jupyter
- 16. Logistic regression ➔ Simple, communicable, interpretable ➔ May not perform as well as extreme gradient boosting or a CNN, but what are you optimizing for? ➔ What is your (business) question? ➔ Claudia Perlich (DStillery, Two Sigma), Chris Volinsky (AT&T) + many more...
- 17. How data scientists think
- 18. Humans are horrible at statistics: thought experiment ➔ Police have breathalyzers have a 5% false positive rate, 0% false negative rate. ➔ 1 in 1,000 drivers are driving drunk. ➔ Police stop a driver at random & the test is positive. ➔ What is the probability that the driver is actually drunk? Michael Betancourt
- 19. We are horrible at statistics: thought experiment ➔ Imagine you do this 1,000 times. ➔ 999 people will NOT be drunk but, due to false positives, ~50 will test positive. ➔ 1 person will be drunk and will test positive. ➔ 1 out of 51 are actually drunk! ➔ <2% of the positive test results are true positives. ➔ This is known as the base-rate fallacy. ➔ It’s a data scientist’s job to correct these heuristics and biases within organisations AND society. Michael Betancourt
- 20. Thinking and talking probabilistically ➔ FiveThirtyEight, highly reputable pollsters, gave Clinton a 71.4% chance of winning and Trump a 28.6% chance. ➔ I.e., Trump winning, according to this model, was more likely than flipping two coins and getting two heads, which has a 25% chance. Andrew Gelman Allen Downey
- 21. Thinking and talking probabilistically ➔ Various pollsters predicted that Secretary Clinton would win with probabilities ranging from 70% to 95%. ➔ It was common to speak of these predictions as though all experts agreed that Clinton would win and simply had different levels of certainty. ➔ That makes the mistake of interpreting the models as deterministic, not probabilistic. ➔ Even if an event is only 10% likely, that’s about as likely as flipping three coins and getting all heads, which we wouldn’t find all too surprising.
- 22. Thinking and talking probabilistically ➔ Various pollsters predicted that Secretary Clinton would win with probabilities ranging from 70% to 95%. ➔ It was common to speak of these predictions as though all experts agreed that Clinton would win and simply had different levels of certainty. ➔ That makes the mistake of interpreting the models as deterministic, not probabilistic. ➔ Even if an event is only 10% likely, that’s about as likely as flipping three coins and getting all heads, which we wouldn’t find all too surprising.
- 23. ➔ Data science at a high level ◆ Data science as hype ◆ What data scientists do (take one) ◆ The emergence of modern data science ◆ Data science and the decision function ➔ The nitty gritty of data science ➔ Future data science Today’s topics of discussion
- 24. Data Science Emerged ➔ Google ➔ BuzzFeed ➔ LinkedIn
- 25. Illustrations you can use, just copy/pasteData Science in Tech 1. Lay Solid data foundation to perform robust analytics 2. Use online experiments to achieve sustainable growth 3. Build machine-learning pipelines and personalized data products to make better business decisions Robert Chang
- 26. Illustrations you can use, just copy/pasteMachine Learning in tech Airbnb: Customer Lifetime Value (Robert Chang)
- 27. Illustrations you can use, just copy/pasteRogati’s AI Hierarchy of Needs The AI Hierarchy of Needs, Monica Rogati
- 28. ➔ Data science at a high level ◆ Data science as hype ◆ What data scientists do (take one) ◆ The emergence of modern data science ◆ Data science and the decision function ➔ The nitty gritty of data science ➔ Future data science Today’s topics of discussion
- 29. Data science and the decision function ➔ Data scientists answer business questions & are one of several inputs into the decision-making process. ➔ Renee Teate (Heliocampus, Becoming a Data Scientist):
- 30. Data science and the communication ➔ Communication skills (and data translation) ➔ ““Which skill is more important for a data scientist: the ability to use the most sophisticated deep learning models, or the ability to make good PowerPoint slides?” (Jonathan Nolis, DS consultant & leader, Seattle Area) ➔ As a data scientist, when do you schedule meetings with decision makers?
- 31. When to schedule meetings? ➔ Extraneous factors in judicial decisions (Danziger et al., PNAS, 2011)
- 32. Data science and the decision function ➔ But wait, there’s more: ➔ “Within each session, unrepresented prisoners usually go last and are less likely to be granted parole than prisoners with attorneys.” -- Overlooked factors in the analysis of parole decisions (Weinshall-Margel et al., 2011, PNAS) ➔ Be wary of the narrative fallacy.
- 33. Data science embeddings in orgs ➔ Centre of Excellence: dedicated data science team ➔ Embedded model: data scientists embedded in every team ➔ Hybrid model (e.g. Airbnb, DataCamp) ➔ Data scientist as “decision intelligence” operative
- 34. Data science embeddings in orgs
- 35. ➔ Data science at a high level ➔ The nitty gritty of data science ◆ What data scientists actually do ◆ Types of data science ➔ Future data science Today’s topics of discussion
- 36. What Data Scientist Actually Do Today’s data scientist...
- 37. What Data Scientist Actually Do ➔ Data Collection and Cleaning ➔ Building dashboards and reports ➔ Data visualization ➔ Building models (statistical inference & machine learning) ➔ Communicate results to stakeholders ➔ Business decisions are then made! Today’s data scientist...
- 38. What Data Scientist Actually Do This will change...
- 39. Data Scientist Becoming More Specialized ➔ Type A data scientist ◆ Is an analyst, traditional statistician ➔ Type B data scientist ◆ Is building machine-learning models Generalist ---> Specialist Emily Robinson
- 40. Coda: data science consultancy ➔ “For consulting particularly, it makes sense if you are a generalist...the skillset of understanding what the client needs and figuring out how to get to the point where you can offer a solution.” -- Vicki Boykis
- 41. ➔ Data science at a high level ➔ The nitty gritty of data science ◆ What data scientists actually do ◆ Types of data science ➔ Future data science Today’s topics of discussion
- 42. Types Of Data Science 1. Business Intelligence 2. Machine Learning 3. Decision Science Jonathan Nolis breaks data science into 3 components
- 43. Types Of Data Science 1. ➔ Taking data company already has ➔ Getting that data to the right people ➔ In form of dashboards, reports, emails 1. Business Intelligence (descriptive analytics)
- 44. Types Of Data Science 1.
- 45. Types Of Data Science ➔ Put models continuously into production ➔ E.g. LTV at airbnb 2. Machine Learning (predictive analytics)
- 46. Types Of Data Science ➔ Take the insight discovered from the data science work ➔ Use it to help company decision making ➔ E.g. what do you if your data science work tells you a particular type of customer will churn? 3. Decision Making (prescriptive analytics)
- 47. ➔ Data science at a high level ➔ The nitty gritty of data science ➔ Future data science ◆ What data scientists need to learn ◆ The future of data science Today’s topics of discussion
- 48. Skills Data Scientists Should Have ➔ Deep learning ➔ AI ➔ Recurrent neural networks Skills To Focus On
- 49. Skills Data Scientists Should Have ➔ Asking the right questions ➔ Turning business questions into data science questions (and answers!) ➔ Learning on the fly ➔ Communicating well ➔ Explaining complex results to non-technical stakeholders ➔ Critical thinking and quantitative skills will remain in demand Skills To Focus On
- 50. ➔ Data science at a high level ➔ The nitty gritty of data science ➔ Future data science ◆ What data scientists need to learn ◆ The future of data science Today’s topics of discussion
- 51. “Do you think that imprecise ethics, no standards of practice, and a lack of consistent vocabulary are not enough challenges for us today?” Hilary Mason
- 52. Data ethics or lack thereof
- 53. Ethics of Data Science ➔ We’re approaching a consensus that ethical standards need to come from within data science itself, as well as from legislators, grassroots movements and other stakeholders.
- 54. “We need to have that ethical understanding, we need to have that training, and we need to have something akin to a Hippocratic Oath.” Github Senior Machine Learning Data Scientist Omoju Miller
- 55. Ethics of Data Science But do we need oaths, checklists and/or codes of conduct? This conversation is just now heating up! DrivenData has put together Deon, a command line tool, that allows you to easily add an ethics checklist to your data science projects.
- 56. Ethics of Data Science Part of this movement involves a re-emphasis on interpretability in models, as opposed to black-box models.
- 57. Ethics of Data Science ➔ Used to predict recidivism rate ➔ Then used to inform parole decisions COMPAS Recidivism Risk Score
- 58. Ethics of Data Science: Cathy O’Neil ➔ Old definition of DS: “A data savvy, quantitatively minded coding literate problem solver.” ➔ New definition: “Data science doesn't just predict the future. It causes the future.” ➔ Algorithmic audits, including a sensitivity analysis ➔ Ethical matrix: “rows are the stakeholders, the columns are the concerns.”
- 59. Ethics and OSS
- 60. Ethics and OSS “If you’re using the open source data science stack, then you’re benefiting from others’ work given freely to you. I’d say it makes perfect sense to give back.” -- Eric Ma
- 62. The garden of forking assumptions ➔ “One assumption people make a lot is that data is objective.” -- Lukas Vermeer, Booking.com.
- 63. The garden of forking assumptions ➔ First impressions last and humans look for patterns ➔ When data scientists perform analyses, they are also prone to confirmation bias ➔ The same data is often used for inspiration AND validation ◆ “Split your damned data” -- Cassie Kozyrkov ➔ Humans use confidence and precision as a proxy for accuracy ➔ The industry lacks consistent workflow (although see projects such as Cookiecutter data science) ➔ The state of the industry is similar to medicine before randomized control trials.
- 64. The looming credibility crisis
- 65. “Will we even have data science in 10 years? I remember a world where we didn’t and it wouldn’t surprise me if the title goes the way of ‘webmaster’.” Hilary Mason
- 66. Where is data science in the Gartner Hype cycle?
- 67. What will the slope of enlightenment look like? “Very few companies expect only professional writers to know how to write. So why ask only professional data scientists to understand and analyze data, at least at a basic level?” -- Jonathan Cornelissen, CEO, DataCamp in HBR