What Does Open Data Mean to Data Science
Download as PPTX, PDF1 like438 views
This document discusses what open data means for data science. It provides three stories as examples of how open data can drive data science and change the way things are done. Open data is driving data science, which in turn is changing and will continue to change everything. However, there is a lot of data that is not accessible or persistent because it is hard to find and not maintained long-term. Solutions discussed include moving from data pipes to platforms, as funders and institutions now see the need to do. Platforms help break down silos and support key pillars of data science. Open data defines much of the new economy and contributes to social good.
What Does Open Data Mean to Data Science
- 1. What Does Open Data Mean to Data Science? Philip E. Bourne PhD, FACMI Stephenson Chair of Data Science Director, Data Science Institute Professor of Biomedical Engineering peb6a@virginia.edu https://www.slideshare.net/pebourne 06/07/18 OA @ UNT 1 @pebourne
- 2. Let me answer the question with a story… The case of the trauma surgeon… May 22, 2018 2
- 3. Not convinced … try this one... • North Virginia Technology Council Announces a Hackathon • Teams compete internally – undergrads led by Daniel Mietchen and Pete Alonso • Team selected and competes against GMU, VT, VCU • UVA wins! May 22, 2018 3
- 4. Source: U.S. Department of Veterans Affairs, US Census Bureau, data.gov The Problem May 22, 2018 4 Latest Estimate – 20 vets commit suicide every day In 2014, the latest year available, more than 7,400 veterans took their own lives, accounting for 18 percent of all suicides in America
- 5. Age Ranges Veteran Demographic s Period Served Veteran Population Total Male Female Source: 2012, 2014, 2015 Data acquired from U.S. Department of Veterans Affairs, US Census Bureau, Department of Defense Race Distribution Race Distribution May 22, 2018 5
- 6. Veteran and General Suicide Victim Demographics Suicide Rate by Age Key General Population Veteran Population Suicide by Race General Populatio Veteran Population Suicide Rate by Gender Key General Population Veteran Population Total Male Female Source: 2012, 2014 data acquired from U.S. Department of Veterans Affairs, data.gov, 2014 Centers for Disease Control and Prevention reports May 22, 2018 6
- 7. Correlation: Health Care by State ● ~2M Veterans lack health insurance ● 42% Unaware of VA benefits ● Complicated priority system (VA) ○ False PTSD diagnosis – est. 47,000 undiagnosed each year Source: 2014 data acquired from Veterans Affairs and census.gov May 22, 2018 7
- 8. Source: 2014 data acquired from Veterans Affairs, data.gov and census.gov Correlation: Social Isolation by State Key Rural Urban May 22, 2018 8
- 9. Source: 2014 data acquired from Veterans Affairs, data.gov and census.gov Correlation: Social Isolation as Measured by State Key Rural UrbanUtah Arizona New Mexico Nevada May 22, 2018 9
- 10. Source: 2014 data acquired from Veterans Affairs, data.gov and census.gov Firearm Regulation State Total Gun Laws Ammunition Regulations Background Checks Buyer Regulations Dealer Regulations Gun Trafficking Arizona 11 0 0 0 0 0 Nevada 11 1 0 0 0 0 New Mexico 10 0 0 0 0 0 Utah 11 0 0 0 0 2 All States 26.5 0.72 2.46 2.4 2.7 0.76 May 22, 2018 10
- 11. Firearm Access Veteran Suicide Firearm Regulation Source: 2014 data acquired from U.S. Department of Veterans AffairsMay 22, 2018 11
- 12. Social Media: Mining APIs and URLs 57.6 9% 90.0 0% Unsuccessful **Data collected using various Python data mining methods and respective Social Media APIsMay 22, 2018 12
- 13. May 22, 2018 13
- 14. Community: Content Creation and Mental Health Training Utilize social media data to increase outreach Have VA Resources complement private health care Recommendation s Limit firearm possession based on mental health status May 22, 2018 14
- 15. Open data is driving data science which in turn is/will change the way we do everything… 06/07/18 OA @ UNT 15
- 16. Digitization Deception Disruption Demonetization Dematerialization Democratization Time Volume,Velocity,Variety Digital camera invented by Kodak but shelved Megapixels & quality improve slowly; Kodak slow to react Film market collapses; Kodak goes bankrupt Phones replace cameras Instagram, Flickr become the value proposition Digital media becomes bona fide form of communication From a presentation to the Advisory Board to the NIH Director Example - photography 1604/08/18
- 17. So what is the problem? … 06/07/18 OA @ UNT 17 There is lots of data but it is hard to find and is not persistent …
- 18. We are not FAIR • Digital assets (objects) within that system are data, software, narrative, course materials etc. • Assets are to varying degrees FAIR – Findable, Accessible, Interoperable and Reusable https://www.workitdaily.com/job-search-solution/ FAIR: https://www.nature.com/articles/sdata201618 6/19/17 18
- 19. There is lots of data, but it gets lost quickly • Big Data • Total data from NIH-funded research currently estimated at 650 PB* • 20 PB of that is in NCBI/NLM (3%) and it is expected to grow by 10 PB this year • Dark Data • Only 12% of data described in published papers is in recognized archives – 88% is dark data^ • Cost • 2007-2014: NIH spent ~$1.2Bn extramurally on maintaining data archives * In 2012 Library of Congress was 3 PB ^ http://www.ncbi.nlm.nih.gov/pubmed/26207759 6/15/17 Dataverse 2017 19
- 20. Funders and publishers come at this from a perspective of reproducibility … 06/07/18 OA @ UNT 20
- 21. I Cant Reproduce My Own Work It took several months to replicate this work 6/19/17 21
- 22. The problem is more profound .. It inhibits doing the data science research in the first place … 06/07/18 OA @ UNT 22
- 23. Stating the problem is easy .. What are some of the solutions? 06/07/18 OA @ UNT 23
- 24. Both funders and institutions see the need to move from pipes to platforms… 6/15/17 Dataverse 2017 24 https://blog.lexicata.com/wp-content/uploads/2015/03/platform-model- 750x410.png
- 25. Example: NSF and NIH Approaches 6/15/17 Dataverse 2017 25
- 26. What evidence is there that platforms work? • Airbnb is a platform that supports a trusted relationship between consumer (renter) and supplier (host) • The platform focuses on maximizing the exchange of services between supplier and consumer and maximizing the amount of trust associated with a given stakeholder • It seems to be working: • 60 million users searching 2 million listings in 192 countries • Average of 500,000 stays per night. • Evaluation of US $25bn Bonazzi & Bourne 2017 PLOS Biology 15(4) e2001818 6/15/17 Dataverse 2017 26
- 27. 06/07/18 OA @ UNT 27 OpenDataLab
- 28. Platforms support 4 of the 5 pillars of data science 28 Data Integration & Engineering Machine Learning & Analytics Visualization & Dissemination Data Acquisition Ethics, Law, Policy, Social Implications May 22, 2018
- 29. In summary • Open data defines much of the new economy and contributes to social good • This may be incentive enough • We are all part of this fourth paradigm • To fully realize the potential of open data we must be FAIR • We need to breakdown silos – platforms help 06/07/18 OA @ UNT 29
- 30. Since libraries are experienced with open knowledge for the public good they have a key role to play. But how? 06/07/18 OA @ UNT 30