Data Science towards the Digital Enterprise
2 likes180 views
The document details Jake Bouma's journey into data science within a digital enterprise, emphasizing the significance of open data science as a movement that integrates tools and technologies. Key themes include the challenge of leveraging messy data, the importance of scalability, and the need for agile methodologies in project management. The narrative highlights the underutilization of data as an asset and suggests that building relationships and fostering a supportive culture are critical to success in data science initiatives.
Data Science towards the Digital Enterprise
- 1. Data Science a n u n e x p e c te d j o u r n e y t o t h e Digital Enterp ris e J a k e B o u m a A p r i l 2 0 1 7
- 2. What I’m about “Open Data Science is not a single technology, but a revolution within the data science community; an inclusive movement that makes open source tools for data science - data, analytics and computation - easily work together as a connected ecosystem.” https://www.linkedin.com/in/jaketbouma https://github.com/jaketbouma
- 3. Project Switzerland ( w e ’ r e n e u t r a l l i k e t h a t ) “To develop the business intelligence engine that enables BLT to aggregate, process, analyse and prioritise large sets of customer data to reveal patterns and trends for improved customer engagement, increased revenue and operating efficiencies”
- 4. On your marks, get set, data science!? CEO Mark Levy Data Scientist Jake Bouma CIO Andrew Murray
- 5. The first two months • Extensive meetings with Business Units to identify opportunities • The data battle: building relationships and scraping sources together • Hacking POCs out <Some preliminary results that I couldn’t share publicly>
- 6. Value Demonstrated “Data is the organization’s most underutilized asset.” -- Andrew Murray
- 7. Actually, this is a big data problem
- 8. To Hadoop or not to Hadoop This thing is big. Data science works better on more data. The most interesting data is messy. Speed is king. Performant access to the most granular, rawest form of everything
- 9. Hadoop has taught us some important lessons Stop trying to force one tool to do everything. Scale out works, send the code to the data. Relational databases are still quite nice.
- 10. DevOps for big data DATA COMPUTE Build what you need, when you need it Decouple data from applications Always be ready to scale
- 11. 6 Months later: State of the art BDP * 3 Networks, > 10 different types of server, >20 running servers, cloud native
- 12. BDP cruising “There is just too much going on.” -- Jake Bouma
- 13. Agile Methodologies to the Rescue! Small ideas Small parts of a big idea Big Ideas Fail Fast Prioritize and Track Long Term Objectives Timescale of quarters Timescale of weeks
- 14. Building out a team “Practical solutions to not very well defined problems.” -- Owen Zhang Top Ranked Kagglers list on their profiles: • Electronics Engineer • Quantitative Researcher • Accounting, Finance and Risk • Computer Science • Credit risk modelling • Entrepreneur • Actuarial science • Business Analyst • No Physicists!?
- 15. Are we there yet? “I think I missed a trick along the way...” -- Jake Bouma
- 16. Culture is the spice that makes it Digital
- 17. Agile Platform BIG DATA TECHNOLOGY DATA SCIENCE OPENNESS CULTURE AGILE Questions Action FeedbackImprovement DEVOPS The working digital system of insight?
- 20. Something to think about