Why Data Science Projects Fail?

Editor's Notes

• #6: Expect magic to happen! YOLO (You Only Look Once) is a lightweight real-time object detection – can detect objects on a video-stream. It took 5 years to get to this version.. Dan Ariely: Big data is like teenage sex: everyone talks about it, nobody really knows how to do it, everyone thinks everyone else is doing it, so everyone claims they are doing it...
• #7: Precision  accuracy False positive vs false negative… Think of automatic cancer prediction – Can reduce false negatives of a human proffesional. - - like a radiologist
• #9: Outputs of data science: Descriptive analysis (a report): clear answer to a clear question like what should be the platform to release the new product: Android or iOS. This is usually on offline system. Dashboard: helps a human decide and take action continuously, or again and again. This is usually an online/real-time system. Automated decision making: based on the dashboard, take automatic action. (“self driving” system) Data-set: data that is then used by another algorithm. For example, a cleaned-up list of addresses that were given by users on a form. A data-set used for training and benchmarking object extraction from images: COCO dataset or IMAGE-NET dataset. If your dataset is no good you will never get a good results. Qualities of a dataset: Leverage: the potential of the dataset - - what it can be used for Friction to impact: what is the additional work needed on the dataset to get a significant Cleanness: percentage of errors in the dataset that may sabotage the learning process. A methodology: the system/algorithm that is used to take a dataset and create a model that can then be used to answer a question. For example, how to estimate national poll results based on a sample questioner of 500 ppl >> a "Bias correction" system A Recommendation system Framework: an API or SDK that is used to build (code) methodologies. For example, Google's AI framework, TensorFlow. The framework should assist in lowering the Friction to impact. Proof-of-concept: it does not give the business impact but it gives the notion that the methodology is viable. Used for "fail-fast" or as a first milestone in a larger project.
• #10: Computer science >> computer engineering Math & stats - - many times done by physicians Need ppl that can do the data tagging It’s a research project! Data Science is more than machine learning The importance of diversity in a data-scientists team: based on the diagram it is clear that it's very hard to find ppl that are able to answer all the above, especially for a team that is meant to answer questions from a diverse set of domains. Some like offline-analysis Some like real-time systems Some are about processes some are about tools Some are very good in one domain but has zero knowledge about anything else… Etc..   A good data scientist better have at least excellent proficiency in one side _and_ at least some understanding in the other 2 sides.  
• #11: Example of how complicated an ML project can be…YOLO (You Only Look Once) is a lightweight real-time object detection – can detect objects on a video-stream. It took 5 years to get to this version..
• #12: Google Translate’s Maori dataset is too small, leading to some funny mistakes. Better not train your model on these cat pictures… A satiations would know this… but a computer system engineer would not. Internal politics – would the engineer get access to the transactions database???
• #14: Feedback-loop in results => need to understand causality. e.g. testing a 'like' btn size. clicking 'like' on a big-btn brings the item to top of list for everyone so it affects control-group clicks. Must make sure the observational inference matches causality. You need to be able to understand the result.
• #15: … Boosting Consider changing to Yoav’s chart - - give examples
• #16: Building an application with a good UX is outside the scope of a Data Scientist team Tay is an AI-based chat bot created by Microsoft and “unleashed” on Tweeter in 2016. It soon absorbed what people talking with her as the truth..
• #17: URME Personal Surveillance Identity Prosthetic – by http://www.urmesurveillance.com/ Kerckhoffs-Shannon principle: “one ought to design systems under the assumption that the enemy will immediately gain full familiarity with them”. Don’t rely on the privacy of the model because one day or another, it will be leaked. You should not base your code entirely on open-source algorithms. You should not base your model on open data-sets
• #18: Generative Adversarial Networks (GAN) – is sometimes used to fool the original network. In the example: projected gradient decent Synthesizing adversarial examples for neural networks is surprisingly easy: small, carefully-crafted perturbations to inputs can cause neural networks to misclassify inputs in arbitrarily chosen ways. Given that adversarial examples transfer to the physical world and can be made extremely robust, this is a real security concern.
• #19: In the GIF: Tesla Model S adaptive cruise control 1 second before crashes into a parked Van on the roadside - - May 2016
• #20: KPI: Key Performance Indicator
• #21: Generative Adversarial Networks (GAN) – is sometimes used to fool the original network >> it can be used to understand how the neural network works.
• #25: Original map (interactive): http://scikit-learn.org/stable/tutorial/machine_learning_map/
• #26: Types of data - 2 axis: Is it a qualitive (e.g. questionnaire) or a quantitative (sales transaction logs) Our data (e.g. logs) or 3rd pty data (e.g. Wikipedia dataset)