Intelligent Big Data analytics for the future.
Download as PPT, PDF0 likes790 views
The document discusses the limitations of human data scientists in big data analytics, including their high cost and error-proneness, and explores the potential of artificial intelligence systems to reduce reliance on them. It highlights the capabilities of intelligent big data engines to process large volumes of data, learn from it, and automatically identify patterns. The text also notes the significant costs involved in conventional big data solutions and emphasizes the need for automation and enhanced computational power in data analysis.
Intelligent Big Data analytics for the future.
- 2. The current process of Big data analytics involves considerable presence of human element in form of data scientists and analysts who are Difficult to find because of their unique skill set. Expensive. Prone to errors common with any human and only work on principles of limited but well definite set of rules and algorithms that operate within limited scope of learning. Can we reduce the involvement of data scientists and analysts by using Artificially Intelligent systems for big data processing?
- 3. An intelligent big data engine can… Process and predict based on huge volumes of data. Learn from the data. Identify patterns and cause and effect relationship. Utilize Combinatorics computational model to overcome the limitations of a human working on the same problem.
- 4. Facebook AI analysis system Google’s Deep Learning Big Data Analytics in Medical Field IBM Watson Labs
- 5. ’ Facebook aims to Use AI to analyze the profile semantically from the activities. A data scientist would limit the pace by finding which pattern to apply. The engine will use its computational power to find a pattern, learns that pattern and apply the same pattern to other profiles. Image: www.theverge.com
- 6. ’ 16000 computers, 10 million images from YouTube video frames and three days to see a cat?
- 7. ’ “We don’t understand how our deep-learning decision-making computer systems have made themselves so good at recognizing things in photos. This means that we may need fewer experts in future as it can instead rely on its semi-autonomous, semi-smart machines to solve problems all on their own.” --Quoc V. Le, Google software engineer, Machine learning conference San Francisco. Source: http://www.theregister.co.uk/2013/11/15/google_thinking_machines/
- 8. ’ What it means for you?
- 9. NuPIC :: www.numenta.org GROK :: www.numenta.com Quill :: www.narrativescience.com Yseop :: www.yseop.com
- 11. Case-study to monitor cloud servers •Stream data from Amazon CloudWatch. •It builds hundreds of models automatically and identifies the best model. •Get Insights, take action. Image: www.groksolutions.com
- 12. How much does your “conventional” Big Data Solution Cost? $740 million to Implement Enterprise Data Warehouse on Hadoop in 5 years for 500TB of data !! “$219 spent on Analysis” Image: Big Data: What Does It Really Cost? A WinterCorp Report
- 13. How much does your organization spent on Data Scientists? 200 TB = Need 50 Data Scientist Average of $120,000 - $180,000 = $150,000/annum Total Cost = 50 x 150,000 = $7,500,000 ($7.5 million)/annum Image: http://www.kdnuggets.com/2013/02/salary-analytics-data-mining-data-science-professionals.html
- 14. Forbes Survey on 211 Senior Marketers Need to Get 84% of agencies and non-agencies Smarte indicated it as critical for the success of r Faster… their marketing campaigns Fast-automated systems collect and analyze data critical for: Maintaining Data Quality Optimizing Processes Generating Good Return on Investment (ROI). Image: http://www.creditcards.com/credit-card-news/consumers-getting-smarter-about-credit_scores-1270.php
- 15. Leverage How much time and cost to destination? Predictive Impact after taking a particular route? Prescriptiv e Report on route congestion Proactive Analysis Image: http://www.wired.com/autopia/2009/03/fedex-gets-mad/
- 16. • Branch of Artificial Intelligence • Self aware and self learning system • Solves complicated problems where multiple predictions are required Image Annotation Retrieval Scenario Image: http://www.ngdata.com/wp-content/uploads/multi_target_prediction.pdf
- 17. Advertising Campaigns Identifies right time and communication medium to market product Performs real time analysis on big data and accounts variable change (feedback mechanism) Utilizes streaming analytics techniques to identify data for advertisement targeting Takeaway: Imagine, cost involved if data scientists carry all these tasks Image: http://thedesigninspiration.com/articles/40-clever-advertising-campaigns-of-mcdonalds/
- 18. Increased Computational Power Large Hadron Collider-LHC generates 5 trillion bits of data every second Increasing computational is NOT about adding processors Use past data sets to train system for future data sets Chop data into bits and distribute across fixed processors for machine learning Takeaway: Imagine, ROI and performance on achieving even 5% of Image:http://www.govtech.com/computing/Baltimore-Weaves-New-Infrastructure-with-Fabric-Based-Computing.html computational power similar to LHC
- 19. • Akshay Wattal: Analyzing cost effectiveness and efficiency of working with Intelligent Big data with fewer data scientists. • Mohana Kumaran S: Present Big Data infrastructure and justifying the need for Intelligent Big Data systems. • Mohul Kaila: Introduction to Big data and its evolution. • Shashank Garg: Identifying solutions to achieve intelligent big data systems and current state of art.