Big data introduction
Download as PPTX, PDF0 likes1,184 views
This document summarizes topics related to big data including: the size of big data in petabytes and exabytes; technologies like Hadoop and MapReduce that can handle large, unstructured data; and examples of big data systems like airline reservation predictions, Google Translate, Netflix movie recommendations, and Amazon book recommendations. These systems analyze huge amounts of past user data to make personalized predictions and recommendations. Technologies like MapReduce allow parallel processing of big data across many servers to achieve scalability that is not possible with traditional databases and SQL queries.
Big data introduction
- 1. Faiz ul haque Zeya MS CS University of Tulsa,OK,USA
- 2. Topics covered 1. Introduction 2.Bigdata: how big it is 3.Bigdata Technology. 4. Few examples of Big Data. 5. Airline reservation system 6. Google Translate. 7.Amazon recommendation. 8. Netflix recommendation. 9. Hadoop, Map reduce. 10. Q&A.
- 3. Introduction Large set of data. Site of peta byte, exa byte. Not stored relational. Massive scale computational. NO SQL queries. New technology like MAP REDUCE,HADOOP. Reason: Scalability and poor performance on large scale.
- 4. How large it is Peta byte 10^15 Zetta byte 10^21 Exabyte 10^ 18 Google processed about 24 petabytes of data per day in 2009.[ Yahoo stores 2 petabytes of data on behavior. eBay.com uses two data warehouses at 7.5 petabytes and 40PB as well as a 40PB Hadoop cluster for search, consumer recommendations, and merchandising.
- 6. BigData Technologies Relational database,SQL queries cannot handle such amount of data. Therefore other technologies are requried MAP REDUCE parallel computation.
- 7. Few examples of Big Data Airplane reservation system. Google Translate. Netflix Movie recommendation Amazon Book recommendation
- 8. Airline reservation system Oren Etzioni of Washington ‘s venture capital based startup Farecast. It predicts based on past data whether airline prices will go up or down. Etzioni uses predictive model for that. Microsoft purchase it for 110 M $ Make it part of BING search engine.
- 9. GOOGLE Translate Whole internet as training data.Corpus Google release Trillion word corpus in 2009. They accept messy data. Candide uses 3 million translated sentences. Google uses billions of pages from intenet.
- 10. Netflix Million $ prize Netflix announced to award 1M$ prize for the team who improves the recommendation algorithm by 5%. They are movie recommender. Most of the sales are due to recommendations from the site. Reason is that so many shows that the user don’t even know.
- 11. Amazon’s recommendation Amazon uses item to item recommendation instead of traditional collaborative recommendation. Item to item recommendation search for similar items rather than similar users. This approach is scalable to large data set.
- 12. Map Reduce
- 13. Q&A