Big data&hadoop
Download as PPTX, PDF0 likes294 views
Bigdata and Hadoop are techniques for managing and analyzing large datasets. Hadoop is an open-source framework that allows processing of bigdata across clusters of computers. It addresses issues like high volume, velocity, and variety of bigdata by distributing storage and parallelizing processing. Hadoop uses MapReduce and HDFS to split data into parts, process it in parallel on different machines, and integrate the results for scalability.
Big data&hadoop
- 1. BIGDATA AND HADOOP By Ram and Raghavendra
- 3. BIGDATA What is bigdata…..? Bigdata is the type of data which contains large volume of files in the form of vedios, audios, Pictures, documents etc……….
- 5. BIG DATA Face book Media ,Government,Flipkart etc……………
- 6. Types of bigdata 1.Structured data 2.Un structured data
- 7. Structured data: It is the similar type of data which contains same category of files. ex: 1.text files Text file1 Text file2 ……….. picture1 picture22.pictures ……..
- 8. Unstructured data: It is the combination of different types of data. vedios audios pictures documents
- 9. Three Characteristics of Big Data V3s Volume •Data quantity Velocity •Data Speed Variety •Data Types
- 10. ABOUT BIGDATA • Everyday we are creating 2.5 quintillion bytes of data • 90% of data in the world has been created in the last two years • Facebook generates 500+ terabytes of data per a day
- 11. Difficults with bigdata It is too difficult to manage this bigdata for 1. analysis 2. capture 3.curation 4.search 5.sharing 6.storage 7.transfer 8.visualization and information privacy , with standard database management systems like DBMS and RDBMS.
- 13. WHAT IS HADOOP....? Hadoop Framework Of Tools Is Open source(APACHE)
- 14. Objective : Hadoop Running applications on Bigdata SUPPORTS
- 15. Challenging points to Hadoop velocity varietyvolume
- 16. Traditional Approach • Enterprise Approach: Big Data Processed By Powerful computer
- 17. Traditional Approach: • Enterprise Approach: Big Data Processing limit Powerful computer Only so much data could be processed
- 18. Breaking the Data Big Data Is broken into pieces
- 19. move computation to the data Big Data Combined result COMPUTATION
- 21. DISTRIBUTED MODEL • 1.THESE ARE LOW COST COMPURTERS • 2. WORKS ON LINUX BASED MACHINES LINUX LINUX LINUX LINUX
- 22. TASK TRACKER AND DATA NODES TASK TRACKER DATA NODE TASK TRACKER DATA NODE TASK TRACKER DATA NODE TASK TRACKER DATA NODE SLAVES
- 23. TASK TRACKER DATA NODE TASK TRACKER DATA NODE TASK TRACKER DATA NODE TASK TRACKER DATA NODE SLAVES TASK TRACKER NAME DATA NODE NODE MASTER JOB TRACKER
- 25. TASK TRACKER DATA NODE TASK TRACKER DATA NODE TASK TRACKER DATA NODE TASK TRACKER DATA NODE SLAVES TASK TRACKER NAME DATA NODE NODE Map Reduce JOB TRACKER MASTER
- 27. TASK TRACKER DATA NODE TASK TRACKER DATA NODE TASK TRACKER DATA NODE TASK TRACKER DATA NODE SLAVES TASK TRACKER NAME DATA NODE NODE Batch processing JOB TRACKER MASTER Application Queue
- 28. TASK TRACKER DATA NODE TASK TRACKER DATA NODE TASK TRACKER DATA NODE TASK TRACKER DATA NODE SLAVES TASK TRACKER NAME DATA NODE NODE Job Tracker JOB TRACKER MASTER
- 29. TASK TRACKER DATA NODE TASK TRACKER DATA NODE TASK TRACKER DATA NODE TASK TRACKER DATA NODE SLAVES TASK TRACKER NAME DATA NODE NODE FAULT TOLERANCE FOR DATA NODE JOB TRACKER MASTER
- 30. TASK TRACKER DATA NODE TASK TRACKER DATA NODE TASK TRACKER DATA NODE TASK TRACKER DATA NODE SLAVES TASK TRACKER NAME DATA NODE NODE FAULT TOLERANCE FOR PROCESSING JOB TRACKER MASTER
- 31. TASK TRACKER DATA NODE TASK TRACKER DATA NODE TASK TRACKER DATA NODE TASK TRACKER DATA NODE SLAVES TASK TRACKER NAME DATA NODE NODE Master Backup JOB TRACKER MASTER
- 32. Easy programming Do not worry about 1.Where the file is located 2.How to manage failures 3.How to break competitions into pieces programmers 4.Scalability
- 33. Name •Name was given by Doug cutting •Created by Doug cutting Mike cafarella(yahoo) in 2005 •Yahoo donated HADOOP to Apache in 2006
- 34. Usage Areas •Social media •Retail •Financial services •Searching tools •Government •Intelligence
- 35. Companies • Yahoo • Facebook • Amazon • eBay • American airlines • The NEW YORK Times • Chevron • IBM • Federal Reserve Board
- 36. Future outlook yahoo By 2015 50% of enterprise data will be processed by Hadoop
Editor's Notes
- #10: Acco.to IBM