Big Data Hadoop Training by Easylearning Guru
2 likes2,064 views
The document provides an overview of big data, its types, sources, and the global market trends, focusing on how large datasets challenge traditional data processing tools. It introduces Hadoop as a scalable solution for managing big data, detailing its components like the Hadoop Distributed File System (HDFS) and MapReduce framework. The content also emphasizes the increasing job demand for Hadoop-related skills and technologies in the big data industry.
Big Data Hadoop Training by Easylearning Guru
- 1. Welcome to the World of Big Data & Hadoop www.easylearning.guru
- 2. Agenda What is Big Data ? Different Kinds of Big Data Big Data Global Market Hadoop Global job trends What is Hadoop ? www.easylearning.guru
- 3. What is Big Data? Big data is the term for a collection of data sets so large and complex that it becomes difficult to process using on-hand database management tools or traditional data processing applications. www.easylearning.guru
- 4. Types of Big Data ? Traditional RDBMS deals with only Structured data. Semi-Structured Data Need of a technology which deals with Semi-structured data, Unstructured data and Structured data as well www.easylearning.guru
- 5. The 3V’s of Big Data www.easylearning.guru
- 6. Sources of Data Social Media & Networks (All of us are generating data) Mobile Devices (Tracking all the objects all the time) Sensor Technology & Networks (Measuring all kinds of data) Scientific Instruments (Collecting all sorts of data) www.easylearning.guru
- 7. Where Big Data is used ? www.easylearning.guru
- 8. Face book Scenario Facebook on an average generates 70 thousand MB in 1 minute. 1 hour = 70,000 MB *60 = 4.2 Million MB 1 Day = 4.2 Million *24 MB = 10.8 Billion MB = 98438 GB 1 week = 6.9 thousand GB = 690 TB 4 weeks = 690 TB * 4 = 2756 TB = 2.7 PB 52 weeks = 2.7 PB * 52 = 143.3 PB AŶd that’s aloooooooooot of data ! www.easylearning.guru
- 9. Various Bigdata Technologies www.easylearning.guru
- 10. Big Data Global Market Big Data Implementation Implemented Big Data Yet to Implement Big Data DATA SCIENTIST BIG DATA VISUAL IZER BIG DATA RESEARCH ANALYST Sources : Dice, LinkedIn. 60 50 40 30 20 10 0 2012 2013 2014 2015 2016 2017 Big Data Growth (in USD Billions) BIG DATA ENGINEER BIG DATA ARCHITECT BIG DATA ANALYST 50 44 43 31 23 18 50 56 57 69 77 82 Filled Unfilled FILLED/VACANCY(%) www.easylearning.guru
- 11. Hadoop Global Job Trends Top Hadoop Technology Companies Sources : Dice, LinkedIn. More than 17,000 employees with Hadoop skill across these companies www.easylearning.guru
- 12. DEMAND FOR BIG DATA IN CITIES 2% 2% 3% 4% 8% 8% 10% 11% 14% 38% As of February 2014 Hadoop Global Job Trends 120 100 80 60 40 20 0 SALARY (USD P.A. IN THOUSANDS) Sources : Dice, LinkedIn. www.easylearning.guru
- 13. What is Hadoop ? Hadoop was created by Doug Cutting and Mike Cafarella. Hadoop provides the reliable shared storage and analysis system. It is designed to scale up from a single server to thousand of machines, with a high degree of fault tolerance. www.easylearning.guru
- 15. Hadoop Core Components Core Hadoop has two main systems: • Hadoop Distributed File System: The Hadoop file system is a Distributed file system which holds the large amount of data across multiple nodes in a cluster. • MapReduce: MapReduce is a distributed programming paradigm used to analyze the data in the HDFS. www.easylearning.guru
- 16. Hadoop Distributed File System (HDFS) A given file is broken down into blocks (default=64MB), then blocks are replicated across cluster (default=3). Optimized for throughput. HDFS allows you to put/get/delete files. Follows the philosophy ͞Write OŶce aŶd Read Multiple tiŵes͟ Block Replication for: - Durability, High Availability and Throughput. www.easylearning.guru
- 18. MapReduce Framework Map Reduce works by breaking the processing into two phases : Map Phase and Reduce Phase. www.easylearning.guru
- 20. What we offer… www.easylearning.guru
- 22. Syllabus Introduction a)Big Data b)Hadoop Hadoop a)HDFS b)MapReduce PIG a)Pig 1 b)Pig 2 Hive a)Hive 1 b)Hive 2 Hbase Zookeeper Sqoop Yarn Project Class www.easylearning.guru
- 23. Thank you for watching the Live Demo for Hadoop. You can always contact us on: Phone : +91 124 4763660 (India) Email : contact@easylearning.guru Skype Id : easylearning.guru Website : www.easylearning.guru Your queries are always welcome. www.easylearning.guru