Hadoop content
1 like764 views
This 40-hour course provides training to become a Hadoop developer. It covers Hadoop and big data fundamentals, Hadoop file systems, administering Hadoop clusters, importing and exporting data with Sqoop, processing data using Hive, Pig, and MapReduce, the YARN architecture, NoSQL programming with MongoDB, and reporting tools. The course includes hands-on exercises, datasets, installation support, interview preparation, and guidance from instructors with over 8 years of experience working with Hadoop.
Hadoop content
- 1. COURSE: HADOOP DEVELOPER DURATION: 40 HOURS www.beinghadoop.com 1
- 2. COURSE: HADOOP DEVELOPER DURATION: 40 HOURS www.beinghadoop.com 2 HADOOP BIG DATA TRAINING Big Data is huge amount of structured or unstructured data, to process this data using traditional methods having a problem of execution time and cost, and needs to process Bytes of data from different locations. The Big Data technologies can do this work four or more times faster than traditional data warehousing approaches Apache Hadoop is an open source framework designed to maintain huge amount of data. It helps organizations to analyse huge volumes like data size in Peta bytes. The data may be structured or unstructured. Hadoop scales virtually without limits and handles any type of data. No matter how data is encoded or formatted and structured or unstructured. Hadoop is Enterprise proven, already delivering on the promise of Big data across numerous industries - Cloudera Apache Hadoop is an open source framework that supports data intensive distributed computing application to process huge amount of data. It is licensed under Apache v2 license. It supports the running of applications on large clusters of commodity hardware - En.wikipedia We have designed this course to provide information and technical skills to become successful Hadoop developer. About certification: Cloudera delivers a developer-focused certification program CLOUDERA CERTIFIED HADOOP DEVELOPER (CCDH) CDH-410 that closely analyses Hadoop’s structure and provides hands-on exercises that teach you how to import data from existing sources; process data with a variety of techniques such as Java Map Reduce programs and Hadoop Streaming jobs; and work with Apache Hive and Pig
- 3. COURSE: HADOOP DEVELOPER DURATION: 40 HOURS www.beinghadoop.com 3 Training content: MODULE 1: HADOOP AND BIG DATA CHAPTER 1: INTRODUCTION TO BIG DATA In This Chapter you will understand what Big Data, Hadoop is Sub-Projects, How Big Data Analytics used to analyse data. Big data and Big Data Analytical services What is Hadoop? History of Hadoop Hadoop Eco-Systems Hadoop Framework Hadoop Distribution Hadoop in Retail, Healthcare, Telecom, etc., use cases…, CHAPTER 2: HADOOP DISTRIBUTED FILE SYSTEMS In this chapter you will understand the difference between Local and Hdfs file system, identifying the Nodes; how to check the status of job, you can able to identify file locations and how user request or job processed. Hdfs design Hadoop file systems Hdfs Daemons Master and Slave node architecture Hdfs webUI How to load the data into HDFS File system API –java Serialization in Hadoop – Reading and writing files from/to Hadoop URL
- 4. COURSE: HADOOP DEVELOPER DURATION: 40 HOURS www.beinghadoop.com 4 CHPTER 3: ADMINISTERING HADOOP CLUSTER: In this chapter you will learn how to install Hadoop in production environment and practice environment. You can able to modify the Hadoop configuration files. Cluster specification, Hadoop cluster setup and installation --standalone --pseudo distributed mode --fully distributed mode Step-by-step multiNode installation Hadoop Configurations nameNode and dataNode directory structure User commands: fs, fsck,distcp,archive, ----- Administration commands dfsadmin, balancer,jobtracker,tasktracker,namenode---- MODULE 2: DATA IMPORTING AND EXPORTING USING SQOOP CHAPTER 4: WORKING WITH SQOOP In this chapter you will understand how to import data from Rdbms to Hdfs. How to get the data from Mysql tables to Hadoop Hdfs, how to perform import and export between the data stores, Sqoop installations Sqoop1 vs Sqoop2 Sqoop commands Sqoop connectors for oracle, Mysql, Teradata and other Rdbms Importing the data from Mysql with all possible imports Exporting the data from HDFS to Rdbms Creating hive tables by importing data
- 5. COURSE: HADOOP DEVELOPER DURATION: 40 HOURS www.beinghadoop.com 5 MODULE 3: DATA PROCESSING USING HIVE, PIG AND MAPREDUCE CHAPTER 5: HIVE In this chapter you will understand what is Hive, how to load the data from Hdfs or local file system to hive tables, How to query for large volumes of data Hive concepts Hive installation Hive configuration, Hive services Hive query processing Hive Metastore configurations Hive data types - primitive and complex types Hive operators Hive Built-in functions Hive Tables --creating tables --partitions and buckets --browsing tables and partitions --storage formats --loading data --joins --aggregations and sorting --insert into local files --altering, dropping tables --importing data Word count in Hive Hive Compression Hive Security
- 6. COURSE: HADOOP DEVELOPER DURATION: 40 HOURS www.beinghadoop.com 6 CHAPTER 6: PIG In this chapter you will understand what data flow is, how pig is tightly coupled with map reduce. How efficiently you can retrieve the data from data store Why pig? Pig installation Pig Latin commands Pig Latin relational operators Pig Latin diagnostic operators Data types and Expressions Built-in functions Data processing in pig --load and store Filtering the data Grouping the data Joining the data Sorting the data Word count in Pig CHAPTER 7: MAPREDUCE In this chapter you will get an idea of how Map-Reduce framework works. How to create and run a map reduce program MapReduce Overview and Architecture Developing MapReduce Jobs Word count program in MapReduce Input and Output Formats Job Configuration Packaging, Launching, Debugging jobs Job Submission MapReduce workflows MapReduce types Input and Output Formats Running MapReduce jobs in Eclipse Practicing MapReduce Programs
- 7. COURSE: HADOOP DEVELOPER DURATION: 40 HOURS www.beinghadoop.com 7 MODULE 4: YARN ARCHITECTURE CHAPTER 8: NEW MAPREDUCE (MRV2) Yarn: Yet Another Resource Negotiator Hadoop YARN - a resource-management platform responsible for managing compute resources in clusters and using them for scheduling of users' applications. Yarn Architecture Yarn Configurations Yarn and MapReduce 2.0 Daemons Yarn New MapReduce API Yarn command line tools MODULE 5: NOSQL PROGRAMMING CHAPTER 9: MONGODB Introduction to NoSql Programming MongoDB Overview Installing mongoDB on Windows, Linux, Mac OS communication between Hadooop to mongoDB mongoDB management services MongoDB Programming - creating Databases -Creating Documents, adding Data, Retrieving data from Documents Update,Delete data in mongoDB -Datatypes,Arrays, Regular Expressions CHAPTER 10: OTHER HADOOP SUBPROJECTS Hbase introduction Oozie basics Working with Impala.
- 8. COURSE: HADOOP DEVELOPER DURATION: 40 HOURS www.beinghadoop.com 8 MODULE 6: HADOOP BI/REPORTING TOOLS CHAPTER 11: REPORTING TOOLS Generating Reports using Pentaho BI Tools : Reporting on Hdfs file data, Reporting on Hive, Hbase data Jaspersoft: Generating reports using iReportdesigner CHAPTER 12: HADOOP CLUSTER MONITORING TOOLS: Working with Cloudera Manager Hadoop cluster monitoring using Ganglia Course Highlights: 8+ years of Experience instructors Less theory, More Practical Environment Hadoop Installation support on Personal computers Datasets, Programs to practice Notes on Each topic Post training Guidance and Support Interview questions, Resume preparation Course Duration 40 Hours Each Session Duration: 60 to 90Mins Five sessions in a Week Session Environment on GotoWebinar or Webex Practical Environments: Vmplayer with Hadoop 0.20, Oracle VirtualBox Hadoop 2.0 Yarn for MapReduce2, cloudera Quickstart vm CDH4.0 AJAY KUMAR www.beinghadoop.com hadoopframework@gmail.com https://www.facebook.com/hadoopinfo http://www.slideshare.net/hadoopframework
- 9. COURSE: HADOOP DEVELOPER DURATION: 40 HOURS www.beinghadoop.com 9