HAdoop presentation
Download as PPT, PDF1 like1,056 views
This document provides an overview of the Hadoop framework. It describes the key components of Hadoop including the NameNode, DataNodes, JobTracker, TaskTracker, and SecondaryNameNode. The NameNode manages file metadata and location information stored across DataNodes. The JobTracker schedules and tracks jobs on TaskTrackers running on slave nodes. The SecondaryNameNode helps recover metadata if the NameNode fails. Hadoop uses a master/slave architecture with the NameNode and JobTracker on the master and DataNodes and TaskTrackers on slave nodes.
HAdoop presentation
- 1. Hadoop Edit by Cassell Hsu 2013.04.19
- 2. Hadoop Master Slave Slave Slave
- 3. Hadoop • Master • NameNode • JobTracker • SecondaryNameNode
- 4. NameNode • Where is NameNode? • Master • HDFS(Hadoop Distributed File System) • What is NameNode? • 資料之位置資訊 ？？ • 資料之屬性
- 5. NameNode • 位置資訊? • 所有資料皆存放在 – DataNode
- 6. DataNode • What is DataNode • 存放資料 • Where is DataNode • HDFS • Slaves (and Master)
- 7. UUsseerr DDaattaaNNooddee NNaammeeNNooddee DDaattaaNNooddee DDaattaaNNooddee A 64Mb 128 Mb B 64Mb A 64Mb B 64Mb Check hdfs-site.xml
- 8. Hadoop Master NNaammeeNNooddee DDaattaaNNooddee Slaves DDaattaaNNooddee DDaattaaNNooddee DDaattaaNNooddee
- 9. Hadoop • Master • NameNode • JobTracker • SecondaryNameNode
- 10. JobTracker • What is JobTracker? • 排程工作 • Where is JobTracker? • Master 誰來工作？
- 11. JobTracker & TaskTracker JobTracker TaskTracker Where Master Slaves What 排程工作執行工作
- 12. Hadoop Master NNaammeeNNooddee DDaattaaNNooddee Slaves DDaattaaNNooddee JJoobbTTrraacckkeerr TTaasskkTTrraacckkeerr
- 13. Hadoop • Master • NameNode • JobTracker • SecondaryNameNode
- 14. SecondaryNameNode • What is SecondaryNameNode? • NameNode發生錯誤時補救 • Where is SecondaryNameNode? • Master
- 15. Hadoop Master NNaammeeNNooddee DDaattaaNNooddee Slaves DDaattaaNNooddee JJoobbTTrraacckkeerr TTaasskkTTrraacckkeerr SSeeccoonnddaarryyNNooddee
- 16. MapReduce HDFS Final Result User Master Reduce Slave1 Slave2 A A1 A2 Task NameNode Task Result1 Result2
- 17. MapReduce • 檔案切割 • Hadoop上區塊切割 • 程式指定