Introduction to SARA's Hadoop Hackathon - dec 7th 2010
0 likes712 views
This document summarizes an agenda for the SARA Hadoop Hackathon on December 7, 2010. It provides background on Hadoop and how it relates to earlier technologies like Nutch and MapReduce. It then outlines the agenda for the day which includes introductions, presentations on MapReduce at University of Twente and a kickoff for the hackathon project building period. An optional tour of the SARA facilities is also included. The day will conclude with presentations of hackathon results.
Introduction to SARA's Hadoop Hackathon - dec 7th 2010
- 1. SARA Hadoop Hackathon Evert.Lammerts@sara.nl December 7, 2010
- 2. DJOERD HIEMSTRA (UTwente) EDGAR MEIJ (UvA) SARA Hadoop Hackathon, December 7, 2010
- 3. 2002 2004 2006 Nutch* MR/GFS** Hadoop * http://nutch.apache.org/ ** http://labs.google.com/papers/mapreduce.html http://labs.google.com/papers/gfs.html SARA Hadoop Hackathon, December 7, 2010
- 4. 2010: A Hype in Production http://wiki.apache.org/hadoop/PoweredBy SARA Hadoop Hackathon, December 7, 2010
- 5. Super computing Cloud computing Grid computing Cluster computing GPU computing http://www.sara.nl/ SARA Hadoop Hackathon, December 7, 2010
- 6. :-( Data Expensive! Computation :-) Data Cheaper! Computation Ref: Luiz André Barroso and Urs Hölzle, Google Inc. The Datacenter as a Computer: An Introduction to the Design of WarehouseScale Machines SARA Hadoop Hackathon, December 7, 2010
- 7. NameNode JobTracker DN TT DN TT DN TT DN TT DN TT DN TT DN TT DN TT DN DataNode TT TaskTracker SARA Hadoop Hackathon, December 7, 2010
- 8. File Map Shuffle Reduce Output $ echo “${email#*@}, ${name}” $ sort $ wc l ewi.utwente.nl, 1 gmail.com, 2 nbic.nl, 1 nikhef.nl, 3 sara.nl, 1 SARA Hadoop Hackathon, December 7, 2010
- 9. From: Hadoop, The Definitive Guide (2nd Edition), Tom White SARA Hadoop Hackathon, December 7, 2010
- 10. Today 09.30 - 09.50 Welcome & Introduction 09.50 - 10.15 Map/Reduce @ University of Twente 10.15 - 10.30 Kick-off hackathon 14.00 - 15.00 Optional: SARA tour 10.30 - 17.00 Hackathon 17.00 - 17.30 Results and closing SARA Hadoop Hackathon, December 7, 2010