Map Reduce in Hazelcast - Hazelcast User Group London Version
2 likes1,786 views
The document discusses the use of MapReduce in Hazelcast for big data processing, highlighting its in-memory data grid capabilities and distributed computing features. It covers the basic workflow of MapReduce, including mapping, combining, and reducing data, as well as various use cases like log analysis and data querying. Additionally, it emphasizes the advantages of automatic partitioning and fault tolerance that Hazelcast offers for high-speed data operations.
![Multiple values are combined to an
intermediate resultto preserve traffic
SOME PSEUDO CODE (2/3)
COMBINING
combine(word:Word,counts:List[Int]):Void->
emit(word,sum(counts))
www.hazelcast.com](https://image.slidesharecdn.com/london-140612025020-phpapp01/85/Map-Reduce-in-Hazelcast-Hazelcast-User-Group-London-Version-25-320.jpg)
![Values are reduced /aggregated to the requested result
SOME PSEUDO CODE (3/3)
REDUCING
reduce(word:String,counts:List[Int]):Int->
returnsum(counts)
www.hazelcast.com](https://image.slidesharecdn.com/london-140612025020-phpapp01/85/Map-Reduce-in-Hazelcast-Hazelcast-User-Group-London-Version-26-320.jpg)
![FOR MATHEMATICIANS
Process: (KxV)*→(LxW)* ⇒ [(l1,w1),…,(lm,wm)]
Mapping: (KxV) →(LxW)* ⇒ (k,v) →[(l1,w1),…,(ln,wn)]
Reducing: LxW*→X* ⇒ (l,[w1,…,wn]) →[x1,…,xn]
www.hazelcast.com](https://image.slidesharecdn.com/london-140612025020-phpapp01/85/Map-Reduce-in-Hazelcast-Hazelcast-User-Group-London-Version-27-320.jpg)
Map Reduce in Hazelcast - Hazelcast User Group London Version
- 1. BIG DATA - FAST DATA USING MAPREDUCE IN HAZELCAST Source: www.hazelcast.com
- 2. Christoph Engelbert(@noctarius2k) 8+ years of JavaWeirdoness Performance, GC, traffic topics Apache Committer Gaming, TravelManagement, ... CastMapRMapReduce for Hazelcast3 www.hazelcast.com
- 5. WHAT IS HAZELCAST? In-MemoryData-Grid DataPartioning(Sharding) JavaCollections Implementation Distributed ComputingPlatform www.hazelcast.com
- 6. WHY HAZELCAST? Automatic Partitioning FaultTolerance Sync /Async Backups FullyDistributed In-Memoryfor HighestSpeed www.hazelcast.com
- 12. DISTRIBUTED COMPUTING OR MULTICORE CPU ON STEROIDS www.hazelcast.com
- 13. THE IDEA OF DISTRIBUTED COMPUTING Source: www.hazelcast.com
- 15. MULTICORE IS NOT NEW Source: www.hazelcast.com
- 19. MAP & REDUCE THE BLACK MAGIC FROM PLANET GOOGLE www.hazelcast.com
- 20. USE CASES LogAnalysis DataQuerying Aggregation Distributed Sort ETL (ExtractTransform Load) and more... www.hazelcast.com
- 22. FULL STEPS Read Map /Transform Combine Group /Shuffle Reduce Collate www.hazelcast.com
- 24. Dataare mapped /transformed in asetof key-value pairs SOME PSEUDO CODE (1/3) MAPPING map(key:String,document:String):Void-> foreachw:Wordindocument: emit(w,1) www.hazelcast.com
- 25. Multiple values are combined to an intermediate resultto preserve traffic SOME PSEUDO CODE (2/3) COMBINING combine(word:Word,counts:List[Int]):Void-> emit(word,sum(counts)) www.hazelcast.com
- 26. Values are reduced /aggregated to the requested result SOME PSEUDO CODE (3/3) REDUCING reduce(word:String,counts:List[Int]):Int-> returnsum(counts) www.hazelcast.com
- 27. FOR MATHEMATICIANS Process: (KxV)*→(LxW)* ⇒ [(l1,w1),…,(lm,wm)] Mapping: (KxV) →(LxW)* ⇒ (k,v) →[(l1,w1),…,(ln,wn)] Reducing: LxW*→X* ⇒ (l,[w1,…,wn]) →[x1,…,xn] www.hazelcast.com
- 28. MAPREDUCE PROGRAMS IN GOOGLE SOURCE TREE Source: www.hazelcast.com