introduction to hadoop
Download as PPTX, PDF0 likes322 views
Hadoop is an open-source framework for reliable, scalable, distributed computing and data storage, designed to handle large and growing datasets. Developed by Doug Cutting and Michael J. Cafarella in 2005, and later contributed to the Apache Software Foundation, it facilitates processing using commodity hardware with features like fault tolerance and high availability. It is commonly used for applications in advertisement, search, and security.
introduction to hadoop
- 1. Apache top level project, open- source implementation of frameworks for reliable, scalable, distributed computing and data storage. It is a flexible and highly-available architecture for large scale computation and data processing on a network of commodity hardware.
- 2. Hadoop’s Developers Doug Cutting 2005: Doug Cutting and Michael J. Cafarella developed Hadoop to support distribution for the Nutch search engine project. The project was funded by Yahoo. 2006: Yahoo gave the project to Apache Software Foundation.
- 4. • Hadoop: • an open-source software framework that supports data-intensive distributed applications, licensed under the Apache v2 license. • Goals / Requirements: • Abstract and facilitate the storage and processing of large and/or rapidly growing data sets • Structured and non-structured data • Simple programming models • High scalability and availability • Use commodity (cheap!) hardware with little redundancy • Fault-tolerance • Move computation rather than data
- 7. Three main applications of Hadoop: • Advertisement (Mining user behavior to generate recommendations) • Searches (group related documents) • Security (search for uncommon patterns)