Algo Project Proposal
1 like297 views
This project proposal aims to solve map-reduce problems using a distributed cluster of web browsers instead of dedicated servers. It proposes making web browsers thin clients that can perform computations in the background without requiring installation. The server would divide problems and assign parts to browser clients. This could utilize unused computational resources of personal systems and bandwidth. Key challenges include implementing map-reduce algorithms across clients and server, optimization strategies, and handling failures. The technology could allow large-scale information mining by delegating intensive tasks to client browsers without companies needing additional hardware.
![Project Proposal Solving map – reduce problems using a distributed cluster via http 2007095 -Ritesh M Nayak [email_address]](https://image.slidesharecdn.com/algo-project-proposal555/85/Algo-Project-Proposal-1-320.jpg)
Algo Project Proposal
- 1. Project Proposal Solving map – reduce problems using a distributed cluster via http 2007095 -Ritesh M Nayak [email_address]
- 2. Objective Solving map reduce problems by using a distributed cluster made up of many transient state nodes. Nodes are made of WYSIWYG browsers. Make a Proof of concept of the technology and package it into a library for generic usage.
- 3. The current technology Server Client Client Client Sockets(tcp)
- 4. Problems Clients are usually executables that have to be downloaded and installed Requires opening up of ports and firewalls which does not work in all settings(due to TCP) Examples are the @home project, SETI project and also the Google science project.
- 5. Solution Make the clients thin – remove the download and install overhead Use the resource that the browser uses like the javascripting engine which can do a lot of computation which is untapped. Make the entire process seamless without user intervention or even without the users knowledge
- 6. Inspiration People spend hours on the net. Home Pages, social networks, mail boxes remain open for long hours at times. During this time, the bandwidth and the computational capabilities of the system can be utilized to do server’s work ( if the user desires ) Bandwidth is becoming cheaper and unlimited which isnt much of a concern.
- 7. How it will look Server Http No firewalls Browser Browser Browser Server (Master) : will act as the master which will do the work of dividing the problem Browsers ( Slaves) : will form the computational node when connected to the server.
- 8. Problems to be solved Implementing map reduce algorithms on the server and client Calculating the problem size and deciding division strategies plus optimization. Selection algorithms for the most stable clients . Synchronization algos in case of critical section problems
- 9. Problems to be solved In case of problems that cannot be tackled via divide and conquer, parallelizing the serial code. Algos for error recovery, node failure, new node addition And tons more….
- 10. Technology Any technology that uses the web model (J2EE) Client side javascript or activex plugins(addons) Comet paradigm for async comm between server and client.
- 11. Technology Hurdles Current web models aren’t mature enough to handle such reqs, will require new way of looking at web apps (asynchrony and push technology). No known design patterns exist. Client side resources include js which is limited in its capabilities. Efficient js library has to be developed.
- 12. Prospects With concepts like semantic web, large scale information mining becoming reality, this technology can give a great edge to companies and researchers alike who can delegate processor intensive computations as small computations to clients. Companies don’t need to invest in more hardware. A new paradigm of programming where the browser acts as a server.
- 13. Examples to learn from Yahoo and Apache’s Hadoop technology which solves the map reduce on distributed servers. Amazon’s EC2 computational facility. Tons of material on grid and cluster computing.