![TERAGRID RESOURCE PROVIDERS
• Indiana University - Big Red - IBM BladeCenter JS21 Cluster[7]
• Louisiana Optical Network Initiative (LONI)[8]
• National Center for Atmospheric Research (NCAR)
• National Center for Supercomputing Applications (NCSA)
• Pittsburgh Supercomputing Center (PSC) operated by University of
Pittsburgh and Carnegie Mellon University.
• San Diego Supercomputer Center (SDSC)
• Texas Advanced Computing Center (TACC)
11-03-2018Grid Computing Systems and Resource Management
33](https://image.slidesharecdn.com/finalppt2-180311064557/85/Grid-Computing-Systems-and-Resource-Management-33-320.jpg)
![REFERENCES
[1] Kai Hwang, Jack Dongarra Geoffrey Fox, Distributed and Cloud Computing: From Parallel
Processing to the Internet of Things, Morgan Kaufmann Publishers Inc. San Francisco,
CA, USA,1st Edition, eBook ISBN:9780128002049, Paperback ISBN: 9780123858801.
[2] https://www.nsf.gov/
[3] http://www.garudaindia.in/
[4] http://www.eu-datagrid.org/
[5] http://www.computerworld.com/article/2552339/networking/open-grid-
servicesarchitecture.html
[6] Hai Jin and Li Qi,” ChinaGrid and its Impact to Science and Education in China”, IEEE,
Dec. 2005
11-03-2018Grid Computing Systems and Resource Management
43](https://image.slidesharecdn.com/finalppt2-180311064557/85/Grid-Computing-Systems-and-Resource-Management-43-320.jpg)
Grid Computing Systems and Resource Management
- 1. GRID COMPUTING SYSTEMS AND RESOURCE MANAGEMENT Presented by: Souparnika Padaki Patil 11-03-2018 1 Grid Computing Systems and Resource Management
- 2. CONTENTS • Introduction • CPU Scavenging and Virtual Supercomputers. • Open Grid Services Architecture (OGSA). • Data-Intensive Grid Service Models. • National Grids and International Projects. • NSF TeraGrid in the United States. • DataGrid in the European Union. • The ChinaGrid Design Experiences. • Resource Management and Job Scheduling. • Grid Resource Monitoring with CGSP • Resource Brokering with Gridbus 11-03-2018Grid Computing Systems and Resource Management 2
- 3. INTRODUCTION • The idea of the grid was pioneered by Ian Foster, Carl Kesselman and Steve Tuecke in 2001. • The grid is a metacomputing infrastructure that brings together computers to form a large collection of compute, storage, and network resources. • The goal of grid computing is to explore fast solutions for large- scale computing problems. 11-03-2018Grid Computing Systems and Resource Management 3
- 4. CPU SCAVAGING AND VIRTUAL SUPER COMPUTERS It is the process of grid resource aggregation from local and remote sources. It consists of two parts: 1. CPU Scavenging and Virtual Supercomputers. 2. Virtual Organization 11-03-2018Grid Computing Systems and Resource Management 4
- 5. CPU SCAVAGING • The concept of creating a “grid” from the unused resources in a network of computers is known as CPU scavenging. • At present, many volunteer computing grids are built using the CPU scavenging model. • The most famous example is the SETI@Home, which applied over 3 million computers to achieve 23.37 TFlpos as of Sept. 2001 • In practice, these virtual grids can be viewed as virtual supercomputers. 11-03-2018Grid Computing Systems and Resource Management 5
- 6. EXAMPLES OF CPU SCAVAGING Fastest virtual supercomputers: 1. BOINC – 16.92 PFLOPS 2. SETI@Home 0.741 PFLOPS 3. GIMPS 0.313 PFLOPS 4. MilkyWay@Home 0.217 PFLOPS 11-03-2018Grid Computing Systems and Resource Management 6
- 7. VIRTUAL ORGNIZATION • The grid is a distributed system integrated from shared resources to form a virtual organization (VO). • The VO offers dynamic cooperation built over multiple physical organizations. • The virtual resources contributed by these real organizations are managed autonomously. 11-03-2018Grid Computing Systems and Resource Management 7
- 8. EXAMPLE OF VIRTUAL ORGNIZATION 11-03-2018Grid Computing Systems and Resource Management 8
- 9. FEATURES OF VIRTUAL ORGNIZATION 11-03-2018Grid Computing Systems and Resource Management 9
- 10. OPEN GRID SERVICE ARCHIECURE(OGSA) OGSA defines standards for • what Grid services are • what they should be capable of • what type of technologies they should be based on. • OGSA does not give a technical and detailed specification. They use WSDL 11-03-2018Grid Computing Systems and Resource Management 10
- 11. ARCHITECTURE OF OGSA It Comprised of 4 main layers • Grid Applications Layer • OGSA Architected Grid Services Layer ( core, program execution and data services) • Web Service Layer ( including OGSI) • Physical and Logical Resources Layer 11-03-2018Grid Computing Systems and Resource Management 11
- 12. OSGA FRAMEWORK 11-03-2018Grid Computing Systems and Resource Management 12
- 13. OPEN GRID SERVICES INFRASTRUCTURE (OGSI) • OGSA services are built around OGSI mechanism • OGSI specification defines grid services and builds upon web services. • OGSI is based on WSRF ( web services Resources Framework) • The Globus Toolkit is an implementation of OGSI 11-03-2018Grid Computing Systems and Resource Management 13
- 14. SERVICES IN THE WEB AND THE GRID OGSA, OGSI 11-03-2018Grid Computing Systems and Resource Management 14
- 15. 11-03-2018Grid Computing Systems and Resource Management 15 • OGSA: (The Definition) is the blueprints the architect creates to show how the building looks like. • OGSI: (The Specification) is the structural design that the engineer creates to support the architect's vision of the building. • GT: (The implementation) is the bricks, cement and beams used to build the building with the engineer's specifications.
- 16. DATA-INTENSIVE GRID SERVICE MODEL • Data intensive grid service models need to handle large volume of data. • So the grid systems designed must be able to discover, transfer, and manipulate these massive data sets • Desirable properties • Less time-consuming • Low storage costs • High-speed data movement 11-03-2018Grid Computing Systems and Resource Management 16
- 17. DATA-INTENSIVE GRID SERVICE MODEL… • Methods to handle data efficiently: • Data Replication (High availability) • Grid Data Access Models Monadic model Hierarchical model Federation model Hybrid model 11-03-2018Grid Computing Systems and Resource Management 17
- 18. DATA REPLICATION • Data caching : Data access operation in database is called caching • Data replication : same data is scattered and stored in multiple grid locations Users access data from multiple locations parallel based on the locality of reference ( Google search) • Benefits Data availability is improved One data storage becomes backup for another data storage • Two types of replication Dynamic Static 11-03-2018Grid Computing Systems and Resource Management 18
- 19. GRID DATA ACCESS MODELS Monadic model • Used when centralized data repository is required • All data stored in repository • Repository ( all data) is replicated within grid • To access data User submits request to central repository Permission given based on prior registration • Fault tolerance, performance, reliability very poor 11-03-2018Grid Computing Systems and Resource Management 19
- 20. Hierarchical model • Data centers designed as first, second…levels • First level data replicated to second level etc. • Data in each level accessed by its own grid users 11-03-2018Grid Computing Systems and Resource Management 20
- 21. Federation model • Multiple databases • Geographically distributed • Known as “mesh data access model” • Authorized specific location users can only access respective dB 11-03-2018Grid Computing Systems and Resource Management 21
- 22. Hybrid model • Combination of hierarchical and federation models • Needs high bandwidth network • Uses Grid FTP protocol 11-03-2018Grid Computing Systems and Resource Management 22
- 23. NATIONAL GRIDS AND INTERNATIONAL PROJECTS • Like supercomputers, national grids are mainly funded through government sources. • National grids are developed to promote: research discovery middleware products utility computing 11-03-2018Grid Computing Systems and Resource Management 23
- 24. NATIONAL GRID PROJECT • GARUDA(Global Access to Resource Using Distributed Architecture) is India's Grid Computing initiative connecting 17 cities across the country. • The 70 participating institutes in this nationwide project include all the IITs and C-DAC centers and other major institutes in India. • From April 2008 the Foundation phase is in progress with an aim to include more users’ applications. 11-03-2018Grid Computing Systems and Resource Management 24
- 25. GARUDA ACCESS MATHODS Anyone can access garuda in two ways: • Command line Interface: • Useful for Advanced Users • Requirements: Valid IGCA Certificate, Garuda User Account. • Web Interface. • A Web Interface for Job Submission. • Browse Resources. • Can Select the desired OS • http://192.168.60.40/GridPortal1.3/ 11-03-2018Grid Computing Systems and Resource Management 25
- 26. GARUDA PARTNERS Current Participation • Total of 45 institutions • 36 research & academic institutions in 17 cities • ERNET-HQ in Delhi • 8 centers of C-DAC 11-03-2018Grid Computing Systems and Resource Management 26
- 27. GARUDA COLLABORATIONS • Space Application Centre, Ahmedabad • Disaster Management • Grid Middleware for Satellite Grids • Indian Institute of Science, Bangalore • Climate Modeling • Indian Institute of Technology, Mumbai • Computational Fluid Dynamics Package on PARAM Padma • University of Pune • Quantum Chemistry, Materials Modeling, Bioinformatics • INDMOL Molecular Package developed 11-03-2018Grid Computing Systems and Resource Management 27
- 28. NATIONAL GRID PROJECTS OF DIFFERENT COUNTRIES • GridPP (UK) • CNGrid (China) • D-Grid (Germany) • GARUDA (India) • VECC (Calcutta, India) • IsraGrid (Israel) • INFN Grid (Italy) • National Grid Service (UK) • Open Science Grid (USA) • TeraGrid (USA) 11-03-2018Grid Computing Systems and Resource Management 28
- 29. INTERNATIONAL GRID PROJECTS • Grid applications cannot be restricted to geographical boundaries. • These projects promote volunteer computing, utility computing, and specific software applications that utilizes grid infrastructure. • International grids involve both government and industrial funding. • The European Union has been a major player in grid computing. 11-03-2018Grid Computing Systems and Resource Management 29
- 30. NSF TERAGRID IN UNITED STATES • TeraGrid was an e-Science grid computing infrastructure combining resources at eleven partner sites. • The project started in 2001 and operated from 2004 through 2011. • The TeraGrid resources included more than a petaflop of computing capability and more than 30 petabytes of online and archival data storage. • TeraGrid construction was also made possible through corporate partnerships with Sun Microsystems, IBM, Intel Corporation, Qwest Communications, Juniper Networks, Myricom, Hewlett-Packard Company, and Oracle Corporation. 11-03-2018Grid Computing Systems and Resource Management 30
- 31. TERAGRID HARDWARE COMPONENTS • High-end compute hardware • Intel/Linux clusters • IBM POWER3 and POWER4 clusters • SUN visualization systems • Cray XT3 • IBM Blue Gene/L • Large-scale storage systems • hundreds of terabytes for secondary storage • Visualization hardware • Very high-speed network backbone (40Gb/s) • bandwidth for rich interaction and tight coupling 11-03-2018Grid Computing Systems and Resource Management 31
- 32. TERAGRID ARCHITECTURE • TeraGrid resources are integrated through a service-oriented architecture. • Computational resources run a set of software packages called "Coordinated TeraGrid Software and Services" (CTSS). • TeraGrid uses a 10 Gigabits per second dedicated fiber-optical backbone network, with hubs in Chicago, Denver, and Los Angeles • CTSS provides a familiar user environment on all TeraGrid systems, allowing scientists to more easily port code from one system to another. • CTSS includes:- • Globus Toolkit • Account management software • Set of compilers • Programming tools • Environment variables 11-03-2018Grid Computing Systems and Resource Management 32
- 33. TERAGRID RESOURCE PROVIDERS • Indiana University - Big Red - IBM BladeCenter JS21 Cluster[7] • Louisiana Optical Network Initiative (LONI)[8] • National Center for Atmospheric Research (NCAR) • National Center for Supercomputing Applications (NCSA) • Pittsburgh Supercomputing Center (PSC) operated by University of Pittsburgh and Carnegie Mellon University. • San Diego Supercomputer Center (SDSC) • Texas Advanced Computing Center (TACC) 11-03-2018Grid Computing Systems and Resource Management 33
- 34. DATA GRID IN EUROPEAN UNION • The European DataGrid Project (EDG) was funded by the European Union. • This grid aims to build the next-generation, high-throughput, production-quality grid infrastructure. • It supports I/O-intensive experiments in high-energy physics, earth observation, and bioinformatics. • The system moves and replicates data at high speeds from one geographical site to another 11-03-2018Grid Computing Systems and Resource Management 34
- 35. EU DATA GRID PROJECT OBJECTIVE • Use Grid technology to develop a sustainable computing model for effective share of computing resources and data for large scientific communities • Collaborate with and complement other European and US projects • Contribute to Open Standards and international bodies 11-03-2018Grid Computing Systems and Resource Management 35
- 36. EU DATA GRID PROJECT PARTNERS • CERN – International (Switzerland/France) • CNRS - France • ESA/ESRIN – International (Italy) • INFN - Italy • NIKHEF – The Netherlands • PPARC - UK 11-03-2018Grid Computing Systems and Resource Management 36
- 37. ARCHITECTURE OF THE EUROPEAN DATA GRID 11-03-2018Grid Computing Systems and Resource Management 37
- 38. RESOURCE MANAGEMENT AND JOB SCHEDULING • In a grid system, resources are usually autonomous • Organizations set their own Resource Management System (RMS) have there own resource management policies. • RMS can be in upper as well as lower level of organization. • In upper level it is considered as resource consumer and in lower level as resource provider. 11-03-2018Grid Computing Systems and Resource Management 38
- 39. COMPONENTS OF RMS 11-03-2018Grid Computing Systems and Resource Management 39
- 40. GRID RESOURCE MONITORING WITH CGSP • In ChinaGrid the monitoring system is essential to keep complex distributed system efficient. • CGSP is a grid middleware developed for the construction and evolution of ChinaGrid. • CGSP is designed to be an adaptable, stream-integrated grid monitoring system. • CGSP guarantees the integrity and uniformity of ChinaGrid platform by a global monitoring system. • Function modules in CGSP are: • Portal • Grid Developing Environment • Information Center • Security 11-03-2018Grid Computing Systems and Resource Management 40
- 41. RESOURCE BROKERING WITH GRIDBUS • The resource broker takes care of the user desire to automate the search for resources rather than keep track of the previously known ones. • The resource broker adds a layer of abstraction by making direct user and resource interaction unnecessary and allowing job submission accessible and user friendly. 11-03-2018Grid Computing Systems and Resource Management 41
- 42. 11-03-2018Grid Computing Systems and Resource Management 42
- 43. REFERENCES [1] Kai Hwang, Jack Dongarra Geoffrey Fox, Distributed and Cloud Computing: From Parallel Processing to the Internet of Things, Morgan Kaufmann Publishers Inc. San Francisco, CA, USA,1st Edition, eBook ISBN:9780128002049, Paperback ISBN: 9780123858801. [2] https://www.nsf.gov/ [3] http://www.garudaindia.in/ [4] http://www.eu-datagrid.org/ [5] http://www.computerworld.com/article/2552339/networking/open-grid- servicesarchitecture.html [6] Hai Jin and Li Qi,” ChinaGrid and its Impact to Science and Education in China”, IEEE, Dec. 2005 11-03-2018Grid Computing Systems and Resource Management 43
- 44. IEEE Paper Hai Jin and Li Qi,” ChinaGrid and its Impact to Science and Education in China”, IEEE, Dec. 2005 The focus of this paper is mainly on current status of ChinaGrid project. The impact of ChinaGrid project. five different grid computing applications Bioinformatics Grid Computational Fluid Dynamics Grid Image Processing Grid Course Online Grid Massive Data Processing Grid 11-03-2018Grid Computing Systems and Resource Management 44
- 45. 11-03-2018Grid Computing Systems and Resource Management 45