Grid proxy architecture for network resources
0 likes62 views
The document outlines a patent application for a grid proxy architecture (GPAN) developed to enable grid applications to access and utilize resources in communication networks without needing to install grid software on every network device. It facilitates the interaction between grid service overlays and network service overlays, allowing for resource discovery, reservation, and management through a unique grid node that operates the GPAN proxy. This innovation aims to enhance resource sharing in grid networks, potentially reducing costs and improving efficiency in distributed computing tasks.
Grid proxy architecture for network resources
- 1. US 2015.0058490A1 (19) United States (12) Patent Application Publication (10) Pub. No.: US2015/0058490 A1 Wang et al. (43) Pub. Date: Feb. 26, 20159 (54) GRID PROXY ARCHITECTURE FOR Publication Classification NETWORK RESOURCES (51) Int. Cl. (71) Applicant: Microsoft Technology Licensing, H04L 12/24 (2006.01) Redmond, WA (US) (52) U.S. Cl. CPC .................................. H04L 41/0803 (2013.01) (72) Inventors: PhilWang, Nepean (CA); Indermohan USPC .......................................................... 709/226 Monga,Acton, MA (US); Tal Lavian, Sunnyvale, CA (US); Ramesh Durairaj, (57) ABSTRACT Santa Clara, CA (US); Franco Travostino, Arlington, MA (US) Various embodiments allow Grid applications to access resources shared in communication network domains. Grid ProXV Architecture for Network Resources (GPAN) bridges(21) Appl. No.: 14/532,990 y 9. Grid services serving userapplications and networkservices (22) Filed: Nov. 4, 2014 controlling network devices through proxy functions. At times, GPAN employs distributed network service peers Related U.S. Application Data (NSP) in networkdomains to discover, negotiate andallocate (63) Continuation ofapplication No. 13/725,646, filed on network resources for Grid applications. An elected master Dec. 21, 2012, now Pat. No s 898 274 which is a NSP is the unique Grid node that runs GPAN and represents - . . . s . c. • Y-sy--- Y- s1 is the whole network to share network resources to Grids with continuation ofapplication No. 13/295.283, filed on - 0 Nov. 14, 2011, now Pat. No. 8.341,257, which is a outGridinvolvement ofnetworkdevices. GPAN providesthe . . . . s or . . . . . . . Grid Proxy service (GPS) to interface with Grid services andcontinuation ofapplication No. 11/018,997, filed onDec. 21, 2004, now Pat No. 8,078708 applications, and the Grid Delegation service (GDS) to inter • 1- us s vs. vs V vs w . face with network services to utilize network resources. In (60) Provisional application No. 60/536,668, filed on Jan. Some cases, resource-based XML messaging can be 15, 2004. employed forthe GPAN proxy communication. Computing 102 120 100 GridManagement t <''1. Computing 104 : - - - - - - - || GRs E.Grid info RMS E RDs ar
- 2. US 2015/0058490 A1Feb. 26, 2015 Sheet 1 of3PatentApplication Publication 9}{2}}}__------->|-zzixpui ––––––-?* Ly.)|
- 3. US 2015/0058490 A1Feb. 26, 2015 Sheet 2 of3PatentApplication Publication •***************••••••••••••••••••••••••••••••••••••• No.?L?||<—————— }?!“º”No.uuos*……………………………………………..:t?íqŠN.^(sebessew...********************•••••••••••••••••••••••••***••••••••••••••••••••••••••••••••••
- 4. US 2015/0058490 A1Feb. 26, 2015 Sheet 3 of3PatentApplication Publication d{SAI40)S2014-19S3/40419AN Z09
- 5. US 2015/0058490 A1 GRD PROXY ARCHITECTURE FOR NETWORK RESOURCES RELATED APPLICATIONS 0001. This application is a continuation of, and claims priority to, U.S. patent application Ser. No. 13/725,646, filed Dec. 21, 2012, which is acontinuation of, andclaimspriority to, U.S.patentapplicationSer. No. 13/295,283, filedNov. 14, 2011, entitled GRID PROXY ARCHITECTURE FORNET WORK RESOURCES. U.S. patent application Ser. No. 13/295.283 is a continuation of, and claims priority to, U.S. patent application Ser. No. 11/018,997, filed Dec. 21, 2004, entitled GRID PROXY ARCHITECTURE FORNETWORK RESOURCES, which claims priority to U.S. Provisional Patent Application Ser. No. 60/536,668 entitled GRID PROXY ARCHITECTURE FOR NETWORK RESOURCES, filed Jan. 15, 2004, the entireties ofeach are incorporated herein by reference. BACKGROUND 0002 Grid networks are emerging as the “killer applica tion of next-generation networks through the building of overlay networks upon existing network infrastructures for thepurposes ofnetwork-wide computingand data collabora tion. In a Grid network, which is also known as a virtual organization, Grid nodes are distributed widely in the physi cal networks and share their available hardware and software resources such as CPU, storage, data mining, and visualiza tion centers. The resource sharing is actually providedby the Grid services running on Grid nodes. Those Grid services form the Grid overlay over the Grid nodes as they function under the same Grid rule. 0003. According to theavailable servicesand resources,a Griduserdistributes portionsofhis/herapplicationtaskinthe Grid nodes and puts those services and resources togetherto complete the usertask with very high performance. Benefits ofthis technology includepreventing resource wasting in the network and saving the user capital expenditure on equip ment. Typical use cases ofGrids include data-intensive and computation-intensive business tasks such as enterprise con current product design, utility computing, large-scale data synchronization, and many high-performance computations including nuclear energy simulations and long-term global weather forecasting. 0004 Distributed computing in general, and grid comput ing in particular, is desirablebecause complexand resource intensive computingtasks can beaccomplished withoutpur chasing and maintaining costly Super-computers. Rather, relatively smallportions ofthe overall computingtaskcanbe distributed among multiple computers and devices of rela tively modest capability. It will be appreciated that the grid networkcapability mayalsobemoreeasilyscalablethanhigh performancedevices and Super-computers. Further,thereis a potential business model for the rental ofgrid services. 0005. ThedispersedGrid nodesareconnectedby network infrastructure such as the Internet. Grid nodes include com puters, dataserversand networkdevices. Resourcesharingof network devices in a Grid is critical because it provides the interconnectivityofGrid nodes toform apipelineofresource supply in the Grid. In other words, interconnectivity enables reliable sharing of resources such as computing power and data services. Feb. 26, 2015 0006. One problem hindering the advance ofGrid tech nology isthatthe requisiteresourcesharingofnetwork nodes is not entirely supported by traditional networks. First, Grid nodes generally requirea piece ofGrid software installed on each Grid node. While this installation is relatively easy on a computer, it can be impractical on network devices such as routers and switches. Without installation of the Grid soft ware,a networkdevicecannotpurposefully makeitsresource availabletotheGrid,andaGridnodecannotaccessanetwork device through the same Grid rule. 0007 Another problem hindering the advance of Grid technology is that network devices are typically located in autonomous network domains, and controlled by network management facilities such as network managers and Ser vices. The network management facilities form an actual overlay network which is the network service overlay and gives the API for network control functions. In other words, the networkserviceoverlay consists ofnetworkservices that are running on network devices. Thus, in a Grid network, there are two overlay networks: the network service overlay andtheGrid serviceoverlay. But there is a gap between these two overlays. The gap isthattheyare notcompatiblebecause networkservicesandGridservices use differentservice infra structures. Moreover, due to the modest capability ofCPU and memory on network devices, network services are gen erally implemented in lightweight network software while Gridservicesaregenerallyimplementedinheavyweightsoft ware and thus require much more CPU powerand storage. 0008. It would therefore be desirable to have a technique forcausingthetwo overlaysto worktogether. In otherwords, network nodes should be capable ofacting as Grid nodes in order to provide its resources such as network links and bandwidth in a Grid. SUMMARY 0009 Various embodiments overcome the above-men tionedandotherdrawbacks byproviding a Grid-basedproxy mechanism to allow Grid services andapplications to access and utilize resources available in the physical network domains. In accordance with one or more embodiments, a Grid Proxy Architecture for Network Resources (“GPAN) bridges a grid services overlay that includes ofGrid services to serve user applications, with a network Services overlay thatincludes networkservices to control the network devices. The GPAN proxy gives the Grid services in the GridService overlay the potential to access the network services in the Network Serviceoverlay in order to control the networks for resource use. With GPAN, network nodes do not need to install and run a piece ofGrid service software in order to provide their resources to the Grids. Network nodes are rep resented in a whole by the GPANproxy andjoined the Grids by a unique Grid node which runs theactual proxy software/ system. 0010. In at least one embodiment, GPAN works with the networkservice overlay to gather networkresourceinforma tion and provide that information to the Grid resource index service within the Grid network. GPAN may also facilitate brokerage and reservation of network resources upon the request of a Grid resource broker service within the Grid network by translating application requests from the Grid services to networkoperations ofthe network services. 0011. At least one embodiment employs network service peers/agents (“NSPs') distributed in the network domains in order to obtain network resource information and allocate
- 6. US 2015/0058490 A1 network resources for the Grids. Each NSP may include an individual network domain, including various Sub-nets, and maybeoperatedby a differentserviceprovider.Ofthe NSPs, only the master NSP executes the GPAN proxy in a Grid network. The master NSP is usually selected from an NSP which is close to the location where run Grid resource ser vices suchas index, brokerandscheduler.A masterNSPmay beelectedbasedon how theGridis organizedandhoweasily the Grid accesses the network resources. Under the GPAN proxy, the master NSP sends network instructions to each NSP for resource operations such as information collection and allocation. Further, the masterNSPprovides Grid-based proxyAPIs ofresourceoperations to Gridservices andappli cations.Consequently,thenoderunningthemasterNSPisthe unique Grid node representingthe whole networks.TheGrid services and applications talk to this master NSP node for networkresource supply. TheyusetheGPANAPIswhichare provided in the form ofGrid services to access the network resources. Itwill thereforebeapparentthat noactual network nodes need to execute native grid protocols. 0012. In atleastoneembodiment, GPAN provides atleast two proxy functions, resource data proxy and resource man agementproxy.Theresourcedataproxyfunctionistoprovide resource discovery, status (create?destroy/use), availability and service policy for the Grid network. The resource man agement proxy function provides resource scheduling, new/ allocate/reallocate/release in the Grid network. GPAN sends resource information to the global Grid resource services such as index, through the resource data proxy. GPAN receivesresourceallocationrequestsofGridapps fromaGrid resource broker or scheduler service through the resource managementproxy. Each proxy function may implementdif ferent proxy mechanisms. For example, the resource data proxy implements a network information provider (“NIP') which is a Grid-based software to facilitate collection of network information and provide updates and reports to the Gridresourceindexservice.Theresourcemanagementproxy function implements a resource allocation and management provider (“RAMP) which is also a Grid-based software to receive resource allocation requests from the Grid broker/ meta-schedulerservices and schedule resource operations in the communications network. 0013. In at least one embodiment, GPAN includes two main components: a Grid Proxy service (“GPS) and a Grid Delegationservice(“GDS”). GPSinterfacestheGPANproxy with the Grid overlay ofGrid services to meet applications. GDS interfaces the proxy with the Network Service overlay of network services to utilize network resources. Resource based XML messaging may be utilized for communication between GPS and GDS. 0014. By implementing at least some of the features described above, GPAN advantageously causes the commu nications network to appear as grid resources from the per spectiveofthegrid network. Forexample, the resourceavail ability data is periodically gathered from each network servicepeerandprovidedto an index ofthegrid networkin a format that is compliant with the protocols utilized by the index. Further, GPAN implements an API that is compliant with the signaling utilized by the grid manager in order to facilitate resource brokerage and reservation. Following receipt ofa request from theGrid application, and identifica tion of requisite resource from the index, a broker/meta schedulerofthegridnetworksignalstothegridresourcesand GPAN in order to reserve and utilize the resources. The bro Feb. 26, 2015 ker/meta-scheduler signals for resource reservation to both thegridresourcesandGPAN inthesame,grid-standardcom pliant manner. GPAN implements anAPI which is operative to translate commands from the broker/meta-scheduler into a format that may be utilized by the communications network nodes. Consequently, two-way communications between the network resources and grid manager are possible, thereby enabling operations such as bandwidth allocation, negotia tion, agreement, information feedback, and status tracking. Once the appropriate communications networkandgrid net work resources are made available for the application, por tions ofthe overall task are distributed to the grid resource services either directly by the broker/meta-scheduler or via GPAN. BRIEF DESCRIPTION OF THE FIGURES 0015 FIG. 1 isa block diagram ofgrid networkarchitec ture illustrating gathering of communications network resource information for the grid resource index by GPAN. 0016 FIG. 2 illustrates reservation of communications network resources in the architecture of FIG. 1. (0017 FIG. 3 is a block diagram that illustrates the GPAN proxy in greater detail. DETAILED DESCRIPTION 0018 Referring to FIG. 1, a grid computing network architectureincludes multiple grid resources thatare in com munication via multiple communications networkelements. Thegridresources may includeagridresourcemanager100, computing resource provider 102, 104 and storage resource provider 106. The grid manager, computing resource pro vider and storage resource provider execute Grid-based Resource Services (GRS) such as Resource Management Services (RMS) and Resource Data Services (RDS). The communicationsnetworkelementsincludevariousSwitching and routing devices which make up network services peers (“NSPs') 108-114, and a master network service peer 116 that implements a Grid Proxy Architecture for Network Resources (“GPAN') 118 for the grid network. An applica tion device 120 which originatesarequestforexecution ofan application on the grid communicates with the grid network through the grid manager 100. 0019. The grid manager 100 manages grid-enabled resources shared in the Grid network to user applications. It may usean indeX node 122 which is operativeto executegrid computing protocols to produce an index of available grid resources. Forexample,theindex may includeaddress infor mationandan indication ofavailability foreachgridresource through respective RDS on each resource provider. It may also use a gridbroker/meta-scheduler 124 which is operative in response to a request from the application node 120 to identify a set ofgrid resources from the index with which to satisfy theapplication request. Once the set ofgrid resources is determined, the broker/meta-scheduler signals the grid resourcestopreparethemtobeutilized. Forexample,thegrid resources may be verifiedasavailable, reserved,and charges forthe services may bearranged. Oncethegrid resourcesare prepared, thebroker/meta-scheduler 124 distributes portions of the overall task to individual ones of the grid services through respective RMS in each resource provider. The bro ker/meta-scheduler is also operative to coordinate responses from the grid services 102-106 for the particularapplication task.
- 7. US 2015/0058490 A1 0020. The GPAN 118 is executed at least in-part by the masternetwork service peer 116and is operative tocausethe communications networkto appear, relativeto thegrid man ager 100, as a grid-compliant resource provider ofnetwork. One function ofthe GPAN 118 is to gathercommunications network resource information on behalf of the index 122. Data indicatingthe availability ofnetwork resources Such as NSPs 108-114 canbeactively gatheredusingany ofa variety of network protocols, such as simple network management protocol (“SNMP). Alternatively, some network devices may be configured to automatically provide resource avail ability data to the GPAN. In addition to obtaining resource availability data, the GPAN 118 generates a mapping ofthe available resources, including the various communications network nodes and links which interconnect the grid resources.Theresourceavailabilityinformationandmapping are then translatedinto a format thatcan be processed by the index 122,andtransmittedto theindexforstorage.Theinfor mationintheindexcanthenbeprocessedby thebroker/meta scheduler 124 in a Substantially similar manner as normally generated and stored grid resource information. 0021 Referringnow to FIG. 2, the GPAN 118also imple ments its Grid service API so that it can be accessed by the grid manager 124though standardGrid service invocation in order to facilitate resource brokerage and reservation. Fol lowing receipt of a request from the application 120 and identification of requisite resource from the index 122, the broker/meta-scheduler 124 signals to the grid resources 102 106 and the GPAN 118 in order to reserve and utilize the identified resources. In particular, thebroker/meta-scheduler 124signalsforresourcereservationtoboth thegridresources and theGPAN in the same, grid-standard compliant manner. The GPAN 118 is operative to translate the signaled com mands from thebroker/meta-scheduler 124 into a format that may be utilized by the communications network nodes of NSPs 108-114. Further, the GPAN is operative to translate messages returnedfrom thenetworknodes oftheNSPs to the broker/meta-scheduler. Consequently, two-way communica tions between the network resources and grid manager are possible. Communications between the network resources and grid manager may support operations such asbandwidth allocation, negotiation, agreement, information feedback, and status tracking. Once the appropriate communications network and grid network resources are made available for the task sent to the grid, portions of the overall task are distributed to thegrid resource services eitherdirectly by the broker/meta-scheduler or via the GPAN. 0022 Referring now to FIGS. 2 and 3, the GPAN 118 includes two main components: a Grid Delegation Service (“GDS) 300 and a Grid Proxy Service (“GPS) 302. The GPS 302 is operative to accept network resource requests from upper layer grid services in the Grid Service overlay such as thebroker/meta-scheduler 124a, index 122a, andend userapplication 120a. Resourcerequests includerequests for resource information, resource allocation, and related opera tions.TheGPS302passes thoserequests to theGDS300,and returns feedback received in response to the requests backto therequestinitiator.TheGDSprocessesthe resourcerequests from the GPS to determine which NSPs 108-114 in the Net workServiceoverlay are indicatedtobepartoftheparticular grid operation. The GDS also collects resource information and results from the NSPs for return to the GPS. As shown specifically in FIG. 3, the GPS and GDS components ofthe GPAN logically reside on two overlay networks: GPS on the Feb. 26, 2015 GridServiceOverlayandGDSontheNetworkServiceOver lay. But they may reside on ahost Such as the masterservice peerortwo differenthosts. Forexample,GPS 302 can reside in a grid-based hosting environment that is logically proxi mate to the grid manager100 (FIG. 2), and theGDS 300 can reside in a network service hosting environmentthat is logi callyproximate to the masternetworkservicepeer 116 (FIG. 2). Consequently, GPS is the GPAN contactpoint in the grid networkoverlay whileGDS is the GPAN contactpoint in the network service overlay. (0023 The GPAN 118 may provide respective network resourceprovideras necessaryandappropriate. Forexample, a network information provider (“NIP) 304 facilitates col lection of network information and provides updates and reports to the grid manager index. A resource allocation and management provider (“RAMP) 306 may receive resource allocation requests from the broker/meta-scheduler and pro vide scheduling resources in the communications network. 0024. While the various embodiments described through the above exemplary embodiments, it will be understood by those of ordinary skill in the art that modification to and variation ofthe illustrated embodiments may be made with out departing from the concepts herein disclosed. Moreover, while some embodiments are described in connection with various illustrative structures, one skilled in the art will rec ognize that the system may be embodied using a variety of specificstructures.Accordingly,theembodiments shouldnot be viewed as limited except by the scope and spirit ofthe appended claims. 1. A method ofSupporting operation ofa plurality ofdis tributed computing resources interconnected by a communi cation network, the method comprising: receiving information indicative ofavailability ofat least Some resources of the communication network, the information being in a Supply format; generating a mapping ofavailable resources ofthe at least Some resources; translating at least Some ofthe received information from theSupplyformattoatargetformatusablebyatleastone ofthe distributed computing resources; translating the mapping to the target format; and providingatleastsome ofthetranslated information in the target format and the translated mapping to the at least one of the distributed computing resources to enable analysis ofavailability ofat least some resources ofthe communication network by the at least one ofthe dis tributed communications resources. 2. The method ofclaim 1, wherein receiving the informa tion indicative ofavailability ofat least some resources fur thercomprises receiving the information periodically. 3. The method ofclaim 1, further comprising: receiving at least one message from at least one of the distributed computing resources; processing the at least one received message to derive at leastone commandusablebyatleastoneresourceofthe communication network; and providing the at least one command to the at least one resource ofthe communication network. 4. The method ofclaim 3, wherein the receiving, translat ingandprovidinginformation,andthereceivingandprocess ing messagesandprovidingcommandsderivedfromthemes sages are further configured to Support two-way communicationsbetweentheatleastonecomputingresource and the at least one resource ofthe communication network.
- 8. US 2015/0058490 A1 5. The method ofclaim 4, wherein thetwo-waycommuni cations Support at least one of: bandwidth negotiation; bandwidth allocation; bandwidth agreement; status tracking; or information feedback. 6. The methodofclaim 1, wherein thesupply formatcom prises a Simple Network Management Protocol (SNMP). 7. The method ofclaim 1 further comprising: distributing portions ofat least one task between at least Some nodes of the communication network based, at least in part, on the analysis ofthe availability. 8. A system comprising: at least one processor; and a computer program product embodied on one or more computer-readable storage memory devices, the com puter program product configured to, responsive to execution by the at least one processor, perform a method ofSupporting operation ofa plurality ofdistrib uted computing resources interconnected by a commu nication network, the method comprising: receiving information indicative of availability of at least some resources ofthe communication network, the information being in a Supply format; generating a mapping of available resources of the at least some resources; translating at least Some of the received information from theSupply formattoa target format usableby at least one ofthe distributed computing resources; translating the mapping to the target format; and providing at least Some ofthe translated information in the target formatandthetranslated mapping to the at least one of the distributed computing resources to enable analysis of availability of at least some resources of the communication network by the at least one of the distributed communications SOUCS. 9. The system ofclaim 8, wherein receiving the informa tion indicative ofavailability ofat least some resources fur ther comprises receiving the information periodically. 10. The system ofclaim 8, the method furthercomprising: receiving at least one message from at least one of the distributed computing resources; processing the at least one received message to derive at leastonecommandusableby atleast oneresourceofthe communication network; and providing the at least one command to the at least one resource ofthe communication network. 11. The system ofclaim 10, wherein the receiving, trans lating and providing information, and the receiving and pro cessing messages and providing commands derived from the messagesarefurtherconfiguredto Supporttwo-way commu nicationsbetweentheatleastonecomputingresourceandthe at least one resource ofthe communication network. 12. Thesystem ofclaim 11, wherein the two-way commu nications Support at least one of: bandwidth negotiation; bandwidth allocation; bandwidth agreement; status tracking; or information feedback. Feb. 26, 2015 13. The system of claim 8, wherein the supply format comprisesaSimpleNetworkManagementProtocol(SNMP). 14.The system ofclaim 8, themethod furthercomprising: distributing portions ofat least one task between at least Some nodes of the communication network based, at least in part, on the analysis ofthe availability. 15. A computerprogram product forSupporting operation of a plurality of distributed computing resources intercon nected by a communication network, the computer program product comprising: logic operating to receive information indicative ofavail ability ofat least Some resources ofthe communication network, the information being in a Supply format; logic operating to generate a mapping of available resources ofthe at least some resources; logic operating to translate at least Some of the received information from the Supply format to a target format usable by at least one of the distributed computing resources; logic operating to translate the mapping to the target for mat; and logic operating to provide at least Some ofthe translated information in the target format andthetranslated map ping to the at least one of the distributed computing resources to enable analysis ofavailability ofat least Some resources ofthe communication networkby theat least one ofthe distributed communications resources. 16. The computer program product of claim 15 further configured to receive the information indicative ofavailabil ity ofat least Some resources periodically. 17. The computer program product of claim 15 further comprising: logic operatingtoreceiveatleast onemessagefromatleast one ofthe distributed computing resources; logic operatingtoprocesstheatleastonereceived message to derive at least one command usable by at least one resource ofthe communication network; and logic operating to providetheat least one commandto the at least one resource ofthe communication network. 18.Thecomputerprogramproductofclaim 17whereinthe logic operating to receive, translateandprovide information, and the logic operating to receive and process messages and providecommandsderived from the messages is furthercon figured to Support two-way communications between the at least one computing resourceand the atleast one resource of the communication network. 19. The computer program product ofclaim 18, wherein the two-way communications Support at least one of bandwidth negotiation; bandwidth allocation; bandwidth agreement; status tracking; or information feedback. 20. The computer program product of claim 15 further comprising: logic operating to distribute portions of at least one task between at least Some nodes ofthe communication net workbased, at least in part, on the analysis ofthe avail ability.