Services for Science

Services for Science Ian Foster Computation Institute Argonne National Lab & University of Chicago

Thanks! DOE Office of Science NSF Office of Cyberinfrastructure National Institutes of Health Colleagues at Argonne, U.Chicago, USC/ISI, OSU, Manchester, and elsewhere

Scientific Communication, ~1600 Brahe Kepler

Scientific Communication, ~2000 Data Archives Gateway Figure: S. G. Djorgovski Service-Oriented Science User Analysis tools Discovery tools

caBIG: sharing of infrastructure, applications, and data. Data Integration! Services & Cancer Biology Globus

Raffaele Montella, U. Napoli Parthenope Services & Environmental Science

Service-Oriented Science People create services (data, code, instr.) … which I discover (& decide whether to use) … & compose to create a new function ... & then publish as a new service.  I find “someone else” to host services, so I don’t have to become an expert in operating services & computers!  I hope that this “someone else” can manage security, reliability, scalability, … ! ! “ Service-Oriented Science”, Science , 2005

Creating Services People create services (data, code, instr.) … which I discover (& decide whether to use) … & compose to create a new function ... & then publish as a new service.  I find “someone else” to host services, so I don’t have to become an expert in operating services & computers!  I hope that this “someone else” can manage security, reliability, scalability, … ! ! “ Service-Oriented Science”, Science , 2005

Anatomy of a Service op1 opN (meta)data Implementation(s) Clients Registry Management Clients … Service Service Attribute Authority Attribute Authority Persistence

Exposing Service State State Interfaces  Get resource property  Get multiple RPs  Set RP state  Query a set of RPs  Subscribe: notifications  Manage termination time  Immediate destruction Service GetRP GetMultRPs SetRP QueryRPs Subscribe SetTermTime Destroy EPR EPR EPR State representation Resource, resource property State identification Endpoint Reference Resource property Resource

Globus Toolkit Web Services Container PIP PDP WorkManager DB Conn Pool JNDI Directory Security Persistence Management State Authorization Globus Toolkit Version 4: Software for Service-Oriented Systems, LNCS 3779, 2-13, 2005 Apache Tomcat Service Container RPs Resource Service GetRP GetMultRPs SetRP QueryRPs Subscribe SetTermTime Destroy EPR EPR EPR ResourceHome RPs Resource Service GetRP GetMultRPs SetRP QueryRPs Subscribe SetTermTime Destroy EPR EPR EPR ResourceHome RPs Resource Service GetRP GetMultRPs SetRP QueryRPs Subscribe SetTermTime Destroy EPR EPR EPR ResourceHome Globus

Creating Services (~2005) “ This full-day tutorial provides an introduction to programming Java services with the latest version of the Globus Toolkit version 4 (GT4). The tutorial teaches how to build a Java Service that makes use of GT4 mechanisms for state management, security, registry and related topics.”

Creating Services in 2008 Introduce and gRAVI Introduce Define service Create skeleton Discover types Add operations Configure security Grid R emote A pplication V irtualization Infrastructure Wrap executables Index service Repository Service Introduce Container Ohio State University and Argonne/U.Chicago Appln Service Create Store Advertize Discover Invoke; get results Transfer GAR Deploy Globus

Demonstration: Creating Services Introduce + gRAVI Shannon Hastings Scott Oster David Ervin Stephen Langella Kyle Chard Ravi Madduri

Discovering Services People create services (data or functions) … which I discover (& decide whether to use) … & compose to create a new function ... & then publish as a new service.  I find “someone else” to host services, so I don’t have to become an expert in operating services & computers!  I hope that this “someone else” can manage security, reliability, scalability, … ! ! “ Service-Oriented Science”, Science , 2005

Discovering Services Assume success Syntax, semantics Permissions Reputation  The ultimate arbiter?  Types, ontologies  Can I use it?  Billions of services A B

Discovery (1): Registries Globus

Discovery (2): Standardized Vocabularies Core Services Grid Service Uses Terminology Described In Cancer Data Standards Repository Enterprise Vocabulary Services References Objects Defined in Service Metadata Publishes Subscribes to and Aggregates Queries Service Metadata Aggregated In Registers To Discovery Client API Index Service Globus

Discovery (3): Tagging & Social Networking GLOSS : Generalized Labels Over Scientific data Sources (Foster, Nestorov)

Discovery (3): Tagging & Social Networking David de Roure, Carole Goble, et al.

Composing Services People create services (data or functions) … which I discover (& decide whether to use) … & compose to create a new function ... & then publish as a new service.  I find “someone else” to host services, so I don’t have to become an expert in operating services & computers!  I hope that this “someone else” can manage security, reliability, scalability, … ! ! “ Service-Oriented Science”, Science , 2005

Composing Services: E.g., BPEL Workflow System Analytic service @ osu.edu Analytic service @ duke.edu caBiG: https://cabig.nci.nih.gov/; BPEL work: Ravi Madduri et al. link link link See also Kepler & Taverna Data Service @ uchicago.edu <BPEL Workflow Doc> <Workflow Inputs> <Workflow Results> BPEL Engine link Globus

Demonstration: Composing Services Taverna + GT4 Taverna team Wei Tan Ravi Madduri

Publishing Services People create services (data or functions) … which I discover (& decide whether to use) … & compose to create a new function ... & then publish as a new service.  I find “someone else” to host services, so I don’t have to become an expert in operating services & computers!  I hope that this “someone else” can manage security, reliability, scalability, … ! ! “ Service-Oriented Science”, Science , 2005

Publishing Services Description  Syntax, semantics State  Availability, load, … Policies  Who, what, when, … Hosting  Location, scalability, …

Defining Community: Membership and Laws Identify VO participants and roles For people and services Specify and control actions of members Empower members  delegation Enforce restrictions  federate policy Access granted by community to user Site admission-control policies Effective Access Policy of site to community A 1 2 B 1 2 A B 1 10 1 10 1 16

Authorization: SAML & XACML VOMS Shibboleth LDAP PERMIS … GT4 Client GT4 Server PDP Attributes Authorization Decision PIP PIP PIP SAML XACML Globus

Hosting Services People create services (data or functions) … which I discover (& decide whether to use) … & compose to create a new function ... & then publish as a new service.  I find “someone else” to host services, so I don’t have to become an expert in operating services & computers!  I hope that this “someone else” can manage security, reliability, scalability, … ! ! “ Service-Oriented Science”, Science , 2005

The Importance of “Hosting” and “Management” Tell me about this star Tell me about these 20K stars Support 1000s of users E.g., Sloan Digital Sky Survey, ~10 TB; others much bigger

The Two Dimensions of Service-Oriented Science Decompose across network Clients integrate dynamically Select & compose services Select “best of breed” providers Publish result as new services Decouple resource & service providers Function Resource Data Archives Analysis tools Discovery tools Users Fig: S. G. Djorgovski

Dynamic Provisioning : Falkon Applied to Stacking Purpose On-demand “stacks” of random locations within ~10TB dataset Challenge Rapid access to 10-10K “random” files Time-varying load Solution Dynamic acquisition of compute, storage S 4 Sloan Data Web page or Web Service Joint work with Ioan Raicu & Alex Szalay + + + + + + = + Globus

Dynamic Provisioning with Falkon: Release after 15 Seconds Idle

Dynamic Provisioning with Falkon: Release after 180 Seconds Idle

Building Scalable Service Implementations Functional MRI Ben Clifford, Mihael Hatigan, Mike Wilde, Yong Zhao Globus

AIRSN Program Definition (Run snr) functional ( Run r, NormAnat a, Air shrink ) { Run yroRun = reorientRun ( r , "y" ); Run roRun = reorientRun ( yroRun , "x" ); Volume std = roRun[0]; Run rndr = random_select ( roRun, 0.1 ); AirVector rndAirVec = align_linearRun ( rndr, std, 12, 1000, 1000, "81 3 3" ); Run reslicedRndr = resliceRun ( rndr, rndAirVec, "o", "k" ); Volume meanRand = softmean ( reslicedRndr, "y", "null" ); Air mnQAAir = alignlinear ( a.nHires, meanRand, 6, 1000, 4, "81 3 3" ); Warp boldNormWarp = combinewarp ( shrink, a.aWarp, mnQAAir ); Run nr = reslice_warp_run ( boldNormWarp, roRun ); Volume meanAll = strictmean ( nr, "y", "null" ) Volume boldMask = binarize ( meanAll, "y" ); snr = gsmoothRun ( nr, boldMask, "6 6 6" ); } (Run or) reorientRun (Run ir, string direction) { foreach Volume iv , i in ir.v { or.v[i] = reorient( iv , direction); } }

Dynamic Provisioning: Swift Architecture SwiftScript Abstract computation Virtual Data Catalog SwiftScript Compiler Specification Execution Yong Zhao, Mihael Hatigan, Ioan Raicu, Mike Wilde, Ben Clifford, et al. Virtual Node(s) Virtual Node(s) Provenance data Provenance data Provenance collector launcher launcher file1 file2 file3 App F1 App F2 Scheduling Execution Engine (Karajan w/ Swift Runtime) Swift runtime callouts C C C C Status reporting Provisioning Falkon Resource Provisioner Amazon EC2 Globus

Services for Science They’re new A new approach to communicating A (not-so new) approach to structuring systems They’re real Excellent infrastructure and tools (Globus, Introduce, gRAVI, Taverna, Swift, etc.) Substantial numbers of services out there They’re challenging Sociology: incentives, rewards Infrastructure: hosting Provenance: justifying “results” Scaling: services, requests

Services for Science

More Related Content

Similar to Services for Science (20)

More from Ian Foster (20)

Recently uploaded (20)

Services for Science