Gsoc airavata
- 6. Hi, I’m Nolram. I’m a computational physicist. I run computational experiments everyday This is how typically I run my experiments
- 7. Scientific Application Another Scientific Application First I collect my observed data And then pass data to my applications & get the result This is starting to become a very tiring task
- 8. How can I make this much simpler…? Logically, this is how my life would be made easier… Is it possible to automate this flow sequence without my guidance?
- 9. Scientists from many different fields face this problem everyday. The solution is to use a workflow-powered science gateway to manage the experiment online. What is a workflow you ask? Well, you just saw one in our previous animation…
- 10. We introduce Apache Airavata, a system capable of composing, managing, executing, and monitoring small to large scale applications and workflows Want to see how it works? A Typical Workflow
- 11. Apache Airavata I will handover my data & my experiment details (the workflow) to the Airavata server The Gateway Airavata will complete the experiment & return me the results Results Progress of the experiment … and while I wait for results, Airavata will notify me with progress updates of my experiment
- 12. Let’s look closely how Airavata manages workflows. The GatewayResults Experiment progress Apache Airavata
- 13. Let’s look closely how Airavata manages workflows. The GatewayResults Experiment progress
- 14. Airavata main has 4 components… The Gateway 1. Workflow Interpreter Steer the workflow execution 2. The GFac Steer science app executions & data transfers Workflow Interpreter GFac Message Box Registry 3. The Registry Defines the available applications & records all results of experiments 4. The Message Box Records the progress of the workflow execution
- 15. A Stable API for Airavata Apache Airavata EndUsersGatewayDeveloper Scientific Application Computational Resources
- 16. Application Registration UI Application Developer A1 Airavata Service Interface (wraps client API) Service Map XML Airavata Server Service Map to AWSDL Web Based workflow composerWorkflow Developer Get AWSDL Put XWF A2 A3 W1 W2 W3 Web Based Experiment BuilderExperiment Builder Get WI’s E1 E2 E3 W4 Shred Workflow Inputs Launch Workflow Web Based Workflow MonitorWatch Progress M1 M2 Get Workflow Graph M3 Monitor Workflow
- 17. Goal of the project • Design Web-Based interfaces for Airavata: – Application Registration – Workflow Construction – Workflow Execution – Workflow Monitoring • Provide an opportunity for GSoC to understand Distributed System in action • Scope for Research and Software Engineering papers
- 18. Data Model • Application Description – User describes inputs and outputs of the application. – Currently this information is captured in Service Map Schema. – This schema is stored in Airavata Registry as XML. Also the schema utility generates a application service WSDL from this schema using the Airavata WSDL Generator.
- 19. Application Registration UI Application Developer A1 Service Map XML Service Map to AWSDL Web Based workflow composer Workflow Developer Get AWSDL A2 W1 W2 XML AiravataServerAPI Launch & Manage Jobs Notify progress of job or workflow execution Real-Time Monitoring Messaging Subsystem Registry Execute & Manage Computations Workflow Interpreter Application Factory (Gfac) Applicatio n Desc Applicatio n Desc Workflow
- 20. A peek at one of the cluster Interconnect Nodes
- 21. Scheduling ‘qsub’ batch jobs on the cluster worker node worker node worker node worker node worker node SGE MASTER node Queue-A Queue-B Queue-C ASlot1 ASlot2 BSlot1 CSlot1 CSlot2 CSlot3 BSlot1 BSlot2 BSlot3 BSlot1 CSlot1 CSlot2 ASlot1 BSlot1 CSlot1 Queues Policies Priorities Share/Tickets Resources Users/Projects JOB Y JOB Z JOB X JOB U JOB O JOB N
- 22. System characteristics System status Resources Job policies Resources Resource Matching JOB User User policies Groups Roles Departments Projects Selection Scheduling
- 23. Simplified Gateway Architecture One time Gateway Community Setup Community Account Grid Certificate username, password Gateway Interface Gateway Server Compute Servers Gateway Authentication Job Submit or File Transfer request Output Step 0 Step 1 Step 2,3,,
- 24. ParamChem BioVLab GridChem DES VLAB UltraScan Apache Airavata 1.0 NSG ParamChem BioVLab CIPRES POPLAR GridChem DES UltraScan VLAB Apache Airavata 2.0 Apache Airavata 1.0 Apache Airavata 1.0 Apache Airavata 1.0 Apache Airavata 1.0 Apache Airavata 1.0