Webinar on Galaxy & Galaxy integration with the Dataminer Service in the context of AGINFRA+ Project
Download as PPTX, PDF1 like268 views
The AGINFRA+ project, funded by the EU's Horizon 2020 program, aims to enhance user-driven e-infrastructure innovation in food and agriculture. It demonstrates how scientific communities can effectively develop and deploy applications using the Galaxy platform, which supports accessible, reproducible, and transparent genomic science. Key features include user-friendly tools for data manipulation, a robust history management system, and integration with platform services, while ongoing enhancements aim to improve user feedback and integration capabilities.
Webinar on Galaxy & Galaxy integration with the Dataminer Service in the context of AGINFRA+ Project
- 1. WWW.PLUS.AGINFRA.EU AGINFRA PLUS - Accelerating user-driven e-infrastructure innovation in Food & Agriculture has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 731001. Galaxy for AGINFRA+ VREs Online webinar | January 2019 Enol Fernández EGI Foundation
- 2. WWW.PLUS.AGINFRA.EU Webinar is being recorded! Galaxy for AGINFRA+ VREs 2
- 3. WWW.PLUS.AGINFRA.EU Demonstrate how scientific communities working on agriculture and food topics may carry out rapid and intuitive development and deployment of innovative applications and workflows, powered by open e-infrastructures. Strengthen and illustrate the value and potential of AGINFRA+ as a virtual research environment for the domain of agriculture and food. The AGINFRA+ Vision Galaxy for AGINFRA+ VREs 3
- 4. WWW.PLUS.AGINFRA.EU Enol Fernández Cloud Architect @ EGI Foundation Working since 2003 on computing e-infrastructures (grid, cloud) Support to user communities EGI Cloud Federation Coordinator Development of new services for EGI Galaxy for AGINFRA+ VREs 4 /usr/bin/whoami
- 5. WWW.PLUS.AGINFRA.EU Galaxy Galaxy integration in AGINFRA+ Demo Q&A Outline Galaxy for AGINFRA+ VREs 5
- 7. WWW.PLUS.AGINFRA.EU Web-based platform for computational biomedical research (analysis and data integration) Open source, community driven software that makes integrating your biomed tools simple Galaxy for AGINFRA+ VREs 7
- 8. WWW.PLUS.AGINFRA.EU Galaxy is "an open, web-based platform for performing accessible, reproducible, and transparent genomic science.” Accessibility Users without programming experience can easily upload/retrieve data, run complex tools and workflows, and visualize data Reproducibility Galaxy captures information so that any user can understand and repeat a complete computational analysis Transparency Users can share or publish their analyses (histories, workflows, visualizations) Galaxy for AGINFRA+ VREs 8 Galaxy: core values
- 9. WWW.PLUS.AGINFRA.EUGalaxy for AGINFRA+ VREs 9 Galaxy interface Toolbar HistoryMain
- 10. WWW.PLUS.AGINFRA.EU Tools manipulate and transform your data Each tool is defined by: input datasets, parameters, commands, and outputs help, tests, citations, dependency requirements New versions can be installed without removing old ones to ensure reproducibility Galaxy Tool Shed: application store with +1000 tolos ready to be used Galaxy for AGINFRA+ VREs 10 Tools
- 11. WWW.PLUS.AGINFRA.EUGalaxy for AGINFRA+ VREs 11
- 12. WWW.PLUS.AGINFRA.EU Every action and dataset is tracked on the history Datasets produced by tools Operations performed on data Includes: name, format, size, creation time, datatype-specific metadata tool id, version, inputs, parameters standard output (stdout) and error (stderr) state (waiting, running, success, failed) hidden, deleted, purged Galaxy for AGINFRA+ VREs 12 History
- 13. WWW.PLUS.AGINFRA.EUGalaxy for AGINFRA+ VREs 13 History management
- 14. WWW.PLUS.AGINFRA.EU Series of tools and dataset actions that run in sequence as a batch operation From history, from scratch with visual editor or import Allow to re-run the same analysis on different input data sets Change parameters before re-running a similar analysis Workflows can be annotated, viewed, shared, published, and imported - just like other Galaxy objects. Galaxy for AGINFRA+ VREs 14 Workflows
- 15. WWW.PLUS.AGINFRA.EUGalaxy for AGINFRA+ VREs 15 Workflow editor Image taken from https://mgescan.readthedocs.io/en/latest/workflow.html
- 17. WWW.PLUS.AGINFRA.EU Workflow design and execution technologies are key for the research activities of the use cases Two platforms selected: KNIME Galaxy Integration with AGINFRA+: Access the workflow platforms within the VRE Access other VRE tools form the workflow platforms Galaxy for AGINFRA+ VREs 17 Workflows on AGINFRA+
- 18. WWW.PLUS.AGINFRA.EU Custom wsgi filter for Galaxy Leverages support for external authentication in Galaxy Exposes REMOTE_USER with information from D4Science Stores the D4Science token to further interaction with the VRE Galaxy for AGINFRA+ VREs 18 Authentication
- 19. WWW.PLUS.AGINFRA.EUGalaxy for AGINFRA+ VREs 19
- 20. WWW.PLUS.AGINFRA.EUGalaxy for AGINFRA+ VREs 20 DataMiner and Galaxy Periodic synchronisation of DataMiner as tools in Galaxy Execution using WPS calls to Dataminer With user token Use DataMiner algorithm as any other Galaxy tool
- 21. WWW.PLUS.AGINFRA.EUGalaxy for AGINFRA+ VREs 21
- 22. WWW.PLUS.AGINFRA.EUGalaxy for AGINFRA+ VREs 22 Some limitations Input files must be provided as URLs No easy way to discover the files from the Galaxy interface Output is stored as HTML with pointers to the actual outputs Can still be easily converted to other more friendly format as needed A csv extractor provided to facilitate that case, more can be created.
- 23. WWW.PLUS.AGINFRA.EU Kubernetes Cluster (EGI Cloud Container Compute) Galaxy for AGINFRA+ VREs 23 Deployment SSL Certificate Kubernetes Ingress AuthN filter WPS calls
- 25. WWW.PLUS.AGINFRA.EU Galaxy available now for AGINFRA+ VREs Go and test! Start collecting your feedback: Add new tools as needed by use cases Improve the integration with DataMiner (input and output files) Expand deployment as usage grows Galaxy for AGINFRA+ VREs 25 Next steps
- 26. WWW.PLUS.AGINFRA.EU Galaxy: https://galaxyproject.org/ Galaxy Tutorials: https://galaxyproject.org/learn/ Galaxy training: https://galaxyproject.github.io/training-material/ Galaxy ToolShed: https://toolshed.g2.bx.psu.edu/ DataMiner: https://wiki.gcube- system.org/gcube/DataMiner_Manager AGINFRA+ VREs: https://aginfra.d4science.org/ Galaxy for AGINFRA+ VREs 26 Reference