Introduction to Globus for New Users
0 likes64 views
Globus is a non-profit service designed to enhance data-driven science by offering fast and reliable data transfer and sharing directly from users' storage systems without needing to move files to the cloud. It provides automation tools and supports users through existing identities with a focus on fine-grained access control and data integrity. Additionally, Globus offers APIs and SDKs for building customized research data services and applications.
Introduction to Globus for New Users
- 1. Introduction to Globus for New Users Greg Nawrocki greg@globus.org June 22, 2022
- 2. Globus is … a non-profit service developed and operated by
- 3. Our Mission Increase the efficiency and effectiveness of researchers engaged in data-driven science and scholarship through sustainable software
- 4. Development is funded by... U . S . D E PA R T M E N T O F ENERGY
- 5. Operations are funded by subscribers
- 6. 7
- 7. Transfer and share data
- 8. 9 Globus delivers… Fast and reliable big data transfer, sharing, and platform services… …directly from your own storage systems… ...via software-as-a-service using existing identities with the overarching goal of...
- 9. 10 …unifying access to data across tiers
- 10. 11
- 11. Globus SaaS / PaaS: Research data lifecycle Researcher initiates transfer request; or requested automatically by script, science gateway 1 Instrument Globus controls access to shared files on existing storage; no need to move files to cloud storage! 4 Researcher selects files to share, selects user or group, and sets access permissions 3 …and automating research workflows - ensuring those that need access to the data have it. 8 Collaborator logs in to Globus and accesses shared files; no local account required; download via Globus 5 Personal Computer Transfer Share • Use a Web browser or platform services • Access any storage • Use an existing identity Compute Facility Globus transfers files reliably, securely 2 Build The Timer and Automation Services, the Command Line Interface, API sets, and Python SDK provide the tools… 6 … for building science gateways, portals ,publication services… 7
- 12. Hybrid SaaS – Transfer – Mapped Collections
- 13. Hybrid SaaS – Sharing – Guest Collections
- 14. Globus core security features • Access Control – Identities provided and managed by institution – Institution controls all access policies – Globus is identity broker; no access to/storage of user credentials – Fine grained access control on the collections • Data remain at institutions, not stored by Globus • Data does not flow through the Globus Service but directly between Endpoints and their Collections • Integrity checks of transferred data • High availability and redundancy • Encryption of user files and Globus control data
- 15. Collections and Globus Connect • Globus Connect Server – Multi user Linux Systems – https://docs.globus.org/globus-connect-server/ • Globus Connect Personal – Personal Workstations and Laptops – https://www.globus.org/globus-connect-personal – OS specific instructions o https://docs.globus.org/how-to/
- 16. Demo time! Identities and Accounts Transfer Sharing Transfer Details Bookmarks The Console The Hamburger Menu The Activity Monitor Groups Roles Responsive Interface
- 17. One service, many interfaces 23 GET /endpoint/go%23ep1 PUT /endpoint/vas#my_endpt 200 OK X-Transfer-API-Version: 0.10 Content-Type: application/json … Globus service Web CLI Rest API Flows
- 18. Globus Automation Capabilities Timer Service Scheduled and recurring transfers (a.k.a. Globus cron) Command Line Interface Ad hoc scripting and integration Globus Flows service Comprehensive task (data and compute) orchestration with human in the loop interactions
- 19. 26 Custom portals? Science Gateways? Unique workflows? Our open REST APIs and Python SDK empower you to create an integrated ecosystem of research data services and applications.
- 20. Globus APIs • Auth • Groups • Transfer • Search • Timer • Flows • GCS Manager • Globus Web App consumes public Transfer API • Resource named by URL (standard REST approach) • Globus APIs use JSON for documents docs.globus.org/api/transfer
- 21. Globus Python SDK • Python client library for the Globus REST APIs • Largely direct mapping to REST API • globus_sdk.TransferClient class handles connection management, security, framing, marshaling globus-sdk-python.readthedocs.io/en/stable/ globus.github.io/globus-sdk-python 28
- 22. Support Resources • Globus Documentation: docs.globus.org • Globus Timer: globus.org/blog/scheduled-and- recurring-transfers-now-available-globus-web-app • Globus CLI: docs.globus.org/cli/ • Globus Automation Services and Flows: docs.globus.org/globus-automation-services/ • YouTube Channel: youtube.com/user/GlobusOnline
- 23. Developer References • Globus API / SDK Documentation – Transfer API : docs.globus.org/api/transfer/ – SDK: globus-sdk-python.readthedocs.io/en/stable/ • Globus GitHub: github.com/globus/ – Jupyter Notebooks o Stand alone notebooks and hub integrations that walk through much of the functionality of our SDK o https://github.com/globus/globus-jupyter-notebooks – Automation Examples o Shell scripted CLI and Python module examples of common research data management use cases o https://github.com/globus/automation-examples