Introduction to Globus (APS Workshop)
0 likes150 views
Globus is a non-profit data management service that allows users to transfer, share, and access data across different storage systems and platforms through software-as-a-service. It has transferred over 1.34 exabytes of data and aims to unify access to research data across different tiers of storage through connectors, APIs, and user interfaces. Globus ensures secure data transfers and sharing by using user identities, access controls, encryption, and audit logging without storing user credentials or data.
Introduction to Globus (APS Workshop)
- 1. Globus – Beyond File Transfer A Globus introduction and more APS and ALCF Globus Training October 12, 2021 Greg Nawrocki greg@globus.org nawrocki@uchicago.edu
- 2. Globus is … a non-profit subscription based data management service developed and operated by
- 4. As of this morning – 1.34 Exabytes
- 6. 6 Globus delivers… Fast and reliable data transfer, sharing, and platform services… …directly from your own storage systems… ...via software-as-a-service using existing identities with the overarching goal of...
- 7. 7 Research Computing HPC Desktop Workstations Mass Storage Instruments Personal Resources Public Cloud National Resources Unifying access to data across tiers
- 9. Public / private cloud stores External campus storage EC2 Project repositories, replication stores Public repositories Share with collaborators/community
- 10. Analysis store Next-Gen Sequencer MRI Advanced Light Source Personal system Remote visualization Light Sheet Microscope High-durability, low-cost store Manage data from instruments Cryo-EM
- 11. Use(r)-appropriate interfaces 11 GET /endpoint/go%23ep1 PUT /endpoint/vas#my_endpt 200 OK X-Transfer-API-Version: 0.10 Content-Type: application/json … Globus service Web CLI Rest API
- 12. Globus SaaS / PaaS: Research data lifecycle Researcher initiates transfer request; or requested automatically by script, science gateway 1 Instrument Compute Facility Globus transfers files reliably, securely 2 Globus controls access to shared files on existing storage; no need to move files to cloud storage! 4 Researcher selects files to share, selects user or group, and sets access permissions 3 Collaborator logs in to Globus and accesses shared files; no local account required; download via Globus 5 Streamlining research workflows and ensuring those that need access to the data have it. 8 Personal Computer Transfer Share • Use a Web browser or platform services • Access any storage • Use an existing identity Build The Globus Command Line Interface, API sets, Python SDK and Action Providers give you a platform… 6 … for building science gateways, portals and automations. 7 Search Automate
- 13. Globus core security features • Access Control – Identities provided and managed by institution – Globus is identity broker; no access to/storage of user credentials – Fine grained access control • Data remain at institutions, not stored by Globus • Data does not flow through the Globus Service but directly between Endpoints and their Collections • Integrity checks of transferred data • High availability and redundancy • Encryption of user files and Globus control data
- 14. Hybrid SaaS – Transfer – Mapped Collections
- 15. Hybrid SaaS – Sharing – Guest Collections
- 16. Endpoints, Collections and Globus Connect • Globus Connect Server – Multi user Linux Systems – https://docs.globus.org/globus-connect-server/ • Globus Connect Personal – Personal Workstations and Laptops – https://www.globus.org/globus-connect-personal – OS specific instructions o https://docs.globus.org/how-to/
- 17. Demo time! Authentication Transfer Sharing Transfer Details The Hamburger Menu The Activity Monitor Groups Roles HTTPS Downloads Globus Connect Personal
- 18. Manage Protected Data 18 Higher assurance levels for HIPAA and other regulated data • Support for protected data such as health related information • Share data with collaborators while meeting compliance requirements • Includes BAA option
- 19. Globus for high assurance data management • Restricted data handling – PII (Personally identifiable information) – CUI (Controlled Unclassified Information) – PHI (Protected Health Information) • University of Chicago security controls – NIST SP 800-53 – Superset of NIST SP 800-171 • Business Associate Agreements (BAA) will be between University of Chicago and our subscribers – University of Chicago has a BAA with Amazon
- 20. High Assurance features • Additional authentication assurance – Per storage gateway policy on frequency of authentication with specific identity for access to data (timeout) – Ensure that user authenticates with the specific identity that gives them access within session (decoupling linked identities) • Session/device isolation – Authentication context is per application, per session (~browser session) • Enforces encryption of all user data in transit • Audit logging
- 21. Support resources • Globus documentation: docs.globus.org • YouTube channel: youtube.com/user/GlobusOnline • Helpdesk and issue escalation: support@globus.org • Mailing Lists – globus.org/mailing-lists • Customer engagement team – Office Hours