Fabric And Storage Management
- 1. Fabric and Storage Management GridPP 24/24 May 2001
- 2. Overview GridPP Proposal Other UK Proposals EU-DataGrid US Activities Commercial Activities Timescales Commentary Lots of Activity Duplication? GridPP role
- 3. Compute & Mass Storage BlueChip HPC COTS “HPC” Clusters User Portal Storage No: Networks Brokerage Collective
- 4. GridPP Proposal “Compute” Develop Project Plan (UKDG) COTS system development (UKDG) Integration of existing Fabric(UKPP) Fabric Benchmarking(UKPP) User Portals(UKPP) Fabric Demonstrator cf EU-WP4 API design? Clear Model?
- 5. GridPP Proposal “Mass Storage” Evaluation and API design(EU) Prototype(EU) Refinement of API(UKDG) Definition of Metadata(UKDG) Metadata(UKDG)
- 6. DataGrid Information Architecture Definition Documents WP Milestones Month 9 Deliverables Note GridPP: ATF? reevaluate Deliverables
- 7. Other UK Proposals GenGrid EPSRC testbed E-Commerce related Generic Grid Activities within Universities GridPP Are we really the “ leaders”?
- 8. DataGrid(WP4) Tasks Configuration management Resource management Gridification Installation Monitoring Fault Tolerance
- 9. DataGrid(WP5) Originally Mass Storage part of local fabric E.g. Part of a HSM at a large centre Post Architecture Task Force (ATF) Storage Element Interfaces (SE’s) Applicable at MORE than Tier-1 centres!!!! Directly accessible from the Grid Tape and Disk Pools Publishing Metadata UK Contribution Important: Edinburgh RAL
- 10. US Activities GriPhyN Virtual Data Toolkits Particle Physics Data Grid (to 2004) http://www.ppdg.net/ Very WP4/WP5 based
- 11. Commercial Activities ICT companies monitoring Grid Strategically placing themselves E.g. IBM Development CISCO… We MUST not be blind to E-commerce relationships resources manpower
- 12. Commercial Activities E-commerce Testbeds are vital Companies or Consortia see 10-20% savings Can invest MUCH more than GridPP ever can by at least 2 orders of magnitude Have more access to trained software teams
- 13. WG-D Tasks Definitions Integration of Existing Compute Fabric COTS Compute System Development DataGrid Common API DataGrid Meta-data for Mass Storage DataGrid Tape Exchange Design (!) DataGrid Tape Exchange Implementation (!)
- 15. Commentary GridPP Fabric + Storage Management Priority : deploy effort to satisfy DataGrid contract for PPARC Recommend 6 month report (Nov 2001?) Position GridPP relative to worldwide Grid activities and build NOT compete Evaluate Fabric Development Strategy in light of products and needs of LHC and E-commerce Time to engage Computer Scientists and commercial partners both as “stakeholders” and contractually Time to find the right people (we don’t have enough) ( If cannot need to readjust goals!) resources
- 16. Commentary “ Building something, no matter how bad it is, is better than nothing at all …”(I disagree) National, international academic and commercial collaboration is vital Simplicity,robustness and useability are very important Our “lives” depend on it Note tendency to overambitious plans Skilled programmer crisis will kill the project Must work with industry or subcontract deliverables
- 17. GridPP WG-D Objectives System software for integration of existing Grid components Develop system(s) for COTS integration Work with HPC supplier towards Grid enabled proprietary systems Build fabric management model allowing hot-swapping and hot-addition of new technologies Provide on-going benchmarking and cost-performance analysis of Grid components Produce 1 st level user portals to Grid fabric
- 18. Objectives Continued… Proof of concept for on thousand PC level of scaling to 10k level Define and implement mass storage API Devise method of moving physical tapes and associated meta-data between centres Tools and schema for meta data and mass storage information services (not physical datasets) Integrate with DataGrid WP4 and WP5 programmes