Research network infrastructure engineers
Download as PPTX, PDF0 likes76 views
Tim Chown proposes creating a community of Research Network Infrastructure Engineers to share best practices for optimizing network infrastructure to support large-volume research data storage, access, and transfer. The goals of the community would be to establish and share best practices, learn from each other's experiences, and help Jisc understand support needs. Existing resources like the Science DMZ model, WLCG/GridPP expertise, and Jisc performance testing capabilities could provide a good starting point for discussion. Tim invites comments on the scope and structure of the potential community.
Research network infrastructure engineers
- 1. Networkshop Fringe: Research Network Infrastructure Engineers Tim Chown (Jisc) tim.chown@jisc.ac.uk Online, 14 June 2022
- 2. Networkshop Fringe Something new •The fringe events are an experiment to continue community discussion following on from Networkshop last week •Five events •https://www.jisc.ac.uk/get-involved/community-fringe •Today’s fringe focuses on a potential community of Research Network Infrastructure Engineers •But who or what are they? 2
- 3. Research Network Infrastructure Engineers Who? •I’d say they’re the staff responsible for ensuring an organisation’s network infrastructure is optimised to support its capability to store, access and transfer large volumes of research data •So people involved with design and/or implementation •And the idea of today’s session is to see if there’s interest in creating / fostering a community of such people •The name (and definition) is certainly up for discussion! 3
- 4. What’s driving the need? Many examples •WLCG / GridPP – established, data volumes growing •SKA / UKAEA – presented at NWS50 •Projecting up to 2PB of new data a day, which ~= 200Gbps •EMBL-EBI, Rubin Observatory/LSST, DLS, … •And many more coming, with different needs, different data distribution models (CPU to data or data to CPU?) •Important for Jisc to keep aware of community requirements •Implies communities should have (good) forward looks 4
- 5. High level goals of a community? What could a community do? •Establish and share best practices •Share experiences and examples •Try new ideas •Learn from each other what works well and what doesn’t •Help Jisc understand what it can do to support the community •But this is up for discussion if we decide to start something new 5
- 6. Existing resources and knowledge? What’s out there? •Good experience from a number of science communities, e.g. WLCG / GridPP, JASMIN/CEDA, etc •Great example recently of 100G of IPv6 at Imperial •We have a good starting point •Some great advice at ESnet’s https://fasterdata.es.net/ •Material presented at previous Jisc events •These are great as they’re what Janet sites are doing in practice •And more 6
- 7. Common approaches? Science DMZ is mentioned a lot •A blueprint / model for optimising data transfers •Specific principles for local network architecture •Well-tuned data transfer nodes (DTNs) and software tools •Persistent performance measurement •Security policy implementation doesn’t impair performance •WLCG / GridPP have been using these principles for a long time •Do we feel this is a good model to share and promote? 7
- 8. Should note that Science DMZ is flexible Many ways to adopt it •Local architecture •Some universities dedicate 10G path(s) to Janet for GridPP •Software tools •Globus/GridFTP, xrootd, Aspera, FDT, rsync, … •Persistent monitoring •perfSONAR, Netsight, tstat, FTS logs, streaming telemetry, … •Efficient security •ACLs, internal disk mounts on DTNs, … •The interesting question is what works well? 8
- 9. Some other issues Examples of potential discussion topics •What is the data distribution model? •Where does cloud come in to the picture? •How to determine the most cost-effective solutions •IPv6 support by default? •Encryption / security of data •Designing for 100G •… 9
- 10. Performance testing Openly available Jisc test capabilities •Hosted at our Slough shared DC – currently three capabilities: •iperf - iperf-slough-10g.ja.net •perfSONAR - ps-slough-10g.ja.net and ps-slough-1g.ja.net •DTN application tests, including Globus endpoint •Email netperf@jisc.ac.uk for further info on these •Or to suggest other tools that would help you •We’re currently building out 100G-capable (>10G) endpoints •Rack is set up, servers under installation, testing soon 10
- 11. So, a potential community? Over to you for discussion… •Is this is good and useful idea? •If so, what should the scope of the community be? •How often should we meet? •How should we meet? •Do we need a persistent chat mechanism, if so what? •What should we produce? (Noting Jisc can help authoring) •What should we call it? (RNIE isn’t so catchy…) •Anything else to consider? 11
- 12. Thank you •For general Jisc help contact your Jisc account manager •https://www.jisc.ac.uk/contact/your-account-manager •Or email help@jisc.ac.uk •For network performance, email: netperf@jiscmail.ac.uk •Or ping me at tim.chown@jisc.ac.uk 12