SlideShare a Scribd company logo
Provisioning Janet
Rob Evans, Chief network architect
19/10/2016
Where are we today?
»Janet6 in operation since 2013
› Transmission and IP layer both managed by Janet NOC
› “No hidden extras” for additional capacity
»Currently undergoing a ‘mid-term upgrade’
› Core links being increased
› Backbone routers being replaced
–Greater 100GE density
–ExistingT4000 platform has no development
› Increase in capacity to regions19/10/2016 2Campus network engineering for data intensive science
The Janet6 backbone
19/10/2016
Telehouse
North
Telehouse
West
Lowdham
Leeds
Equinix
HEX
Equinix
PG
Equinix
Manchester
Erdington
Glasgow
Bradley
Stoke
4x100GE
2x100GE
1x100GE
Shared
DC, Slough
Shared
DC, Leeds
3Campus network engineering for data intensive science
Weekly incoming traffic peaks
19/10/2016 4Campus network engineering for data intensive science
To use a more natural scale
19/10/2016 5Campus network engineering for data intensive science
Plumbing the tubes
»34Mbit/s: 2.64 inches
»155Mbit/s: 5.64 inches
»622Mbit/s: 11.4 inches
»2.5Gbit/s: 1.9ft
»10Gbit/s: 3.8ft
»40Gbit/s: 7.6ft
»100Gbit/s: 12ft
»200Gbit/s: 17ft
»400Gbit/s: 24ft
19/10/2016
400Gbit/s
200Gbit/s
100Gbit/s
40Gbit/s
10Gbit/s
6Campus network engineering for data intensive science
If Janet were a sewer…
LeeTunnel, 24 feet in diameter. Photo © Thames Water
19/10/2016 7Campus network engineering for data intensive science
What is it all?
»Much of this is ‘commodity’ IP
»Top graph: traffic between
Janet and GEANT
› Most external R&E traffic
(except dedicated ccts)
»One instance of an overlay
we’ll learn about later is
LHCONE
»Bottom graph: traffic between
Janet and LHCONE
19/10/2016 8Campus network engineering for data intensive science
That’s just the backbone
»Janet6 ensured we installed transmission nodes into the
RNEPs
› Or what used to be called RNEPs
»Regions still have a mix of fibre and leased lines
› Have tried to move towards fibre where we know there’s
a need
»Aiming to rationalise the architecture between backbone
and regions
19/10/2016 9Campus network engineering for data intensive science
Site access
»Aim for access circuits to be contention-free
› Sometimes usage surprises us
–Upgrades have lead times, even if we have fibre to the door
› Account managers are the interface between Jisc and
members
»QoS is not a bag of worms that we want to open
19/10/2016 Campus network engineering for data intensive science 10
Performance problems
»As far as the NOC is concerned ‘just another form of
network fault’
› Report in the usual way via the Janet Service Desk
»Realising that some form of faults require more specialist
knowledge
› Hence the end-to-end performance initiative
19/10/2016 11Campus network engineering for data intensive science
Options
»Can we continue to scale?
› Related: can we afford to continue to scale?
› Is big data anything more than feeding the current
exponential growth?
»Should we offload more traffic?
› Separate DTZ interfaces rather than sharing IP capacity?
»Should we encourage more ‘off-peak’ data transfers?
»Notice the question marks – topics to discuss!
19/10/2016 12Campus network engineering for data intensive science
Flattening the demand curve?
19/10/2016
Provision based
on this
This is never
used
13Campus network engineering for data intensive science
Where next?
»More flexible provisioning of end-to-end 10G paths
› Primary user of transmission layer was expected to be
the Janet IP layer
› Did not forsee scale of requirement and expected them
to be offloaded to Lightpaths / Netpaths
› Optical layer is designed for 40/100G+
–10G doesn’t use spectrum efficiently
19/10/2016 14Campus network engineering for data intensive science
Where next?
»Could be done on the transmission layer
› OTN
»Could be done on an ethernet layer
› SDN, EoMPLS
»Does capacity need to be guaranteed or just segregated?
»Smarter services rather than just bandwidth?
› On-net storage
19/10/2016 Campus network engineering for data intensive science 15
jisc.ac.uk
Questions and discussion
Rob Evans
Chief network architect
@internetplumber
Rob.Evans@jisc.ac.uk
19/10/2016 16Campus network engineering for data intensive science

More Related Content

PDF
Science DMZ at Imperial
PPTX
perfSONAR: getting telemetry on your network
PPTX
110G networking within JASMIN
PPTX
Archiving data from Durham to RAL using the File Transfer Service (FTS)
PPT
Solving Network Throughput Problems at the Diamond Light Source
PPTX
Enabling efficient movement of data into & out of a high-performance analysis...
PPTX
Network monitoring system demonstration
PPTX
Edupert best practices in supporting end users - Networkshop44
Science DMZ at Imperial
perfSONAR: getting telemetry on your network
110G networking within JASMIN
Archiving data from Durham to RAL using the File Transfer Service (FTS)
Solving Network Throughput Problems at the Diamond Light Source
Enabling efficient movement of data into & out of a high-performance analysis...
Network monitoring system demonstration
Edupert best practices in supporting end users - Networkshop44

What's hot (20)

PPT
IX Future: AMS-IX example. English version
PPTX
Dev ops, noops or hypeops - Networkshop44
PDF
Linac Coherent Light Source (LCLS) Data Transfer Requirements
PPT
Grid optical network service architecture for data intensive applications
PPTX
Science DMZ security
PPTX
Evolution of network - computer networks
PDF
HPCS16 - Frederick Lefebvre - Bridging the last mile
PDF
Science DMZ as a Service: Creating Science Super- Facilities with GENI
PDF
Common Design Elements for Data Movement Eli Dart
PDF
HKNOG 10.0: 30 Years of Internet in HK – A Quick Look Back at the First 20 Years
PDF
Network-aware Data Management for High Throughput Flows Akamai, Cambridge, ...
PDF
WINS: Peering and IXPs
PDF
Network State Awareness & Troubleshooting
PPTX
Altitude SF 2017: Building the edge
ODP
Clouds, Grids and Data
PDF
Micheal O'Foghlu - TSSG
PDF
LHCb on RHEA and T-Systems
PDF
LCG project description
PDF
Ben Evans SPEDDEXES 2014
PPTX
Guaranteed Greedy Routing in Overlay Networks
IX Future: AMS-IX example. English version
Dev ops, noops or hypeops - Networkshop44
Linac Coherent Light Source (LCLS) Data Transfer Requirements
Grid optical network service architecture for data intensive applications
Science DMZ security
Evolution of network - computer networks
HPCS16 - Frederick Lefebvre - Bridging the last mile
Science DMZ as a Service: Creating Science Super- Facilities with GENI
Common Design Elements for Data Movement Eli Dart
HKNOG 10.0: 30 Years of Internet in HK – A Quick Look Back at the First 20 Years
Network-aware Data Management for High Throughput Flows Akamai, Cambridge, ...
WINS: Peering and IXPs
Network State Awareness & Troubleshooting
Altitude SF 2017: Building the edge
Clouds, Grids and Data
Micheal O'Foghlu - TSSG
LHCb on RHEA and T-Systems
LCG project description
Ben Evans SPEDDEXES 2014
Guaranteed Greedy Routing in Overlay Networks
Ad

Viewers also liked (12)

PPTX
Challenges in end-to-end performance
PPTX
The Science DMZ
PPTX
Science DMZ
PPTX
Electron Microscopy Between OPIC, Oxford and eBIC
PDF
Protecting our customers - BT security
PPTX
Data and information governance: getting this right to support an information...
PPTX
Cyber Crime - "Who, What and How"
PPT
Role of the CISO in Higher Education
PPTX
Mitigation starts now
PPTX
Certifying and Securing a Trusted Environment for Health Informatics Research...
PPT
Working with students and ISO27001
PPTX
Closing plenary and keynote from Lauren Sager Weinstein
Challenges in end-to-end performance
The Science DMZ
Science DMZ
Electron Microscopy Between OPIC, Oxford and eBIC
Protecting our customers - BT security
Data and information governance: getting this right to support an information...
Cyber Crime - "Who, What and How"
Role of the CISO in Higher Education
Mitigation starts now
Certifying and Securing a Trusted Environment for Health Informatics Research...
Working with students and ISO27001
Closing plenary and keynote from Lauren Sager Weinstein
Ad

Similar to Provisioning Janet (20)

PPTX
Parallel session: supporting data-intensive applications
PPTX
Future services on Janet
PDF
"IPv6: What Does Success Look Like?" - ISOC Briefing Panel at IETF88
PPTX
Future services on Janet
PDF
FCoE or iSCSI doesn't matter, it's about the Ethernet.
PPTX
Future services on Janet
PPTX
Network Engineering for High Speed Data Sharing
PDF
Internet Exchange Points, presented at Peering Workshop at the PITA 29th AGM,...
PDF
The new Janet access infrastructure
PDF
Save 14million-through-wifi-offload
PPTX
presentation on Edge computing
PDF
IP Network Control Turning an Art into a Science (Customer Case Study)
PDF
Networking Challenges for the Next Decade
PDF
5G, A network, and beyond ...
PPTX
Its detroit2018 ndot
PPTX
Benefits of an Agile Data Fabric for Business Intelligence
PDF
IETF 112: Internet centrality and its impact on routing
PDF
Capacity planning
PDF
Institute for Communication Systems Home of 5G Innovation Centre
PPTX
PacketCloud: an Open Platform for Elastic In-network Services.
Parallel session: supporting data-intensive applications
Future services on Janet
"IPv6: What Does Success Look Like?" - ISOC Briefing Panel at IETF88
Future services on Janet
FCoE or iSCSI doesn't matter, it's about the Ethernet.
Future services on Janet
Network Engineering for High Speed Data Sharing
Internet Exchange Points, presented at Peering Workshop at the PITA 29th AGM,...
The new Janet access infrastructure
Save 14million-through-wifi-offload
presentation on Edge computing
IP Network Control Turning an Art into a Science (Customer Case Study)
Networking Challenges for the Next Decade
5G, A network, and beyond ...
Its detroit2018 ndot
Benefits of an Agile Data Fabric for Business Intelligence
IETF 112: Internet centrality and its impact on routing
Capacity planning
Institute for Communication Systems Home of 5G Innovation Centre
PacketCloud: an Open Platform for Elastic In-network Services.

More from Jisc (20)

PPTX
Strengthening open access through collaboration: building connections with OP...
PPTX
Andrew-Brown-JUSP-showcase-20240730.pptx
PPTX
JUSP Showcase - Rebuilding Data presentation
PPTX
Adobe Express Engagement Webinar (Delegate).pptx
PPTX
FE Accessibility training matrix partnership - information session
PPTX
Procuring a research management system: why is it so hard?
PPTX
Adobe Express Engagement Webinar (Delegate).pptx
PPTX
How libraries can support authors with open access requirements for UKRI fund...
PPTX
Supporting (UKRI) OA monographs at Salford.pptx
PPTX
The approach at University of Liverpool.pptx
PPTX
Jisc's value to HE: the University of Sheffield
PPTX
Towards a code of practice for AI in AT.pptx
PPTX
Jamworks pilot and AI at Jisc (20/03/2024)
PPTX
Wellbeing inclusion and digital dystopias.pptx
PPTX
Accessible Digital Futures project (20/03/2024)
PPTX
Procuring digital preservation CAN be quick and painless with our new dynamic...
PPTX
International students’ digital experience: understanding and mitigating the ...
PPTX
Digital Storytelling Community Launch!.pptx
PPTX
Open Access book publishing understanding your options (1).pptx
PPTX
Scottish Universities Press supporting authors with requirements for open acc...
Strengthening open access through collaboration: building connections with OP...
Andrew-Brown-JUSP-showcase-20240730.pptx
JUSP Showcase - Rebuilding Data presentation
Adobe Express Engagement Webinar (Delegate).pptx
FE Accessibility training matrix partnership - information session
Procuring a research management system: why is it so hard?
Adobe Express Engagement Webinar (Delegate).pptx
How libraries can support authors with open access requirements for UKRI fund...
Supporting (UKRI) OA monographs at Salford.pptx
The approach at University of Liverpool.pptx
Jisc's value to HE: the University of Sheffield
Towards a code of practice for AI in AT.pptx
Jamworks pilot and AI at Jisc (20/03/2024)
Wellbeing inclusion and digital dystopias.pptx
Accessible Digital Futures project (20/03/2024)
Procuring digital preservation CAN be quick and painless with our new dynamic...
International students’ digital experience: understanding and mitigating the ...
Digital Storytelling Community Launch!.pptx
Open Access book publishing understanding your options (1).pptx
Scottish Universities Press supporting authors with requirements for open acc...

Recently uploaded (20)

PPT
“AI and Expert System Decision Support & Business Intelligence Systems”
PDF
How UI/UX Design Impacts User Retention in Mobile Apps.pdf
PPTX
Programs and apps: productivity, graphics, security and other tools
PPTX
ACSFv1EN-58255 AWS Academy Cloud Security Foundations.pptx
PPTX
Effective Security Operations Center (SOC) A Modern, Strategic, and Threat-In...
PDF
Encapsulation theory and applications.pdf
PDF
Unlocking AI with Model Context Protocol (MCP)
PDF
Architecting across the Boundaries of two Complex Domains - Healthcare & Tech...
PDF
Dropbox Q2 2025 Financial Results & Investor Presentation
PPTX
Spectroscopy.pptx food analysis technology
PDF
Approach and Philosophy of On baking technology
PPTX
Big Data Technologies - Introduction.pptx
PDF
Agricultural_Statistics_at_a_Glance_2022_0.pdf
PPTX
Cloud computing and distributed systems.
PDF
Empathic Computing: Creating Shared Understanding
PDF
Blue Purple Modern Animated Computer Science Presentation.pdf.pdf
PDF
The Rise and Fall of 3GPP – Time for a Sabbatical?
PDF
Chapter 3 Spatial Domain Image Processing.pdf
PDF
KodekX | Application Modernization Development
PDF
Encapsulation_ Review paper, used for researhc scholars
“AI and Expert System Decision Support & Business Intelligence Systems”
How UI/UX Design Impacts User Retention in Mobile Apps.pdf
Programs and apps: productivity, graphics, security and other tools
ACSFv1EN-58255 AWS Academy Cloud Security Foundations.pptx
Effective Security Operations Center (SOC) A Modern, Strategic, and Threat-In...
Encapsulation theory and applications.pdf
Unlocking AI with Model Context Protocol (MCP)
Architecting across the Boundaries of two Complex Domains - Healthcare & Tech...
Dropbox Q2 2025 Financial Results & Investor Presentation
Spectroscopy.pptx food analysis technology
Approach and Philosophy of On baking technology
Big Data Technologies - Introduction.pptx
Agricultural_Statistics_at_a_Glance_2022_0.pdf
Cloud computing and distributed systems.
Empathic Computing: Creating Shared Understanding
Blue Purple Modern Animated Computer Science Presentation.pdf.pdf
The Rise and Fall of 3GPP – Time for a Sabbatical?
Chapter 3 Spatial Domain Image Processing.pdf
KodekX | Application Modernization Development
Encapsulation_ Review paper, used for researhc scholars

Provisioning Janet

  • 1. Provisioning Janet Rob Evans, Chief network architect 19/10/2016
  • 2. Where are we today? »Janet6 in operation since 2013 › Transmission and IP layer both managed by Janet NOC › “No hidden extras” for additional capacity »Currently undergoing a ‘mid-term upgrade’ › Core links being increased › Backbone routers being replaced –Greater 100GE density –ExistingT4000 platform has no development › Increase in capacity to regions19/10/2016 2Campus network engineering for data intensive science
  • 4. Weekly incoming traffic peaks 19/10/2016 4Campus network engineering for data intensive science
  • 5. To use a more natural scale 19/10/2016 5Campus network engineering for data intensive science
  • 6. Plumbing the tubes »34Mbit/s: 2.64 inches »155Mbit/s: 5.64 inches »622Mbit/s: 11.4 inches »2.5Gbit/s: 1.9ft »10Gbit/s: 3.8ft »40Gbit/s: 7.6ft »100Gbit/s: 12ft »200Gbit/s: 17ft »400Gbit/s: 24ft 19/10/2016 400Gbit/s 200Gbit/s 100Gbit/s 40Gbit/s 10Gbit/s 6Campus network engineering for data intensive science
  • 7. If Janet were a sewer… LeeTunnel, 24 feet in diameter. Photo © Thames Water 19/10/2016 7Campus network engineering for data intensive science
  • 8. What is it all? »Much of this is ‘commodity’ IP »Top graph: traffic between Janet and GEANT › Most external R&E traffic (except dedicated ccts) »One instance of an overlay we’ll learn about later is LHCONE »Bottom graph: traffic between Janet and LHCONE 19/10/2016 8Campus network engineering for data intensive science
  • 9. That’s just the backbone »Janet6 ensured we installed transmission nodes into the RNEPs › Or what used to be called RNEPs »Regions still have a mix of fibre and leased lines › Have tried to move towards fibre where we know there’s a need »Aiming to rationalise the architecture between backbone and regions 19/10/2016 9Campus network engineering for data intensive science
  • 10. Site access »Aim for access circuits to be contention-free › Sometimes usage surprises us –Upgrades have lead times, even if we have fibre to the door › Account managers are the interface between Jisc and members »QoS is not a bag of worms that we want to open 19/10/2016 Campus network engineering for data intensive science 10
  • 11. Performance problems »As far as the NOC is concerned ‘just another form of network fault’ › Report in the usual way via the Janet Service Desk »Realising that some form of faults require more specialist knowledge › Hence the end-to-end performance initiative 19/10/2016 11Campus network engineering for data intensive science
  • 12. Options »Can we continue to scale? › Related: can we afford to continue to scale? › Is big data anything more than feeding the current exponential growth? »Should we offload more traffic? › Separate DTZ interfaces rather than sharing IP capacity? »Should we encourage more ‘off-peak’ data transfers? »Notice the question marks – topics to discuss! 19/10/2016 12Campus network engineering for data intensive science
  • 13. Flattening the demand curve? 19/10/2016 Provision based on this This is never used 13Campus network engineering for data intensive science
  • 14. Where next? »More flexible provisioning of end-to-end 10G paths › Primary user of transmission layer was expected to be the Janet IP layer › Did not forsee scale of requirement and expected them to be offloaded to Lightpaths / Netpaths › Optical layer is designed for 40/100G+ –10G doesn’t use spectrum efficiently 19/10/2016 14Campus network engineering for data intensive science
  • 15. Where next? »Could be done on the transmission layer › OTN »Could be done on an ethernet layer › SDN, EoMPLS »Does capacity need to be guaranteed or just segregated? »Smarter services rather than just bandwidth? › On-net storage 19/10/2016 Campus network engineering for data intensive science 15
  • 16. jisc.ac.uk Questions and discussion Rob Evans Chief network architect @internetplumber Rob.Evans@jisc.ac.uk 19/10/2016 16Campus network engineering for data intensive science

Editor's Notes

  • #2: Background picture is of Janet6 staging, nodes destined for Manchester
  • #4: Mention next steps in the mid-term upgrades here. 6x100 on the southern part of the core, 4x100 on the northern, 2x100 to Glasgow and Bristol
  • #6: To meet this, the backbone bandwidth continues to grow. In the grand tradition of contrived metaphors…
  • #9: Definition of commodity IP is open to discussion How do we know if traffic from Google is a cat video or a genome visualisation? One site in particular, who will probably make themselves known later on.
  • #13: Affordability: most components are priced in US$ Leading onto the next slide.