Understanding Intelligent Military-Grade Optical Ethernet Networks: A Versatile Solution for Achieving DoD's Net-Centric Operations Strategy

Understanding Intelligent Military-Grade Optical Ethernet
Networks:
A Versatile Solution for Achieving DoD’s Net-Centric Operations Strategy

Vishal Sharma, Ph.D. Shahram Davari, MASc.
Principal Technologist & Associate Technical Director,
Consultant Network Switching
Metanoia, Inc. Broadcom, Inc.
vsharma@metanoia-inc.com davari@broadcom.com
650-641-0082 (p)/650-641-0086 (f) 408-972-7436 (p)

1

Metanoia, Inc.
Critical Systems Thinking™

Understanding Intelligent Military-
Grade Optical Ethernet Networks:
A Versatile Solution for Achieving DoD’s Net-
Centric Operations Strategy
Vishal Sharma, Ph.D. Shahram Davari, MASc.
Principal Technologist & Consultant Associate Technical Director,
Metanoia, Inc. Network Switching
vsharma@metanoia-inc.com Broadcom, Inc.
davari@broadcom.com
650-641-0082 (p)/650-641-0086 (f) 408-972-7436 (p)
© Copyright 2010
All Rights Reserved

What We Will Discuss in This Tutorial
 Elements of DoD’s Net-Centric Data Strategy – key attributes and goals

 Requirements and Attributes of Military-Grade Networks
 Implications of the Above for
 Underlying Technology
 System Architecture and Features
 Network Architecture and Design

 Why Discuss Ethernet? Its Benefits and Applications

 Optical Ethernet
 3 Roles of Ethernet – Service, Transport, and PHY
 Carrier Ethernet and Optical Ethernet
 Macro-Architectural Options for Building MAN/WAN Interconnects & Key
Operational Principles

 Key Developments Valuable for Military Adoption of Optical Ethernet

 How Optical Ethernet Technology meets the Initial Requirements
©Copyright 2010
All Rights Reserved Milcom’10, October 31-Nov 3, 2010, San Jose, CA 3

Metanoia, Inc.

Attributes and Goals of DoD’s
Net-Centric Data Strategy

Core Elements of DoDs Net-Centric
Operations/Data Strategy (NCDS)
Proactively Collect User-
Feedback for Improvements

Handle Info. only Once for
Efficiency Visibility to a Wide Audience

Key Attributes
Facilitate Repurposing – Separate of DoDs Net- Rapid & Precise Discovery
Data from Applications Centric Data of Data
Strategy

Rich, Descriptive Meta-
Post-and-Process in Parallel data for Understandability

Shared-spaces for Posting
and Efficient Access
©Copyright 2010

Strategic Goals of DoD’s NCDS
Communities of Interest
- De-centralize data management to dynamically
formed user groups
- Allow prioritization /collaboration on data , based
on immediate operational needs
- Furnish infrastructure for self -synchronization
Institutionalized
Visible
- Establish procedures & policies for
effective data sharing - Discoverable

- Embed data-sharing precepts in the - Facilitate interaction with data for
organization analysis and decision -making insight

Responsive Accessible
Strategic Goals
- Ease of reaching data location
- React to fulfill user needs of the Net-
Centric Data - # of users who can consume data
- Satisfy needs relative to performance ,
content coverage & quality Strategy

Interoperable Understandable
- Shareability of data , while preserving
- Make meaning & purpose of data clear
accuracy , integrity, usability
via use of meta -data
- Understandability via semantic and Trusted
structural meta -data - Data is trustworthy
- Data integrity & quality is assured by backing
of a reliable organization /authority
©Copyright 2010

Metanoia, Inc.

Military-Grade Networks:
Requirements & Attributes

Key Requirements of
Military-Grade Networks
Simultaneous Support of Legacy
& Advanced Services
- Support legacy voice , POTS, low-speed satellite
backhaul links
- In parallel , allow for rich , multi-media traffic ,
Diverse Last-Mile Access video commn, sensor data

- Accommodate multiple access Rugged
technologies /media – copper , fiber, coax,
TDM, satellite , wireless - Hardened for harsh environments –
extreme weather , demanding conditions
- Uniformly aggregate traffic onto the
metro/core network - Need to operate in constrained spaces

Highly Available Military-Grade Secure
- Uptime: 99.9999% or more Networks: - Reliable , uncorrupted data
- Fast error detection and recovery Requirements - Tamper-resistant , high-integrity data

Manageable
- OAM capability Reliable
- Resilient to failures
- Ability to control network elements
& resources - Ability to recover automatically in
min. time
Fast Connection Setup Hard QoS & Determinism
- For dynamic and quick call setup - Controllable delay , jitter, and loss
- Setting up commun . over - Flexible bandwidth distribution across diverse
underlying infrastructure users/applications
©Copyright 2010

Metanoia, Inc.

Implications for Technology, and
System & Network Architectures

Implications of NCDS Requirements (1)

Implications for:
Technology System Design Network Architecture
Property
- Large address space to - Accommodate many end-nodes
support many end-nodes - Hierarchical design & traffic
Scalability - Large memory/processing for
- Capability to create engineering
1 (# locations, # address & routing tables
hierarchy - Support wide geographic reach,
users) - Capacity for large # of tunnels
- Control Plane for discovery seamless across access, metro,
& topology learning core

- Support encryption,
- E2e, segment, and/or Link authentication, ACLs
layer (local) security - DPI on line cards - Admission control
Security
- Isolate different users or - User data isolation - Authentication
2 (data integrity,
user classes - Intelligent memory partitioning - Architecture that integrates
trust)
- Enable detection of across users/functions firewalls, appliances with DPI
breaches - Provision against DoS/security
attacks
- Allow for Out-of-band (OOB)
Manageability - Provide robust OAM tools - Support OAM
control
3 (of network and - Management interface & tools/mechanisms
- Support a data communication
data) protocols (e.g. ELMI) - Permit remote access & mgt.
network (DCN)

©Copyright 2010

Implications of NCDS Requirements (2)

Implications for:
Property
Dynamic setup and
- Discovery
control of - Signaling
- Signaling, CP features - OOB network for signaling (if
4 communications - Dynamic/static tunnel setup
- Dynamic joining of mcast needed)
(within & across - NMS configuration features
groups (e.g. IGMP)
COIs)
Native mp2mp,
- Strategic placement of servers
p2mp - System-level brdcast, mcast
- Native broadcast, multicast (close to consumers)
communication with intelligent replication
5 capability - Redundancy of data (servers)
(for many-to-many - Multicast signaling support -
- Mcast signaling & QoS - Support redundant & disjoint
xchanges, mcast group creation/deletion
network paths
multicast)
- Support multiple i/f speeds
- Allow link bundling to
High-Speed at low - Large fabrics
enable higher speeds - High-speed links -- fiber
cost - Versatile, dense line cards
6 - Have standards for evolving - Support WDM
(rapid - High-rate processing
speeds - Enable link aggregation
communication) - Low power consumption
- Backward compatibility with
earlier i/fs

©Copyright 2010

Implications of Military-Grade
Network Requirements (1)

Implications for:
Property
- Robust conduction cooling
- Ubiquitous, with wide reach and
- Intelligent use of CPUs - Built with robust media
minimal constraints
1 Rugged - Off-load complex processing -- E.g. fiber -- inert, free from
- Delivarable over robust media,
security, protocols -- to central EMI/EFI
e.g. fiber
entity or add-on
- Standards for encryption, security
- Data plane and control plane
that are widely accepted/realizable,
robust to DDoS - Network and overlay mgt.
available
- Apply hardware-based encryption architecture must resist
2 Secure - Tunnel user data in real/virtual
- Isolate users via memory hacking/tampering
tunnels to effect isolation
partitioning, queue mgt., tunnels to - Have rapid alarm propagation
- Raise alarm/signal when data is
minimize data impact
tampered with
- Stds for signaling -- for restoration
- Support topologies supporting
- Setup & control multiple paths via
- Hardware/software redundancy - redundancy in data routing
signaling/NMS
e.g. LCs, fabrics, power supplies - Dual-homing, link aggregation
- Detect/react to faults, mis-routed
3 Reliable - Software redundancy - NSF, (e.g. LAG), multipath (e.g.
data
NSR, hitless upgrades ECMP) support
- OAM capabilities such as:
- Ability to detect/react to failures - 1+1, 1:1, 1:N, ring, mesh
connectivity check, loopback, link
protection
trace

©Copyright 2010


Implications for:
Property
- Support virtualization of
- Traffic isolation via queues,
network b/w (e.g. via tunnels,
scheduling - Support provisioning and
VLANs)
- Separate tables/memories to dimensioning
Hard QoS + - Ability (in technology, e.g. pkt
4 segregate traffic of different - CAC to regulate traffic vols.
Determinism hdrs) to mark, seggregate,
priorities, classes, apps. - Traffic engineering to support
prioritize, aggregate traffic
- Signal tunnels, and control/ traffic placement
- Support perf. measurement
manage tunnels
OAM
- Management constructs for - Control access to/sharing of
- Support remote config. &
config, monitoring system resources between
monitoring
5 Manageable - Measure loss, delay different user types
- OOB or in-band DCN
- Have loopback, link trace, - Create/config policy
- Hierarchical design
continuity check (e.g. Y1731) - Gather stats, diagnose problems

- Fast error detection at L1/L2/L3 - Support alternate routes/paths
- Error detection & config of - Detect h/w, s/w errors - Architecture to enable rapid
multiple alarms - L1/L2/L3 integration for fault recovery from failures (meshy-
6 Available
- Multipath routing/switching alarming architecture)
- Rapid switchover on failure - Hardware/software features to - Support intelligent/flexible multi-
aid redundancy layer protection

©Copyright 2010

Implications for:
Property
- Multi-service capable to support
variety of interfaces
- High-speed, cheap, easily - Intelligent interworking (type,
-- TDM, ATM, FR, IP, EPON/GPON
upgradable #, placement of devices)
-- and protocols
- Simple management or - Provide for aggregation
Diverse Last-Mile - Support vast range of data rates
7 unmanaged points/on-ramps for termination
Access - Ability to aggregate traffic
- Support aggregation of traffic, of diverse traffic and transfer to
(Appropriate processing in h/w and
while keeping different traffic a common (Ethernet, IP/MPLS)
s/w
types/classes seggregated core
Ability to queue & route data
- Enable clock distribution
appropriately)
- Support VPNs to facilitate COIs
- Advanced security mechanisms
- Support ckt emulation, clock
- Smart OAM
distribution - Architect for incremental
Support Legacy & - Virtual partitioning of network
8 - Advanced protection/restoration introduction of advanced
Advanced Services resources (for communities)
- Ablility to introduce new services services
- Scalable multicasting
by minimal system upgrades (e.g.
- Sophisticated security mechs.
just add/modify one LC)
- Versatile arch. -- uses
- Uses technologies with mass
technologies optimized per
adoption in non-military setting
- Leverage COTS segment
(e.g. Ethernet, IP, MPLS)
- Use std. building blocks/sub- - Intelligent policy
- Benefit from operational
9 Low Cost systems, components to benefit enforcement (via policy
experience, cost reductions
from adoption of vol. components servers)
- Use technologies with
(or hardened variants) - Plug and play operation
accumulated deployment
- Powerful signaling and
experience
control
©Copyright 2010

Metanoia, Inc.

Why Discuss Ethernet?
It’s Benefits and Applications

Why Ethernet?
Some Key Benefits …
 Mature technology  Native support of IP
 3 decades of operational experience,  Imp. for GIG and net-centric warfare
~300M+ ports sold in 2008 alone!  Simple IP address management
 Low-cost  Self-replacement capability
 Mass usage lowers cost, so  Largely backward compatible
compelling to use wherever possible
 Easy upgrades, integration of legacy
 High-bit rates & Range of speeds systems
 10 Mbps to 10 Gbps! (40-100 Gbps  Widely available COTS ecosystem
underway) – 3 orders of magnitude
 Easy to adapt commercial h/w & s/w
 Versatile for military use
 Usable as service, transport, PHY  Ethernet expertise widely available
 More discussion of this ahead ...  Network design, planning, architecture
 Provides consistent technology from  Network engineering, troubleshooting
edge-to-core  Practically unlimited interoperability
 Extends reach from LAN→MAN→
WAN
 Solves both networking & real-time
interconnect needs in military environ.

©Copyright 2010

Representative Applications of
Ethernet in the Military
 Switched Ethernet operates as:
 Networking infrastructure for MAN/WAN
 Real-time fabric interconnect in military systems, warfare systems, & military installments
 Critical building block for military devices

 1-10 Gb/s Ethernet used as “fat-pipe” between sub-systems

 Intelligent Ethernet transport adopted for:
 Support of IP-centric service requirements
 Evolution of wireless & fixed-line infrastructures USS Ronald Reagan
 Explicitly defined native Ethernet connections w/ reserved resources, dedicated protection

 Multi-layer Ethernet switches employed in support of DoD plans to leverage IPv6

 Ethernet technology facilitates delivery of:
 Real-time imaging, sensor data, video
 Secure mission-critical defense communication
AH-64 Apache

 Utilized for furnishing precision timing & sub-microsecond synchronization
©Copyright 2010

Metanoia, Inc.

Optical Ethernet Explained:
Three Roles and Its
Characteristics

Versatile Packet Networking with Ethernet
 Ethernet technology can play one of three roles in a data network

Ethernet Service – offered to end-customer, runs
e2e, where traffic flow into/out of customer systems Network Standards
comprises Ethernet frames Organization Technology/
Component
Involved Standard

Service MEF Carrier Ethernet

Ethernet Transport - Ability to switch/route IETF MPLS-TP
Ethernet frames of an Ethernet service, b/ween
network nodes by setting up connection- Transport IEEE PBB, PBB-TE
oriented, traffic engineered paths in the network ITU-T OTN-transport part
with deterministic perf.
IEEE 1GE/10GE/100GE
PHY
ITU-T OTN-PHY part

Ethernet PHY – framing and timing of actual bits of the
Ethernet frame, and their TX over the physical medium
to connect switches at the physical layer
©Copyright 2010

A Word on
Connection-Oriented Ethernet (COE)
 Ethernet transport enables the realization of COE

 COE – set of control-plane protocols & data-plane settings that
create a connection-oriented capability to transfer Eth frames

 Ethernet transport could involve:
 L2 transport -- Switching/routing traffic (data frames) by
 Enhancing Ethernet technology – e.g. PBB-TE (802.1aq)

 Using a different technology – e.g. MPLS, MPLS-TP

 L1 transport – switching/routing traffic at the physical layer (e.g.
optical channel data unit (ODU) level) by
 Embedding in a transport networking layer, such as ITU-T’s G.709 OTN

©Copyright 2010

Optical Ethernet Network Defined
 Network spanning a MAN/WAN that offers a carrier-grade Ethernet service,
running on a COE transport infrastructure over an optical PHY
 Optical PHY: OTN’s optical channel or an Ethernet PHY over optics
 Can be muxed onto fiber using CWDM/DWDM

“Optical Ethernet” Technology
Layers Examples
For p2p
services Carrier Ethernet
Service (E-line, E-LAN, E-Tree)
For p2p or
mp2mp services
Packet Transport
L2 Transport (PBB-TE, MPLS-TP)
Relationship of the Layers and
their corresponding entities
SONET/SDH, OTN
L1 Transport
transport

OTN-PHY part
L0 PHY
IEEE-Ethernet PHY)
©Copyright 2010

OK, So What is Carrier Ethernet?
Carrier Ethernet is therefore the service component of
optical Ethernet networks

Courtesy: Metro Ethernet Forum
©Copyright 2010

How Optical Ethernet Relates to
Carrier Ethernet
Carrier Ethernet: defined by MEF in 2004-05 as “Ubiquitous carrier-
grade Ethernet service with 5 attributes”:
 Standardized Services (better thought of as building blocks)
 Uniformly defined core services, building blocks for applications
 E-line, E-LAN, E-Tree (illustrated ahead)

 Scalability
 Span local, access, national, global range, with millions of MACs & UNIs

 Reliability
 Detect & recover from errors/faults, without impacting customers

 Hard QoS
 E2e performance for loss, delay, jitter, and b/w matching requirements of
voice, video, data traffic over heterogeneous networks
 Service Management
 Robust, standards-based, vendor-independent OAM to monitor, diagnose,
manage networks offering Carrier Ethernet service
©Copyright 2010

MEF’s Service Definitions
or Building Blocks
 MEF building blocks defined in terms of Ethernet Virtual
Connections (EVCs)

 EVC
 Association of two or more User Network Interfaces (UNIs) at the edge
of metro Ethernet network (MEN) cloud
 Exchange of Ethernet frames limited to the UNI’s in the EVC

 Three building blocks specified
 E-Line – p2p EVC
 E-LAN – mp2mp EVC
 E-Tree – p2mp EVC

©Copyright 2010

MEF’s Building Blocks Illustrated

EVC1

EVC2

Point-to-Point EVC (E-Line) Multipoint-to-Multipoint EVC (E-LAN)

Leaf
Root

Leaf

Rooted-Multipoint EVC (E-Tree)
©Copyright 2010

Putting it Together: Optical Ethernet
Network Components in Operation
Ethernet Service
(end-to-end; what the
user perceives)

Service
E-LAN
Service
Ethernet Transport Ethernet
(what the cloud delivers; the Service
“pipe” and its routing)

Transport

Switching/Routing
Optical (WDM) transport
PHY PHY Layer
(how the bits are transported (physical link, fiber)
between systems)

PHY

Framing, timing, and
optical muxing
©Copyright 2010

Metanoia, Inc.

Macro-Architectural Options for
Building MAN/WAN Inter-
connects & Design Principles
Involved

A Word on Network Architecture
 Ultimate goal of a network: to provide end-to-end
connectivity between two entities
 E.g. client-server, user-to-user, …

 Path between entities has many segments, comprising
 Access, aggregation, metro/edge, core

 Different technologies can be used in each segment,
depending on that segment’s requirements

©Copyright 2010

Applicability of Ethernet to
Network Segments
Network Segment
Access Aggregation Core
Parameters

Sophisticated systems
Cost Very cheap Relatively cheap
increase cost

High-speed, vast range High speeds/feeds, 1 Gb/ High speeds, 1 Gb/s – 100
Speed (10 Mbps – 1 Gbps) s – 10 Gb/s, link agg. Gb/s, LAG

Little or no mgt. needed
Comprehensive OAM Fault & Performance Mgt.
Manageability (plug-and-play)
portfolio OAM
Supports ELMI

Linear protection
LAG and Dual Homing Via RSTP, MSTP, ring
Redundancy (IEEE Work-in-Progress) protection (G.8032)
(G.8031), Traffic
engineering

Allows hierarchy (MAC- Via hierarchy, with inter-
Supports 4K services/
Scalability access link
in-MAC), Upto 16M operability with IP/MPLS
services (PBB-VPLS interworking)

Works over diverse Multiple logical rings,
Supports TE, routing
Notable Features access media (E.g. fiber, mesh natively supported,
extensions (e.g. PLSB)
Cu, wireless, coax, ...) native multicast

©Copyright 2010

Flexibility with Ethernet
 Ethernet has features that make it suitable for the 3 key
segments – depending on the operator’s need

 Adaptability of Ethernet implies
 Ethernet is not always needed end-to-end
 Usable in segments where it makes sense
 Incrementally extendable to other segments

 Interoperability of Ethernet  can inter-work with other
technologies for optimum realization of services

©Copyright 2010

Network Architecture Options with
Optical Ethernet
In the following, we
 Discuss key architectural options using Ethernet & optical
Ethernet

 Show how Ethernet migrates from the access (it’s forte) to
the metro and core

 Present the merits & assessment of each architecture

©Copyright 2010

Ethernet in Access: Operation
& Protocol Stack
Core
Metro Metro
Q-in-Q

Access MPLS/PW MPLS/PW MPLS/PW
Access
CE U-PE LSR N-PE N-PE LSR U-PE
CE

MPLS
X IP/MPLS
X
MPLS
IB-BEB

Spoke PWs per
VPLS instance

LSP-Label LSP-Label LSP-Label LSP-Label LSP-Label

VC-Label VC-Label VC-Label VC-Label VC-Label

C-DA C-DA C-DA C-DA C-DA C-DA C-DA

C-SA C-SA C-SA C-SA C-SA C-SA C-SA

S/C-Tag S/C-Tag S/C-Tag S/C-Tag S/C-Tag S/C-Tag S/C-Tag

Payload Payload Payload Payload Payload Payload Payload

©Copyright 2010

Ethernet in Access: Evaluation
 Doable today! and allows gradual “upgrade” to Ethernet in metro
and/or core

 Cheap, flexible, convenient – uses familiar Ethernet tech. in access

 Supports up to 2M services (due to 20b MPLS label) – not scalable

 Needs PWs/tunnels e2e, u-PE to u-PE – potentially millions – which
could become unmanageable

 Metro & core networks can be anything, but are typically IP/MPLS

©Copyright 2010

Ethernet in Access & Metro:
Operation & Protocol Stack
Metro Core Metro
PBB

Access Ethernet MPLS/PW Ethernet
Access
CE U-PE B-BEB N-PE N-PE B-BEB U-PE
CE

PBB
X IP/MPLS
X PBB
B-BEB B-BEB PBB

B-VID locally significant in
PBB, not sent over core Must support B-BEB
and VPLS capability

Internal B-VID,
B-BEB removes enables I-SID LSP-Label
PBB-specific bundling
B-Tag
VD-Label

B-DA B-DA B-DA

B-SA B-DA B-SA B-DA B-SA

B-Tag B-SA B-Tag B-SA B-Tag

I-Tag I-Tag I-Tag I-Tag I-Tag

C-DA C-DA C-DA C-DA C-DA C-DA C-DA

C-SA C-SA C-SA C-SA C-SA C-SA C-SA

S/C-Tag S/C-Tag S/C-Tag S/C-Tag S/C-Tag S/C-Tag S/C-Tag


©Copyright 2010

Ethernet in Access & Metro: Evaluation
 Implementable today, with selected hardware/software

 Allows gradual “upgrade” to Ethernet in core, if needed

 Cheaper, easier, lower cost & maintenance than previous
option (Ethernet in access only)

 Metro PBB network enables scaling of services, while
reducing processing/memory burden on metro/core devices

 Core network can be anything, but is typically IP/MPLS

©Copyright 2010

Ethernet Everywhere: Protocol Stack
Access Metro/Aggregation Core Metro/Aggregation Access
(802.1ad) (802.1ah) (802.1Qay) (802.1ah) (802.1ad)
Provider Backbone Provider Backbone Provider Bridging (PBB)
Provider Bridging (PBB) BCB
Bridging (PBB) Bridging (PBB)
Last Mile
Last Mile IB-BEB BCB B-BEB B-BEB B-BEB IB-BEB PE CE
CE PE

PB B-BEB BCB
PBB – Traffic
Engineered (PBB-TE)
PB
CE
PE PE CE
IB-BEB
IB-BEB BCB B-BEB B-BEB B-BEB
802.1ad/Q-in-Q 802.1ah 802.1ah 802.1ad/Q-in-Q
encapsulation BCB
encapsulation decapsulation decapsulation
B-DA B-DA - Pinned paths B-DA
- Based only on
B-SA B-SA B-DA, B-SA, B-Tag B-SA
- No STP
B-Tag B-Tag B-Tag
- No MAC learning
I-Tag I-Tag I-Tag
Switching based on pre -
C-DA C-DA C-DA C-DA C-DA
configured fwding tables
C-DA C-SA C-SA C-SA C-SA C-SA C-DA

C-SA S-Tag S-Tag S-Tag S-Tag S-Tag C-SA

C-Tag C-Tag C-Tag C-Tag C-Tag C-Tag C-Tag


©Copyright 2010

Ethernet Everywhere: Evaluation
 Uses proven, uniform technology throughout

 Ability to transport Ethernet & IP services (since Ethernet
supports IP)

 Benefits
 Easy procurement, management, troubleshooting
 Cost efficiencies (opex) from understanding, managing a single
technology in the network
 No technology interworking required!
 Supports link, segment, and e2e (service) OAM with one OAM
technology

©Copyright 2010

Ethernet in Mobile Backhaul
 Mobile backhaul architectures derive from the previous basic
types

 We examine them separately due to their unique needs:
 Interface with the core network
 Timing and synchronization requirements
 Evolution requirements – from TDM or ATM to IP/MPLS and/or
Ethernet

©Copyright 2010

Evolution of Cellular Technology
and Backhaul Types
Network Speed Interface

GSM/GPRS 56-114 Kbps TDM

EDGE 236 – 473 Kbps TDM

3G (UMTS/ 384 Kbps Uplink
ATM
WCDMA) R3, R4 384 Kbps Downlink
3G, R5 (HSDPA), 384 Kbps Uplink
IP/Ethernet
R6 (HSUPA) 14.4 Kbps Downlink
500 Mbps Uplink
LTE R8 (20 Mhz) IP/Ethernet
>100 Mbps Downlink

CDMA1X-RTT 100 Kbps TDM Legend

CDMA EV-DO 1.8 Mbps Uplink 2G
IP/Ethernet
Rev A/B 1.8 to 5 Mbps Downlink 2.5G
WiMAX (10 Mhz) 50 Mbps IP/Ethernet 3G
4G
Backhaul Types
©Copyright 2010

Mobile Backhaul Components
 Backhaul network – defined as the network that connects
 Base Transceiver Station (BTS, or Base Station) to Base Station Controller
(BSC) in 3GPP2 – GSM-based cellular networks
 Node-B to Radio Network Controller (RNC) in 3GPP – CDMA-based cellular
networks

 Traditional backhaul networks have used ...
 E1/T1 leased lines
 SONET/SDH TDM channels (for higher rate aggregation)

 Mobile transport infrastructure has hitherto been ...
 Microwave links
 Optical fiber with SDH/SONET

 Evolution to packet-based wireless services creates a push for the
transport itself to be packet-based: Ethernet or IP/MPLS or a combination

©Copyright 2010

Traditional Backhaul Evolution
2G BTS

BSC

TDM
TI/EI Cellsite SONET/SDH SONET/SDH
Gateway XConnect XConnect E1

T1/E1/STM SDH/SONET
Network
ATM RNC
ATM

nxE1

ATM
3G BTS Switch

Separate transmission facilities for different
technologies (TDM and packets)

©Copyright 2010

Evolved Backhaul Network
2G BTS

TDM
TI/EI Cellsite IP/Ethernet IP/Ethernet
To Wireless
Gateway Switch/Router Switch/Router
Core
BSC
ATM Carrier Ethernet
Network
1/10GE
CE 10/100/1GE PE PE Ethernet
nxE1 Ethernet

3G BTS

Ethernet

Common transmission infrastructure for different
technologies (TDM and packets)

3G/4G BTS

©Copyright 2010

A Quick Primer on PseudoWires
Label Label
5 VPN_ID = A Mapping Mapping VPN_ID = A 5
Label = 1004 Label = 2004
4 Targeted LDP

3 Discovery
PE1 PE2
1
6
ACs AC1
VC_LSP (2004)

ACn VC_LSP (1004)
2
VSI PW established VSI
VPN_ID = A VPN_ID = A
Tunnel LSP

1. Bind attachment circuit to Virtual 4. Targeted LDP session established
Switching Interface inside PE router 5. Mapping of label for the VC LSP
2. Assign each PE node a VPN id. (unidirectional virtual circuit (VC))
exchanged between end nodes
3. Nodes discover each other 6. PW established, data transfer enabled
©Copyright 2010

Pseudowires (PW) for Legacy Transport
2G BTS
PW
TDM
PSN Tunnel
TI/EI
To Wireless
AC Core
AC BSC
ATM Carrier Ethernet
Network
10/100/1GE 1/10GE
CE PE PE
Ethernet Ethernet
nxE1
Cellsite
Gateway
3G BTS AC: Attachment Ckt CE : Customer Edge (BTS)
PE: Provider Edge BSC: Base Station Controller
Ethernet
 PSN Tunnels
Encapsulation  May be IP/MPLS, T-MPLS/MPLS-TP, or
PB/PBB/PBB-TE based
3G/4G BTS  Structure-Agnostic TDM-over-IP
(SAToIP) (RFC 4553)  PW Signaling
 Structure-Aware TDM Circuit
Emulation (CESoPSN) (RFC 5086)  IEEE 1588-based timing distribution supported
 ATMoPSN (RFC 4717)
 SyncE (Synchronous Ethernet)
©Copyright 2010

MEF Services for Mobile Backhaul
RNC RNC

BSC BSC

Service
Multiplexing mp2mp EVC

Metro Ethernet
EVC EVC Metro Ethernet

BTS
BTS BTS BTS
BTS
EVPL Service for Backhaul using EVP-LAN Service for Backhaul using
Metro Ethernet Networks Metro Ethernet Networks

Services muxed at RNC UNI Needed when inter-BS communication
is permitted like in LTE/802.16m (WiMAX)
©Copyright 2010

MEF Services for Mobile Backhaul
RNC

BSC

Service
Multiplexing

Metro Ethernet
EVC EVC

BS/ BS/ BS/
BTS BTS
BTS

EVP-Tree Service for Backhaul using
Metro Ethernet Networks
©Copyright 2010

Metanoia, Inc.

Key Developments Valuable for
Military Adoption of Optical
Ethernet

Optical Ethernet: Recent Developments
 Ethernet technology evolving rapidly in the last 3-4 years

 Multiple standards bodies have created valuable stds in:
 OAM
 Interoperability
 Scalability
 Reliability
 Security
 New Services
 Last-mile high speed access
 Interworking
 New capabilities in Ethernet – synchronization/timing, automatic SLA
negotiation, Ethernet demarcation devices, Ethernet as xport

 We summarize these next, and focus on key ones valuable for the military
©Copyright 2010

Recent Advances in Optical
Ethernet Standards: Snapshot
Area Standard and/or Activity Stds. Organization(s)
Reliability/
Linear (G.8031) & ring (G.8032) protection ITU-T SG15
Protection
Connectivity Fault Mgt. (802.1ag), Perf. Mgt.
OAM IEEE, ITU-T SG 15
(Y. 1731)

Security LinkSec, MACSec, Authentication IEEE

Hierarchy via Shortest Path Bridging (PLSB)
Scalability IEEE
Provider Back-bone Bridging (802.1ah)
FCoE, Ethernet PWs, Circuit Emulation over Ethernet
Interworking IETF, MEF
(MEF 8)

New Services E-Tree (p2mp communication for multicast) MEF

Fast last mile access (EPON, 802.11n), HS i/fs
Higher-Speeds IEEE
(40G,100G)
SyncE (link-layer clock distribution)
1588v3 (network level time & clock distribution)
Demarcation devices (MEF NID)
New Capabilities IEEE, MEF, IETF
Automatic SLA negotiation (MEF E-LMI)
Ethernet as transport (PBB-TE)
MPLS-TP (Transport Profile): applicable for COE
©Copyright 2010

Ethernet Security:
LinkSec (MACSec, KeySec)
 Layer 2 link security standard defined by
 MACSec (IEEE 802.1ae)
 KeySec (IEEE 802.1af)

 MACSec provides:
 Origin authentication
 Data integrity checking
 Data confidentiality between two e2e Ethernet switches

 MACSec defines a frame format that includes data
encapsulation, encryption, authentication

 KeySec defines key mgt. protocol for MACSec

©Copyright 2010

Ethernet OAM
 Ethernet OAM supports Layer (domain) Monitoring
 Up to 8 layer levels (domains) per VLAN

 Ethernet OAM has tools for:
 Fault Management (802.1ag): CCM, LB, LT, AIS, RDI
 CCM: Continuity Check Message – verifies one-way connectivity
 LB: Loop Back – checks 2-way (round trip) connectivity
 LT: Link Trace – provides path (nodes) between nodes A & B
 AIS: Alarm Indication Signal – provides fwd alarm propagation
 RDI: Reverse Defect Indication – provides rev alarm propagation

 Performance Measurement (Y.1731): LM, DM
 LM: Loss Measurement – measures loss on an EVC
 DM: Delay Measurement – measures latency on an EV

©Copyright 2010

Ethernet OAM & Maintenance Domains
Customer Service Provider Customer

Access Core Access

Customer OAM Domain

Provider OAM Domain

Operator Operator OAM Domain Operator
OAM OAM
Domain Domain

Independent OAM can be run in each OAM domain for the same VLAN

IEEE provides for 8 levels of Maintenance Domains – allows a level to be
assigned to each entity – customer, provider, operator
©Copyright 2010

Ethernet OAM: Loopback (LB)
Example for Provider & Operator Domains
E2e Ethernet path
Customer Service Provider Customer

Access Core Access

Customer OAM Domain
Provider LB

Provider OAM Domain Operator LBs

Customer LB
Operator Operator OAM Domain Operator
OAM OAM
Domain Domain

Independent OAM can be run in each OAM domain for the same VLAN

We show operator, provider, and customer loopback examples above
©Copyright 2010

Synchronization in IEEE 1588
 1588: a protocol designed to synchronize real-time clocks in the nodes of a
distributed system that communicate using a network
 Synchronizes both – clock & Time-of-Day (SyncE only synchronizes clock)

Network

Master Slave/Boundary Slave/Boundary

©Copyright 2010

Metanoia, Inc.

How Optical Ethernet Meets Key
Technology Requirements of
Military Networks

Role of Ethernet Technology
Ethernet component provides several key capabilities
 Native mp2mp communication
 Easily creates communities of interest (COIs)

 Segregation of traffic and users
 Via VLANs (802.1ad) or B-VID/B-VLAN tags (802.1aq)

 Enables use a common infrastructure for multiple virtual
networks

©Copyright 2010

Role of Optical Technology
Optical component complements Ethernet packet technology, providing
strengths where Ethernet does not suffice
 Robustness against interference/EMI

 Tremendous bandwidth scalability
 Using an optical fiber transmission medium
 Via WDM technology, without needed additional fiber

 Connection-oriented nature
 Allows for traffic engineering
 Sophisticated, ultra-fast protection/restoration

 Long reach/range
 Reliable communication over long distances

 Facilitates deterministic timing/performance

©Copyright 2010

Suitability of Optical Ethernet
for the Military (1)
Military Network
How Today’s Optical Ethernet Technology Meets It
Requirement
- Hierarchy – via MAC-in-MAC encapsulation
Scalability
1 - Routing & Topology capability – PLSB, TRILL (MAC learning
in CP)

- MACSec – providing e2e security between nodes
Security
2 - ACLs – based on address, VLAN, port, …
- Queueing per VLAN, class, app., in systems
- Extensive OAM for fault & perf. management
Manageability - Service-level and link-level OAM, with hierarchy
3
- OOB management capability
- ELMI negotiation at UNI
- RSTP variants
- MSTP
4 Dynamic Setup & Control - ELMI for negotiation at UNI
- LACP helps setup link aggregation groups
- IS-IS in control plane for network topology control

Mp2mp and p2mp - Inherently mp2mp technology
5
communication - E-Tree service from MEF

- Economical deployment
6 Low-Cost
- Capex $1/ 1 Mb/s, which is ~1/4th of TDM circuit cost
©Copyright 2010

Suitability of Optical Ethernet
for the Military (2)
Military Network
How Today’s Optical Ethernet Technology Meets It
Requirement
- ITU-T link and ring protection
7 Reliability - EAPS (Ethernet Automatic Protection Switching), RFC 3619
- Link Aggregation (LAG)
- VLANs for virtualization
- Use of “p” bits for prioritization
- Bandwidth profiles (MEF) for queueing
8 QoS
- Per VLAN, per class traffic management
- Policing, shaping, dropping, metering, marking within
systems for differentiation between traffic
- Linear + Ring protection
9 Availability - EoWDM to increase reach, while decreasing BER
- Dual homing in access & E-NNI (network interfaces)
- P2p Ethernet
- Wi-Fi access
10 Diverse Last-Mile Access
- WiMAX access
- EPON
- Circuit Emulation over Ethernet (MEF8, SATOP,
Support of Legacy CESoPSN)
11
Services - Use of EtherType allows native encapsulation (of different
traffic types) within Ethernet. E.g. FCoE, PPPoE

- SyncE
12 Clock Distribution
- IEEE 1588v2

©Copyright 2010

Metanoia, Inc.

Summary and Conclusion

Wrapping it Up ...
 Optical Ethernet is today a well-established & well-known
technology, with many capabilities

 New capabilities being rapidly added, due to its versatility and
popularity

 Usable in access, metro, core, in mobile backhaul, data centers, ...

 Interoperable – so can be mixed-and-matched with other
technologies

 Suitable for net-centric, military applications

 Adds value in many applications, and a strong candidate to be
used where its characteristics fit the application or network
segment under consideration
©Copyright 2010

Metanoia, Inc.

Thank You!
Questions?

Metanoia, Inc.

Glossary

Glossary (1)
ACL Access Control List ELMI Ethernet Local Management Interface
BCB Backbone Core Bridge EPON Ethernet Passive Optical Network
BEB Backbone Edge Bridge EVC Ethernet Virtual Circuit
B-MAC Backbone MAC GPON Gigabit-capable PON
BSC Base Station Controller H-QoS Hierarchical QoS
BTS Base Transceiver Station Institution of Electrical and Electronic
IEEE
Engineers
B-VID Backbone Virtual ID
IETF Internet Engineering Task Force
CAC Connection Admission Control
IGMP Internet Group Management Protocol
CE Customer Edge
I-SID Individual Service ID
COI Communities of Interest
LAG Link Aggregation Group
COTS Common Off-The-Shelf
LC Line Card
DA Destination Address
LDP Label Distribution Protocol
DCN Data Communication Network
MEF Metro Etherent Forum
DoD Department of Defence
MEN Metro Ethernet Network
DPI Deep Packet Inspection
mp2mp Multi-point to Multi-point
DWDM Dense Wavelength Division Multiplexing
MPLS Multi Protocol Label Switching
e2e End to End
Multi-Protocol Label Switching -
ECMP Equal Cost Multi-Path MPLS-TP
Transport Profile

©Copyright 2010

Glossary (2)
MSTP Multiple Spanning Tree Protocol PON Passive Optical Network
NGN Next-Generation Network POTs Plain Old Telephone Service
NMS Network Management System PSN Packet Switched Network
N-PE Network-facing-Provider Edge device PW Pseudowire
NSF Non-Stop Forwarding QoS Quality of Service
NSR Non-Stop Routing RNC Radio Network Controller
Operations, Administration, and RSTP Rapid Spanning Tree Protocol
OAM
Maintenance
Resource Reservation Protocol - Traffic
ODU Optical Data Unit RSVP-TE Engineering (RSVP protocol with MPLS
OOB Out of Band traffic engineering extensions)

OTN Optical Transport Network SA Source Address

p2mp Point to Multi-point SDH Synchronous Digital Hierarchy
PB Provider Bridging
SONET Synchronous Optical Network
PBB Provider Backbone Bridging
SPT Shortest Path Tree
Provider Backbone Bridging - Traffic
PBB-TE STP Spanning Tree Protocol
Engineering
PE Provider Edge TDM Time Division Multiplexing

PHY Physical Layer Transparent Interconnection of Lots of
TRILL Links
PLSB Provide Link State Bridging https://datatracker.ietf.org/wg/trill/charter/

©Copyright 2010

Metanoia, Inc.

Appendix: Word on Provider
Bridging (PB) and Provider
Backbone Bridging (PBB)

Native Ethernet in Metro Access
 How does one create the notion of a virtual circuit?
 VLAN tagging with point-to-point VLAN

 VLAN stacking
 Outer tag ↔ service instance; Inner tag ↔ individual customer
 802.1Q in 802.1Q (Q-in-Q) - IEEE 802.1ad

6bytes 6bytes 4bytes 4bytes 4bytes

C-DA C-SA S-TAG C-TAG Client data FCS

C-DA: Customer Destination MAC
C-SA: Customer Source MAC
C-TAG: IEEE 802.1q VLAN Tag
C-FCS: Customer FCS
S-TAG: IEEE 802.1ad S-VLAN Tag
©Copyright 2010

Provider Bridge (IEEE 802.1ad)
Architecture

CE-B
CES
Customer
CE-A UNI-B Network
Customer
Network
CES
UNI-A

CES

Spanning tree

UNI-C

CE-C
CE: Customer Equipment

UNI: User-to-Network Interface
Customer
CES: Core Ethernet Switch/Bridge Network
P-VLAN: Provider VLAN

©Copyright 2010

Provider Backbone Bridging (802.1ah)
 Encapsulate customer MAC with provider MAC at edge
 Edge switch adds 24-bit service tag (I-SID), not VLAN tag

 Core switches need only learn edge switch MAC adds.

6bytes 6bytes 4bytes 5bytes 6bytes 6bytes 4bytes 4bytes

B-DA B-SA B-TAG I-TAG C-DA C-SA C-TAG Client data B-FCS

S-TAG: IEEE 802.1ad S-VLAN Tag
B-DA: IEEE 802.1ah Backbone Destination
B-SA: IEEE 802.1ah Backbone Source MAC
I-TAG: IEEE 802.1ah Service Tag

©Copyright 2010

Provider Backbone Bridging (PBB)
Architecture
CPE B CPE A CPE B
CPE A CPE C CPE D

Provider backbone Provider backbone
network (802.1ad) 802.1ad network (802.1ad)

Provider backbone
network (802.1ah)

Provider backbone
network (802.1ad) Provider backbone
network (802.1ad)
802.1q

CPE C CPE B
CPE B CPE A CPE D
CPE C
©Copyright 2010

Understanding Intelligent Military-Grade Optical Ethernet Networks: A Versatile Solution for Achieving DoD's Net-Centric Operations Strategy

More Related Content

What's hot (20)

Similar to Understanding Intelligent Military-Grade Optical Ethernet Networks: A Versatile Solution for Achieving DoD's Net-Centric Operations Strategy (20)

More from Vishal Sharma, Ph.D. (20)

Recently uploaded (20)

Understanding Intelligent Military-Grade Optical Ethernet Networks: A Versatile Solution for Achieving DoD's Net-Centric Operations Strategy

Editor's Notes