Timing over packet demarcation

White Paper

Timing over Packet
Demarcation

An Innovative Approach to
Synchronization Performance Monitoring
in Packet Networks

Alon Geva, Advanced Technologies Manager
Yaakov Stein, Chief Scientist
RAD Data Communications

The Access Company

Abstract
There is a pressing need to distribute accurate timing, i.e., frequency and/or
Time of Day (ToD), across Packet Switched Networks (PSNs) for applications
such as cellular backhaul. Highly accurate delivery of timing over PSNs is
challenging, due to their asynchronous nature and the need to overcome
impediments, such Packet Delay Variation (PDV). This challenge increases for
operators leasing transport services from carriers, as operators have little
control over the timing packets as they traverse the carrier’s network and, as
a result, timing recovery accuracy cannot be guaranteed. While Ethernet OAM
and performance monitoring tools are available for carriers and wholesale
provides, no such demarcation functionality is presently defined specifically
for the timing distribution service.

This paper reviews the main issues involved in timing over packet (ToP)
demarcation and provides best practices for ToP demarcation and
performance monitoring.

White Paper: Timing over Packet Demarcation

Contents

Introduction ........................................................................................................................... 2
Legacy Synchronization Performance Monitoring ............................................................ 4
Synchronization over Packet Networks using PTP ........................................................... 5
PTP Performance Monitoring Today ................................................................................... 7
PTP Demarcation - The Future in PTP Performance Monitoring .................................... 9
Useful Practices of ToP Demarcation ............................................................................... 12
Summary .............................................................................................................................. 15
References ........................................................................................................................... 16

© 2011 RAD Data Communications Ltd 1


Introduction
There is a pressing need to distribute accurate timing, by which we mean frequency and/or Time of
Day (ToD), across Packet Switched Networks (PSNs) for applications and devices that require it.

One of the most demanding cases relates to cellular backhaul. The second and third generations of
cellular networks’ base stations require provisioning of frequency with error less than ±50 ppb
(parts per billion) compared to the network’s Primary Reference Clock (PRC). Base stations
belonging to the next generation of cellular networks, such as the UMTS-TDD and the TD-SCMA,
further require provisioning of ToD with error less than 1.5 microseconds compared to Universal
Time Coordinated (UTC) (see [1]). Future cellular networks will have even stricter requirements in
order to enable new features such as soft handoff between neighboring base stations and Multiple
Input Multiple Output (MIMO). ToD accuracy here will be required to be at a level of hundreds of
nanoseconds!

Delivery of timing of such accuracy over PSNs is challenging, due to their asynchronous nature. In
particular, PSNs based on a conventional Ethernet physical layer are not designed with timing
distribution in mind, and require additional mechanisms for its distribution. Packet-based timing
distribution protocols send packets containing timing information from a timing master, usually
referred to as Primary Reference Clock (PRC) or Primary Reference Time Clock (PRTC), to a timing
slave located close to the application device requiring the precise timing. The Network Time
Protocol (NTP) is the most prevalent synchronization distribution protocol when high accuracy over
a well engineered network is required (e.g., for cellular backhaul).

A major impediment to the proper function of packet-based timing distribution protocols is the
Packet Delay Variation (PDV) that timing packets experience when traveling through the PSN. This
PDV is primarily due to the timing packets being queued in network elements along their path. The
queuing delay at each network element depends on the momentary traffic load when the timing
packet needs to be queued, and thus varies randomly over time. Hence, the overall PDV, being the
sum of all the queuing delays, varies non-deterministically. In particular, the aforementioned
symmetry assumption cannot hold at any given time, and special filtering mechanisms must be
employed to mitigate this effect. Despite these mechanisms, end-to-end frequency and time
distribution performance cannot always be guaranteed to meet end-application requirements.

2 © 2011 RAD Data Communications Ltd


The problem becomes more severe when the service provider relies on a carrier’s infrastructure to
deliver its traffic to the end application. In such cases, the service provider has little control over the
timing packets as they traverse the carrier’s network and timing recovery accuracy cannot be
guaranteed.

Service providers continuously monitor Quality of Service (QoS) parameters experienced by user data
traffic in order to ensure conformance to an agreed upon Service Level Agreement (SLA). This is
usually accomplished by network demarcation devices running Operation Administration and
Maintenance (OAM) protocols (such as ITU-T Recommendation Y.1731 for Ethernet) to monitor the
provided service. OAM functionality may consist of Fault Management (FM) and /or Performance
Management (PM) functions. Fault Management consists of defect detection and reporting
mechanisms, such as Continuity Check (CC), Connectivity Verification (CV), Forward Defect Indication
(FDI), and Backward Defect Indication (BDI). Performance Management involves measurement of
metrics such as Packet Loss Ratio (PLR), one-way delay (1wD), Packet Delay Variation (PDV),
throughput, and availability.

OAM mechanisms have been defined for many network and service types, from low-rate TDM links,
through SONET/SDH networks, to Ethernet Virtual Connection (EVC) services. However, no such
demarcation functionality is presently defined specifically for the timing distribution service. As a
result, service providers rely on performance metrics derived from the timing slave to infer the timing
service’s performance. This can be problematic for several reasons. First, timing slaves, employing
different timing recovery algorithms, may display different performance, leading to inconsistent
results. Second, timing slaves are typically located at the far edge of the service provider’s network,
and thus their performance does not capture the individual behavior of specific segments in the
service provider’s network, but rather a sum of all degradations along the path from master to slave.
Third, the timing slave may be embedded in a network element (e.g., a switch or a router), or end-
application device (e.g., a cellular base station), and not provide visibility to the performance
functions that need to be monitored.

An alternative that addresses all of the aforementioned shortcomings would be to place a
standardized timing slave at, or close to, the required demarcation points in the network, and use it
to monitor the performance at the desired locations. However, as will be described in the next
section, this option would necessitate provisioning a dedicated timing distribution service from the
master to each such location, incurring additional bandwidth and logistics support. Furthermore, this
dedicated timing distribution flow will, in general, experience network impairments different from the
timing flow that it is intended to monitor, introducing an inherent error and inconsistency.



Legacy Synchronization Performance Monitoring
Timing distribution, just like any other service delivered over a telecommunication network, needs to
be continuously monitored in order to validate its correct behavior and conformance to the required
performance level. The on-going migration from synchronous Time Division Multiplexing (TDM)
networks, which inherently distribute frequency, to asynchronous Packet Switched Networks (PSNs)
introduces major challenges for service providers. Not only must they now employ protocols to
distribute frequency, and in many cases ToD, they must also be able to monitor the quality of such
timing services.

Frequency distribution in synchronous networks is a mature technology that relies on the stringent
frequency accuracy requirements of data symbols transmitted over physical links. Frequency
information is automatically distributed downstream from a master clock to the end-application on a
hop-by-hop basis. Each network element along the path recovers the frequency of the symbols it
receives, enslaves a local oscillator to this frequency, and uses this oscillator as its frequency
reference for downstream transmission. This results in a timing (or synchronization) chain, consisting
of local node clocks connected by physical links. At the top of the timing chain usually stands a highly
accurate and stable autonomous clock called the Primary Reference Clock (PRC), usually an
autonomous atomic clock. Thus, the frequency at every point along the timing chain is said to be ‘PRC
traceable’.

Monitoring frequency accuracy and stability at a specific network element may be accomplished by
comparing the frequency of its local clock to a trusted frequency reference, such as another stand-
alone atomic clock or frequency output from a GPS receiver. The comparison is often carried out by
continuous phase comparison of the two signals. The outcome of this continuous phase comparison
can then be used to calculate frequency quality metrics, such as the Maximum Time Interval Error
(MTIE) and Time Deviation (TDEV) ([2]). Measurement equipment designed for this task is readily
available, and is used by service providers to monitor and validate the quality of frequency
distribution at various locations within their network, and thus to trigger corrective actions whenever
anomalous behavior is detected.

A legacy SONET/SDH frequency monitoring system is presented in Figure 1. The monitored clock
signal is introduced into a frequency monitoring device (wander analyzer) that is also referenced with
an accurate reference clock (i.e. a PRC). The outcome phase error measurement (Time Interval Error -
TIE) is offloaded to a PC (or any other metric calculation device) for deriving MTIE and TDEV frequency
accuracy/stability metrics. These metrics may then be used to verify the performance of the
frequency distribution system and detect abnormalities.



Figure 1: Typical SONET/SDH frequency monitoring system

Synchronization over Packet Networks using PTP
Distribution of timing (frequency and/or time of day) in packet networks is achieved by a timing-over-
packet protocol. Such protocols function by sending timing packets between a timing master and a
timing slave. The timing master is usually connected to a highly accurate and stable frequency/time
reference (PRC or PRTC). The timing packets must be formatted as valid packets for the network, and
are transported from master to slave in the same fashion as all packets in the network.

When it comes to very accurate distribution of frequency and time over packet networks, the most
prominent timing-over-packet protocol is the Precision Time Protocol (PTP) version 2 (also known as
IEEE 1588v2 or IEEE 1588-2008 - [3]). The 2nd version, ratified in 2008, is Telecom oriented in the
true sense and includes various features and mechanisms making it an attractive apparatus for
Telecom applications.



In PTP, timing information is carried by three aspects of the timing packets, namely timestamps
carried explicitly in the packets, the precise transmission time, and the precise reception time of the
packets (by the master or slave device). The difference between the latter two properties is sensitive
to Packet Delay Variation (PDV) that obscures the master-to-slave and slave-to-master propagation
latencies, complicating the timing recovery process.

Figure 2 depicts the packets transactions for PTP. The transaction is initiated by the PTP master
(which is timed by a PRC/PRTC) sending a timing packet to the PTP slave. Timestamp t1 is recorded by
the PTP master as the first timing packet passes a well-defined (master) reference point on its way
out to the network. Timestamp t2 is recorded by the PTP slave as soon as the first timing packet
passes the slave’s reference point after reaching the PTP slave. It should also be noted that t1 is
explicitly carried by the first timing packet, and thus becomes known to the PTP slave (if the PTP
master is incapable of accurately timestamping the outgoing packet on-the-fly, a follow-up packet
containing t1 can be sent).

Figure 2: PTP packets transaction

In the reverse direction, timestamp t3 is recorded by the PTP slave as the second timing packet passes a
well-defined (slave) reference point on its way out to the network, and timestamp t4 is recorded by the
PTP master as the second timing packet passes a well-defined (master) reference point on its way in
from the network.



After the second timing packet has been received (by the master) and timestamp t4 is recorded, a third
timing packet is sent by the PTP master to the PTP slave, informing it with the value of t4. At this time
the PTP slave knows all four timestamps {t1, t2, t3, t4} and is able (under symmetry assumptions) to
calculate the network delay between the master and itself, and thereafter derive the instantaneous
time-error of its local clock. As before, the slave can correct its local clock’s frequency and/or time to
match that of the master.

PTP Performance Monitoring Today
Similar to what we discussed for TDM networks, in order to monitor timing performance at some point
along the network path between master and slave, a comparison must be carried out between a
physical timing signal (i.e., a clock signal) recoverable at that point and a trusted timing reference.
However, as timing distribution over PSN is not necessarily performed on a hop-by-hop basis,
intermediate network elements do not necessarily perform timing recovery and do not necessarily
maintain local clocks. To produce the required physical timing signal at the desired location, the
monitoring system itself could perform timing recovery, essentially reproducing the process carried out
by the slave. Alternatively, the timestamp information collected at intermediate locations could be used
to directly calculate packet network performance metrics (specifically PDV-based metrics) that could be
used to predict the attainable timing performance. The use of such PDV-related metrics is relatively
new, and is presently the subject of intensive research. Several promising metrics such as minTDEV and
MAFE (currently planned to be incorporated into the next version of ITU-T Recommendation G.8260
([4]) have been proposed and shown to possess desirable properties.

Unlike synchronization monitoring in TDM networks that relies on the existing timing flow, producing a
physical timing signal at desired intermediate locations cannot be performed by simply tapping onto the
content of existing timing over packet flows. One way to overcome this is to introduce a dedicated
timing packet flow from the timing master to the intermediate location, since existing timing-over-
packet protocols require active interaction between master and slave. Such a PTP timing monitoring
arrangement is depicted in Figure 3, where the PTP master serves both the PTP slave and the ToP
monitoring device using two separate timing flows. In this scheme, the ToP monitoring device appears,
from the timing master’s point-of-view, to be just another PTP slave. The overall timing bandwidth
increase is directly proportional to the number of active PTP monitors installed in the network.
Moreover, as separate independent timing flows are used for the PTP slaves and the ToP monitoring
devices, the true PDV experienced by the PTP slaves is not really being monitored.



Figure 3: PTP performance monitoring using an intermediate slave device

A second approach would be to use standard network OAM protocols (e.g. ITU-T Y.1731[5]) to
monitor network PDV between two demarcation points. Such an approach is presented in Figure 4
where standard OAM packets sent from one MEP (Management Entity Group End-Point), located at
one edge of the network, to a seconds MEP, located at the opposite end, are used to measure
relevant PDV information and compute the required PDV metrics based on that. The problem is that,
here too, the measurements are being carried based on synthetic OAM packets rather than the
original PTP flow we want to monitor. Moreover, any Transparent Clock function located between the
demarcation points will not be taken into account as it will be completely agnostic to the OAM
packets.

Figure 4: PTP performance monitoring using standard OAM techniques



PTP Demarcation - The Future in PTP Performance Monitoring
The new patent pending approach enables monitoring quality and achievable performance of a timing
distribution service in packet networks at any given point in the network, without requiring
introduction of additional traffic.

At the heart of the new approach is a network device, hereinafter referred to as a Timing-over-Packet
Demarcation Function (ToPDF). The ToPDF contains two network interfaces, to be called the Master-
Facing Interface (MFI) and the Slave-Facing Interface (SFI), which enable it to be connected as a
“bump on the wire” at any desired location in the network. The network interfaces can be of any
type depending on the specific physical and link layer technologies employed. The ToPDF is
completely transparent and does not modify monitored packets in any way. Furthermore, the ToPDF
does not interact with other network elements, and does not generate any additional traffic load on
the network.

The ToPDF has the ability to identify incoming and outgoing timing packets (from both its network
interfaces), to extract any information from these packets, and to record accurate timestamps. Based
on the extracted information and the timestamps, the ToPDF can perform calculations in order to
derive the following parameters:

• Timing status of the network

• Timing metrics that might be useful in predicting the currently achievable timing distribution
quality over a given path

• Actual recovered frequency and time achieved in this specific location of the network

Such information can be then used by the service provider to better assess the condition of the
timing distribution in its network.

The ToPDF can be placed at any desired location in the network. In some cases the ToPDF would be
located at (or near) a Customer Edge (CE) switch or router, forming a demarcation point between the
provider network and the customer network. In others, it will be placed at the border between two
service provider networks, or at strategic points inside a single network.



We further assume that one or more timing packet flows traversing the ToPDF in both directions
(going from master to slave and back). The ToPDF is able to detect the dedicated timing packets that
traverse it in both directions, but does not alter them in any way (and certainly does not alter non-
timing packets). However, the ToPDF performs the following non-intrusive operations on timing
packets:

• Extract information from the timing packets, in particular, flow identifiers and timestamps

• Timestamp the timing packets as they traverse well-defined ingress/egress reference points,
based on its local clock

It should be noted that such non-intrusive detection, data-extraction and timestamping mandates the
use of dedicated hardware processing, in order to minimize the additional delay introduced by the
ToPDF and make it completely fixed and symmetrical (This requirement can be easily accomplished for
the PTP case, using an End-to-End Transparent Clock).

In the following discussion, a “timing flow” will mean any persistent flow between a timing master
and a timing slave that distributes timing information from master to slave. Moreover, a “raw
timestamp set” will mean a specific combination of four timestamps {t1, t2, t3, t4} that contains the
complete timing information exchanged between a given timing master and a given timing slave.

Figure 5 presents the ToP demarcation concept with a PTP distribution system for a typical cellular
backhaul scenario where the cellular service provider is buying wholesale transport services from a
local carrier. In this example, the PTP master is part of the carrier network (part of the wholesale
service) and the PTP slave is located at the base station (part of the customer network). The ToPDF is
embedded in the carrier network’s PE router that delimits the two networks, with access to the PTP
distribution timing flows. Thus, it can identify a specific timing flow and generate its own local
timestamps (based on its local clock) referenced to some pre-defined timestamping reference point
on its MFI. Thus, when the entire packet transaction between the PTP master and PTP slave is over
and the slave has the required set of timestamps {t1, t2, t3, t4} to calculate instantaneous time-error
between itself and the master, the ToPDF will hold the required set of timestamps {t1, t’2, t’3, t4} to
calculate instantaneous time-error between itself and the master. Additionally, using this set of
timestamps, the ToPDF can now derive all the required 1-way/2-way network delay and PDV metrics.



Figure 5: PTP ToP demarcation

The ToPDF will also easily handle any intermediate TC nodes by looking at the Correction Field (CF) in
the header of the traversing PTP packets. As described in Figure 6, by extracting the CF values from all
the PTP messages comprising a complete PTP transaction (Sync, Delay Request and Delay Response),
the ToPDF has all the information it needs to take the TC delay compensation for both the
master ToPDF ( ) and ToPDF master ( ) network paths.



Figure 6: PTP ToP demarcation with TCs

Useful Practices of ToP Demarcation
In many cases, end-to-end PTP frequency and time distribution is the only real possibility for the near
and interim future, resulting in a PDV-sensitive solution whose bottom line performance cannot be
always guaranteed. This poses a nasty ‘headache’ for the service provider: how can the
time/frequency distribution service be used in such a way that would guarantee the required level of
performance? More importantly, how can we monitor the service in order to be able to warn that the
service has been degraded, isolate problems and tell where in the network the degradation occurred?

The answer to these questions might seem difficult at first glance. Clearly, as was already discussed,
packet timing flows cannot be naturally monitored by legacy time/frequency-error monitoring devices.
Moreover, with standard end-to-end PTP distribution, only the slave device at the end of the
distribution chain can be monitored, leaving the entire network section under a thick curtain of
uncertainty, and in many other cases even that is not true in reality (for example, when the cellular
base station does not offer any external monitoring/probing capabilities for the integrated PTP slave).



The good news is that using the new ToP demarcation concept the above questions can be mapped
into the following almost equivalent one: Where in our network should we deploy ToP demarcation
entities in order to get:

a. Best coverage of the network (the timing distributing part)

b. Optimal dissection of the network for best detection of fault conditions

Understandably, ToP demarcation entities may be located in many different network locations along
the timing flow(s) distribution path and, indeed, such practice may be extremely useful in order to
isolate timing problems. For example, Figure 7 depicts a PTP timing flow (from a PTP master located
at the 1st provider network to a PTP slave located in the customer network) that traverses three
networks: The first network is operated by provider #1, the second network is operated by provider
#2, and the third network is operated by the customer. By placing ToPDFs at the (two) demarcation
points of these networks, as well as another ToPDF at the edge of the customer network (its SFI
directly connected to the ‘First-Mile’ link), one can estimate the contribution of each of the networks
to the timing distribution degradation, and in particular one can isolate the source of major
performance degradations.

The last ToPDF, located close to the end-application device that comprises the PTP slave, experiences
PDV similar to the PTP slave and thus should provide a reliable estimate of the overall end-to-end
timing recovery performance. This can be exploited by the cellular service-provider to appreciate the
current condition of the PTP slave even when it is embedded in the end-application device (cellular
base station in this case) and its status/recovery outcome is not directly accessible.

The second ToPDF, located at the demarcation point between the provider networks and the
customer network, experiences the PDV introduced by the provider networks only. Bidirectional PDV
metrics computable by ToPDF2, at this point, can be used by the providers to ensure the quality of
service they provide, and by the customer to monitor the service it is being provided.

The first ToPDF, located at the demarcation between the first and second provider networks,
experiences PDV introduced by provider #1’s network only. Bidirectional PDV metrics computable by
ToPDF1, at this point, can be used by the two providers to predict the timing performance that would
have been attained by a PTP slave served only by the first provider.



Figure 7: Multiple ToP demarcation entities along a synchronization distribution chain

The PTP performance monitoring approach described here is only one possible practice out of many
that could be used to closely monitor PTP time/frequency distribution. Clearly, the selected
approach needs to match the underlying network and timing distribution architectures. The
appealing thing about the ToP demarcation technology is its high adaptation capability to almost
any given network device, from large aggregation switches down to SFPs (see Figure 8), as well as
to any timing distribution topology. All that is really needed is to connect the network-
element/module containing the ToPDF to a ‘bump on the wire’ on the desired network link.

Figure 8: ToPDF elements



From this point on, the ToPDF will continuously monitor the selected timing flows, sending its
findings and collected data to a central NMS/PM portal that will synchronize, integrate and analyze
all collected data (from all the ToPDFs), and supplying the user with concurrent, on-going and
comprehensive information about the synchronization distribution status in its network (Figure 9).

Figure 9: Multiple ToP demarcation entities along a synchronization distribution chain

Summary
RAD has developed an innovative PTP performance monitoring technique allowing carriers and service
provides to closely monitor the Timing over Packet distribution in their networks, making sure it is
working well and supplying the required level of performance.

Contrary to existing solutions that are only end-to-end, RAD’s PTP PM solution can provide deep
insight to the contribution of each network segment to the overall service degradation, allowing the
user to quickly isolate problems and take proactive measures to solve them. For more information,
please contact market@rad.com.



References

[1] 3GPP TS 25.402 version 5.2.0 Release 5,

[2] ITU-T Recommendation G.810 (08/96), definitions and terminology for
synchronization networks.

[3] IEEE 1588-2008 (July 2008), IEEE Standard for a Precision Clock Synchronization
Protocol for Networked Measurement and Control Systems.

[4] ITU-T Recommendation G.8260 (08/2010), Definitions and terminology for
synchronization in packet networks.

[5] ITU-T Recommendation Y.1731 (07/2011), OAM functions and mechanisms for
Ethernet based networks.


www.rad.com

International Headquarters North America Headquarters
RAD Data Communications Ltd. RAD Data Communications Inc.
24 Raoul Wallenberg St. 900 Corporate Drive
Tel Aviv 69719 Israel Mahwah, NJ 07430 USA
Tel: 972-3-6458181 Tel: (201) 529-1100,
Fax: 972-3-6498250 Toll free: 1-800-444-7234
E-mail: market@rad.com Fax: (201) 529-5777
www.rad.com E-mail: market@radusa.com
www.radusa.com

The RAD name and logo is a registered trademark of RAD Data Communications Ltd. © 2011
RAD Data Communications Ltd. All rights reserved. Subject to change without notice. Catalog
The Access Company no. 802478 Version 11/2011

Timing over packet demarcation

More Related Content

What's hot (18)

Viewers also liked (20)

Similar to Timing over packet demarcation (20)

More from Nir Cohen (18)

Recently uploaded (20)

Timing over packet demarcation