PTP Synchronization Performance Evaluation with Process Bus Load

Summary

This work approaches the synchronization of the Process Bus using Precision Time Protocol (PTP). Several tests were performed to evaluate PTP performance in the Process Bus with several scenarios of Ethernet network loading. The test system consists of Switch, GPS and test sets for both Master / Slave IEEE 1588 and to simulate MUs for network loading. An external time reference device was used to compare the master and slave clocks in different loading scenarios.

Tables and statistical analyzes were used to resume the results. The paper aims to answer the question: does the Process Bus delay the PTP? 

Keywords

PTP, Time Sync, Process Bus, Ethernet Load

1. Introduction 

The IEC 61850 defines that SAS (Substation Automation System) is divided into three levels: process, bay and station. At the process level are devices such as sensors, actuators, CTs and VTs and circuit breakers, which act as input and output data. In the bay are IEDs (Intelligent Electronic Devices) that perform diverse functions like protection, measurement and control. At the station is the supervisory system that performs a general monitoring of the substation and communicates with external control. Interfacing with the process and bay levels is the Process Bus that brings data from CTs, VTs, actuators and etc to the IEDs in the bay. To interconnect the bay to the station levels there is the station bus connecting the IEDs to the supervisor.

The Sampled Values (SV) are defined in the IEC 61850-9-2 as Ethernet frames containing the digitalized values of current and voltage of the instrument transformers. Due to the implementation of IEC 61850 the paradigm was changed, because the system no longer works with analog signals of secondary, but with sampled values sent by the Ethernet network. In this way, hard cabling by copper wiring was replaced by network cables, resulting in money savings and simplicity of connections.

The SV messages are standardized in order to allow interoperability. In addition, they are Multicast frames with fixed range of Destination MAC addresses defined by the standard and run only on the second layer of the Ethernet network (Link layer), being of high priority and with critical time.

The SV implementation becomes necessary to use Non-Conventional Instrument Transformers (NCITs) or Stand Alone Merging Units (SAMUs). Depending on the technology used, there are different techniques to implement SV. These different forms are dealt in IEC 61869-9: Digital Interface for Instrument Transformers, published in 2016 and which is complementary to IEC 61850-9-2.

IEC 61850-9-2 establishes that all MUs must be synchronized to assume that SV are correctly processed by IEDs. The IEC 61869-9 defines PTP as the preferred synchronization method, regarding 1PPS as alternative for legacy applications.

Among all synchronization protocols that can be used in substations, with the raise of IEC61850-9-2 SV IEEE 1588 PTP becomes more and more relevant. This protocol can be implemented using only the second layer (Ethernet) or using both second and third layer (IP) of OSI model. The PTP differs from others network synchronization protocols (such as SNTP) by the accuracy achieved due to the Master / Slave relationship.

This paper evaluates the performance of PTP applied in process bus under some Ethernet network scenarios. The test scenarios were performed by PTP capable Switch, GPS and test sets verifying the Master / Slave synchronization under several situations changing SV traffic, using or not Virtual LANs and changing who is Grandmaster clock. An external time reference device was used to perform the comparison between the Master clock and the Slave clock, combining a frequency counter and an oscilloscope. Therefore, the result of this study is expected to contribute with the knowledge for the application of this time synchronization protocol in substations based on IEC 61850.

2. Synchronization

IEC 61850-9-2 establishes an operating standard on the SAS process bus and, according to the standard, the MUs must be synchronized in time to guarantee that the SV are processed properly in the IEDs. The sample counter (SmpCnt), whose value is incremented for each sample acquired and inserted in the SV frame, represents the exact moment that each sample was acquired. At each turn of the second, the SmpCnt is reseted. Figure 1 [1] demonstrates the SmpCnt implementation algorithm.

The IED that receives the SV uses the SmpCnt to align the samples in time and thus reconstitute the waveform, becoming independent of any frame transmission delay through the Ethernet network that works with statistical switching. This allows SV packets to travel through different paths on the network and have different transmission times. All algorithms for phase angle verification are done through this mechanism.

The time synchronization importance in the Process Bus can be exemplified on a differential protection function or in breaker-and-a-half topologies, which lead to a decision-making by receiving signals from more than one MU at the same time, where it is imperative that the samples are time aligned.

Time synchronization is specifically important for the case of differential protection in the context of Process Bus of IEC 61850. This is an important case to be highlighted, due to the MUs are not synchronized, the same SmpCnt will be sampled by each MU at different times, which could lead the IED to an incorrect trip due to an erroneous phase difference.

In addition, time synchronization is used to precisely align the internal clock of IEDs, MUs, switches, perform analysis of discrete oscillographs and any process that needs to be synchronized in the SAS. Time synchronization assists the analysis of: how, where and when a fault occurred.

Regarding time synchronization in SAS, Table 1 [2] discriminates the time accuracy requirements in some applications.

The GPS (Global Positioning System) is the source of reference clock for all other devices in process bus. SAS time synchronization system is basically composed by GPS, Switch, MUs and IEDs.

For commissioning and testing of time, GPS plays a fundamental role allowing the verification of synchronism of each system part. As GPS is the most accurate device in the synchronization system, only another GPS can be used as a reference to verify its accuracy. The other parts of the system: Switch, MUs and IEDs can have the synchronism verified through the IRIG-B and 1PPS outputs of the GPS.

Among the main SAS timing protocols, IRIG-B, 1PPS, SNTP and IEEE 1588 PTP can be mentioned.

2.1. IEEE 1588 PTP

The Precision Time Protocol (PTP) is a synchronization protocol for Ethernet networks, and in a local network it reaches accuracy in nanosecond range. It is suitable for applications where timing is critical for the measurement system. The high accuracy of the protocol is obtained by compensating the propagation delay of the information between the sync source and the destination.

The IEEE Standard 1588 - 2002 Standard for a Precision Clock Synchronization Protocol for Networked and Control Systems originally defined the PTP protocol and its 2008 revision (2 nd version) added improvements of accuracy and robustness to the protocol.

The protocol is defined for a hierarchical time distribution network with master-slave relationships, where a Grandmaster clock is chosen as the synchronization source for all other devices connected to the same network.

According to IEC 61850-5 Ed. 2 (performance class T5) and IEC 61869-9 the accuracy defined for synchronization in the Process Bus must be better than 1μs, which means approximately 0.02º of phase precision [3]. Therefore, among the synchronization protocols that use the Ethernet network, the PTP is the only one that has the precision to meet this Process Bus critical time requirement.

In a PTP network, the devices are named according to the function they perform and can be classified as: Grandmaster, Preferred Master, Slave-only clock, Ordinary clock and Transparent clock. So, a device can perform one or more functions. For example, a switch that implements PTP can be Ordinary Clock or Transparent Clock, or also can be Grandmaster or Slave-only clock.

PTP is a very comprehensive protocol and there are several implementation profiles, called PTP Profiles. For the power system, the profile used is the Power Profile, which has specific implementation requirements related to the network layer, master and slave message types, and the time intervals of the PTP messages.

3. Standards Review

The IEC 61869-9 is strongly based on the UCA Implementation Guide (Light Edition - IEC 61850-9-2LE), incorporating several of its definitions, in order to maintain compatibility. One of the differences between the two standards is the sampling rates determined: for IEC 61869-9 these rates are independent of the frequency of the power system, being preferred for Protection at 4800 Hz rate with 2 ASDUs, resulting in an output 2400 packets per second in network [4]; for 9-2LE, the rates were fixed with the frequency of the power system, being for Protection 4800 Hz with 1 ASDU at 60Hz system, resulting in an output of 4800 packets per second in the network [5].

IEC 61869-9 standardizes the use of the Ethernet network for the synchronization task through IEEE 1588, specified by IEC / IEEE 61850-9-3. All SV ports of the MUs must be capable of receiving PTP messages according to this standard. 1PPS stands as an alternative option for compatibility issues with previous technologies.

The standard IEC 61869-9 also describes the concept of "Holdover Mode" defining that, in case of loss of synchronization signal for a short period of time, the Merging Unit should continue to send the SV message normally without interruption. The minimum Holdover Mode time is 5 seconds. This is an artifice in case there is a temporary problem in the sync signal, taking into account that the MU has an internal precision clock, maintaining a irrelevant drift until the sync signal is recovered. The SmpSynch (Sample Synchronism) flag should remain unchanged.

When the MU loses synchronization definitively, the standard defines it as "Free-running Mode" and treats that the SVs should continue to be sent at a sampling rate with a deviation no more than ± 100 x 10 -6 . Even without external timing, the sample counter should continue to be incremented and reset when it reaches the limit, as if it were synchronizing. In this case, the SmpSynch flag must be zero indicating no synchronization.

4. Tests 

To evaluate the PTP performance in the Process Bus, a test scheme consisting of Conprove CE-GPS, Ruggedcom Switch RSG2288, two CE-6710 test sets and a Keysight 53220A frequency counter (time reference) was set up, as well a Tektronix MSO 2012 oscilloscope. The CE-6710 test set has full compliance with IEC 61850 (GOOSE, Sampled Values and Time Sync) and is capable of simulating up to 10 MUs simultaneously by publishing SV packets at various sampling rates and ASDU numbers in accordance with IEC 61850-9-2LE and IEC 61869-9. In addition, it allows time synchronization by IEEE 1588 PTP, IRIG-B and 1PPS. For the PTP, the timing accuracy of the synchronization between Master and Slave is better than 500 ns.

Below, Figure 2 shows the test system set up.

4.1. Test Scenarios

In order to answer the question if the Process Bus can affect the PTP, different scenarios have been set up. 

Six scenarios were set up to check PTP performance on the process bus. In two scenarios the Switch was Master and a test set was Slave and in the other two scenarios the test set was Master and the Switch was Slave. In order to carry out a comparative study of the influence of the presence of the SVs on the PTP network, in one of the two test scenarios a preliminary master and slave synchronization test was performed without loading the network with MUs. The GPS was used to calibrate the Master Clock and a frequency counter was used as a reference to carry out measurements of the master and slave clocks. A second test set was responsible for loading the network, simulating 10 MUs at the rate of 4800 Hz with 1 ASDU according to the test scenario. This publish rate was chosen due to the high load of 4800 packets per second that will traffic on the Ethernet network.

To check the master clock calibration with the GPS clock an oscillography was performed and the accuracy of the calibration was better than 100ns for all scenarios.

4.1.1. Scenario 1: Switch as Master without Process Bus Traffic

In this first test scenario, the Switch was Master and the CE-6710 test set was Slave. Ethernet network was running without load. The synchronization behavior was monitored by the time reference through the 1PPS output of Master and Slave. Figure 3 below illustrates the scheme of the first test.

In order to verify the performance of the PTP in a network without load, the time reference device acquired approximately 1800 measurement points during 30 minutes of test. The clock accuracy between master and slave was better than 500ns, with low standard deviation, throughout the test time as statistical analyzes and graphic shown in Figure 4 below.

4.1.2. Scenario 2: Switch as Master with Process Bus Traffic

In this second scenario, the test scheme remained the same as in Scenario 1, but the PTP network was loaded with 10 MUs traffic. The synchronization behavior was also monitored by the time reference device acquiring the same number of measurement points and time of Scenario 1.

Testing with this load, the clock accuracy measured between master and slave fluctuate more than in Scenario 1, with some measurement points exceeding 1μs of required accuracy defined in Table 1. Even in this case, the average differential time was better than 500ns, but the high standard deviation caused by network loading evidences a jitter behavior between slave and master clocks, as shown in Figure 5 below.

4.1.3. Scenario 3: CE-6710 as Master without Process Bus Traffic

In third test scenario, the CE-6710 was Master and the Switch was Slave. Ethernet network was running without load. The synchronization behavior was monitored by the time reference device through the 1PPS output of Master and Slave. Figure 6 below illustrates the scheme of the third test.

In order to verify the performance of the PTP in a network without load, the time reference device acquired approximately 1800 measurement points during 30 minutes of test. The clock accuracy between master and slave was better than 1μs, with low standard deviation, throughout the test time as statistical analyzes and graphic shown in Figure 7 below. 

4.1.4. Scenario 4: CE-6710 as Master with Process Bus Traffic

In this fourth scenario, the test scheme remained the same as in Scenario 3, but the PTP network was submitted to loading 10 MUs. The synchronization behavior was monitored by the time reference device acquiring the same number of measurement points and time of Scenario 3.

Testing with this load the clock accuracy measured between master and slave fluctuate more than in Scenario 3, with some measurement points exceeding 1μs of required accuracy defined in Table 1. Even in this case, the average differential time was better than 1μs, but the high standard deviation caused by network loading evidences a jitter behavior between slave and master clocks, as shown in Figure 8 below.

4.1.5. Scenarios 5 and 6: VLAN Scenarios

In Scenarios 2 and 4, sometimes, there was a time difference from the Slave relative to the Master above the 1μs allowed by Table 1. Therefore, a final test was performed using Virtual Lans (VLANs) to isolate the Master from the 10 MUs load. To perform the VLAN test, the same test schemes of Scenarios 2 and 4 were used, however the MUs were separated by VLAN to isolate the Master from the SV frames. Scenario 2 with VLAN becomes Scenario 5 and Scenario 4 wtih VLAN becomes Scenario 6.

Similarly, the time reference device was used to carry out the time measurements between the master clock and the slave clock in a time interval of 30 minutes, acquiring approximately 1800 measurement points for the statistical analyzes and graphics. As can be seen in Figures 9 and 10 below, the tests results using VLANs of Scenarios 5 and 6, respectively, shows low standard deviation, characterizing no lost of PTP synchronization.

5. Results Analysis

As verified in the results of the test scenarios, with the PTP network free of SV traffic, in both scenarios 1 and 3, master and slave differential time was better than 1μs as proposed in IEC 61869-9.

However, when the PTP network was loaded with 10 MUs (Scenarios 2 and 4), this differential time exceedes 1 μs sometimes due to loss of PTP packets disrupting the synchronizing mechanism. When it happens, the precisely tuning achieved by slave clock referred to master clock guarantee low time drift until the PTP was reestablished. The statiscal analyzes shows the average time remais below 1μs in Scenarios 2 and 4 with high standard deviation.

Scenarios 5 and 6 repeated the tests done in Scenarios 2 and 4 but with VLANs to isolate the MU traffic. As shown in Figures 9 and 10, the differential time did not exceed 1μs throughout the test time, proving that VLANs is a good choice to avoid disruptings of PTP sync mechanism caused by process bus SV traffic.

Therefore, based on this study, it can be stated that the Process Bus traffic occasionally disrupts PTP synchronism due to packet losses, delaying the Slave clock in relation to the Master beyond the accuracy imposed by IEC 61869-9. The tests also show better results are achieved without MUs traffic or using VLANs instead of sharing the same network for SV and PTP. Then, is recommended to use some technique to isolate PTP (physically or virtually) from the process bus traffic [6].

6. Conclusion

This work has shown the influence of the Process Bus traffic on the PTP network, evaluating if the Process Bus load interferes with PTP synchronization mechanism. Several tests were carried out in different scenarios, loading the process bus through the simulation of up to 10 MUs connected in the network.

Through the results analyzes were possible to conclude that using the load imposes in this paper, the process bus traffic interferes with PTP sync mechanism, causing occasionally differential time between Master and Slave bigger than 1μs, although average time remains smaller than 1μs.

The results of this paper are expected to contribute with the knowledge for the application of PTP time synchronization with Process Bus.

Bibliography

[1] IEC 61850-9-2: Specific Communication Service Mapping (SCSM) – Sampled Values over ISO/IEC 8802-3.

[2] IEC 61869-9: Digital Interface for Instrument Transformers, Edition 1.0 – April 2016.

[3] White Paper – Profile for Use of Precision Time Protocol in Power Systems – Microsemi.

[4] Pereira Junior, P.S.; Bernardino, R.C.; Martins, C.M.; Pereira, P.S.; Lourenço, G.E.; Salge, G.S. – “Experiência de Interoperabilidade no Barramento de Processo (Process Bus) da IEC 61850/IEC 61869-9 e as Questões Relativas à Sincronização” – Seminário Nacional de Produção e Transmissão de Energia Elétrica, XXIV SNPTEE de 2017.

[5] Implementation Guideline for Digital Interface to Instrument Transformers Using IEC 61850-9-2 – UCA International Users Group – 01/03/2004.

[6] Ingram, D.M.E.; Shaub, P.; Campbell, D.A.; Taylor, R.R. – “Performance Analysis of PTP Components for IEC 61850 Process Bus Applications” – IEEE Transactions on Instrumentation and Measurement, vol. 62, no. 4 – April 2013.