We live in a world of virtualised resources and distributed systems. From evolving radio access networks, to algorithmic and high-frequency trading platforms, to smart national power grids and beyond, all these environments are designed for flawless performance and need to be timed with increasing precision. To ensure that performance is maximised, all appliances in a network need to be interconnected at the lowest latency and synchronised to sub-microsecond precision, ensuring tight coordination.
One of the most prominent and demanding applications requiring stringent time synchronisation is the radio access network (RAN). Mobile base stations must be aligned in terms of frequency, as well as absolute time to coordinate the transmission and reception of data from multiple base stations to mobile user devices. The more precisely radio base stations are timed, the more efficiently the radio spectrum available can be utilised and, consequently, the better the resulting user experience. The evolution to LTE-Advanced Pro, and then 5G with its ultra-high speeds and ultra-low latency, will ramp up the pressure on the required accuracy of timing information.
For financial trading, there is an increasing need for trustworthy information about who places a transaction order at what exact time, when this transaction has been processed and when it has been executed. The European Security Market Authority (ESMA) has developed transparency requirements that require accurate time-stamping for reported events. MiFID II (the revised Markets in Financial Instruments Directive) defines a time-stamping accuracy of 100 microseconds synchronised to Coordinated Universal Time (UTC) for algorithmic and high-frequency trading. Similar requirements have been put in place by the US Security and Exchange Commission (SEC). Authorities need to be able to correlate data about each reportable event in order to prevent trading irregularities and market abuse.
National power grids are complex and highly distributed networks that are crucial to consumers and industry alike. Smart grids need to deliver power as reliably and efficiently as possible even though both the input and output to the grid can fluctuate quickly. Precise synchronisation is important because it provides the basis for ensuring quality and reliability of energy delivery as well as optimising efficiency within the system. By precisely synchronising generators and phasor measurements within a power grid, destructive interferences and loss of service are avoided.
To meet the increasing demand for precise timing of applications, metro and regional networks – and with this, optical networks and ITU G.709 OTN protocols – are playing an increasingly important role in the distribution of precise network timing signals.
Distribution of precise timing
In a distributed network environment, there are two main approaches to achieve synchronisation of individual instances and applications to a common time source with sub-microsecond precision. The first uses global navigation satellite systems (GNSS), such as Global Positioning System (GPS), which can deliver a timing signal with an accuracy within 100 nanoseconds of UTC. GNSS receivers must be deployed wherever timing information is required, to recover the synchronisation information from the satellite system and make it available to the application being timed (Figure 1).
Even though GNSS-based timing distribution is widely used today – especially for synchronising mobile radio networks – it will struggle to fulfil the upcoming requirements of more complex applications. Installing GNSS antennas and receivers at every location in large, distributed-system environments is not practical. Acquiring roof rights, deploying dedicated cables in the riser of buildings and installing timing gear requires a lengthy operational process that needs to be planned thoroughly. It’s also cumbersome and costly and does not scale to hundreds and thousands of nodes. What’s more, GNSS-jamming devices have become more widely available, which can have a severe impact on timing capabilities and the availability of precise timing information.
There is an alternative approach: timing information can be distributed via the underlying data network. If well designed, the operational complexity of network-based timing distribution should be significantly lower, with an achievable timing precision in the range of several hundred nanoseconds. Network-based timing distribution can be used alone, or to complement GNSS-based timing. With precise synchronisation becoming an indispensable requirement, operators have started to back up GNSS-based systems with network-based timing to provide high availability in difficult situations such as GNSS outages or jamming.
Legacy SDH/SONET networks employ precise frequency synchronisation technology that is optimised for fixed-bandwidth services. Today, asynchronous packet-based traffic dominates networks, and consequently SDH/SONET is being replaced by OTN, an asynchronous optical transport technology. OTN has been designed to be transparent to the payload, thus enabling it to carry any service, whether synchronous or packet-based. However, this means OTN must rely on client signals to provide timing information.
The IEEE 1588-2008 Precision Time Protocol (PTP) provides a standard method to distribute timing information with sub-microsecond accuracy over packet networks. The principle of PTP is based on the exchange of time-stamped packets between a master and a slave clock. With PTP, the envisaged timing precision does not, however, come for free. When transporting timing information over networks, delay asymmetries play a central role in achieving the desired precision – especially the dynamic component of network delays. Maximum precision requires the underlying network to operate with as little delay fluctuation as possible.
Unfortunately, OTN is not well suited for the distribution of precise synchronisation information due to uncontrolled dynamic latencies. This may result in delay asymmetries of significant size when it comes to timing delivery with the required precision. Current architectures need to be augmented in order to achieve the targeted timing precision of a few hundred nanoseconds that is required for precise synchronisation of radio base stations and other applications.
This need for more precise synchronisation presents a challenge to network operators when it comes to building optical transport networks that can deliver high-speed, time-sensitive information. Whether providing wholesale transport services to a mobile network operator or deploying infrastructure serving a packet-switched architecture on top, optical transport networks need to ensure that data – especially the embedded flows of PTP frames carrying timing information – are forwarded with maximum timing integrity.
Controlling latency variation
New technologies that minimise the fluctuation of delay asymmetries can enable synchronisation information to be transported across optical networks with maximum timing integrity. Two techniques have been developed: an out-of-band optical timing channel, and time-sensitive OTN technology for in-band transportation of PTP timing information. Both approaches enable network operators to address a wide range of applications, providing a solid basis for the precise timing of applications today and in the future.
An out-of-band optical timing channel is a robust method that allows network operators to transport time-sensitive PTP flows originating from PTP grandmaster clocks over longer distances across the metro and core layer of the optical network infrastructure (Figure 2). This technique overcomes typical problems that predominantly result from asynchronous processing of the digital bitstream. The main sources of asymmetry are due to OTN multiplexing and grooming of signals in the electrical domain. Signal buffering creates delay, and as buffer sizes change, time accuracy can show significant degradation of the order of microseconds. An out-of-band optical timing channel is even robust against complete OTN node resets.
Using the optical timing channel, data packets belonging to one or more PTP flows are forwarded on a dedicated wavelength, which can be terminated at strategic nodes. Treating PTP flows in a dedicated manner and applying bi-directional single-fibre operation minimises potential timing errors. To further enhance PTP timing distribution precision, PTP boundary clocks can be deployed at strategic nodes to measure and compensate for static link asymmetries, and mitigate sources of remaining dynamic link asymmetries. This means the main asymmetry-generating functional blocks of a DWDM network are completely bypassed, leading to intrinsically low delay asymmetries.
The optical timing channel enables distribution of precise timing information between local exchanges, meeting the stringent synchronisation requirements. In combination with enhanced primary reference time clocks (ePRTC) deployed at a limited number of strategic sites in the core of the network, operators can now establish a highly precise and stable timing core, fully independent from any satellite system (Figure 3). The growing issue of GNSS vulnerability can therfore be addressed by generating time aligned with GNSS, while at the same time maintaining the autonomy of reference clock-generated phase, time and frequency signal outputs during GNSS outages.
Time-sensitive wholesale services
The second, alternative method for transporting PTP signals without impacting the integrity of the timing information uses buffer-controlled OTN mapping. Unlike conventional OTN mapping and aggregation implementations, this method uses a buffer control that monitors and averages out buffer fill levels. Dynamic node delays and consequently latency asymmetries are therefore minimised, enabling slave clocks to recover timing signals with the intended level of accuracy. Latency variations caused by resetting nodes are eliminated by predefined buffer fill levels at a cold start. This function is unique to ADVA’s TrueTime OTN Buffer Control technology (Figure 4).
Time-sensitive OTN transport enables optical network operators to offer Gigabit Ethernet and 10 Gigabit Ethernet wholesale services with minimum delay fluctuation to mobile operators, carrying both backhaul traffic and timing information for precise synchronisation of base stations (Figure 5). This type of service has become a popular alternative to packet-switched mobile backhaul. It provides better and more transparent decoupling of mobile and fixed network operator network domains and constitutes a cost-efficient wholesale service alternative in the context of ever increasing bandwidth demand. The latest LTE Advanced Pro technologies have already triggered an upgrade of mobile backhaul capacity from 1Gb/s to 10Gb/s.
Time-sensitive wholesale services can form the basis of an integrated enterprise service offering. Combining low-latency connectivity with Timing as a Service (TaaS) will become an interesting offering for an increasing number of industry verticals, where evolving network-distributed applications require stringent equipment time synchronisation to consume available resources as efficiently as possible, increase performance and meet regulatory requirements.
Proper delivery of time information with sub-microsecond accuracy is, however, no easy task and poses significant challenges for optical network infrastructures. New technologies have been developed that allow timing information for highly precise application synchronisation to be delivered efficiently and reliably over OTNs. Many applications including RANs, algorithmic and high-frequency trading platforms and smart national power grids can now be timed precisely over the network with little additional effort.
• Michael Ritter is vice president of technical marketing and analyst relations, ADVA Optical Networking, based in Munich, Germany
