Putting 50-ms In Perspective
2 likes5,294 views
The document discusses the historical context and implications of the 50 ms convergence time in network services, originating from early digital transmission systems and APS subsystems. It highlights the disconnect between this benchmark and actual application needs, explaining that many service providers struggle to meet this requirement due to the complexity of modern networks and various application demands. Ultimately, the article stresses the need for a clearer understanding of what 50 ms means in different contexts and which applications truly require such rapid recovery times.
![Putting 50-ms In Perspective
by Lionel Florit
Abstract not going to be able to increase this time
easily based on a services argument. As a
ms CGA threshold reinforced the need for
50 ms APS switches at the DS transmis-
This article explores the historical reasons
consequence, 50ms is going to remain the sion level, to allow for worst-case reframe
behind the 50-ms requirement for con-
benchmark for some SP service types re- times all the way down the DS, DS2, DS1
vergence. as well as the problems service
gardless of the application’s requirements. hierarchy with suitable margin against the
providers have in achieving it. This article
20-ms CGA deadline. However, it was
also concludes that 50 ms, in itself, is not
relevant to most of the applications running
Where does 50 ms long since realized that a 20 ms CGA time
over Metro Ethernet Networks. Further- come from? was far too short. Many minor line interrup-
tions would trigger an associated switching
more, as networks become more intel- “The 50 ms figure historically originated
machine into mass call-dropping because
ligent, other mechanisms can mitigate the from the specifications of APS (Automated
of spurious CGA activations. As a result,
effect of network outages. Protection Switching) subsystems in early
the persistence time before call dropping
digital transmission systems and was not
Introduction actually based on any particular service
was raised to 2.5 +/- 0.5 s by ITU recom-
mendations in the 180s. Nevertheless,
Subscribers demand reliable services – as requirement. (Section extracted from
the requirement for 50-ms APS switching
they perceive reliability. Historically, service the book: Mesh based survivable net-
stayed in place, mainly because this was
providers (SP) have built their networks works Author: Wayne Grover ) Early digital
still technically quite feasible at no extra
with as much redundancy as they can af- transmission systems embodied 1:N APS,
cost in the design of APS subsystems.
ford in order to be as close as possible to which required typically about 20 ms for
a so-called 50-ms convergence time. On fault detection, 10 ms for signaling, and 10 The apparent sanctity of 50 ms was further
the surface, everything looks consistent. ms for the operation of the tail-end transfer entrenched in the 10s by vendors who
However, as we examine the problem more relay; consequently, the specification for promoted only ring-based transport solu-
closely, we will see that subscribers don’t APS switching times was reasonably set tions and found it advantageous to insist
really need the networks to converge in at 50 ms, allowing a 10 ms margin. Early on 50 ms as the requirement, effectively
50 ms. The 50 msec figure comes from generations of DS1 channel banks (from precluding distributed mesh restoration
historical requirements of a voice compo- the 170s) also had a Carrier Group Alarm alternatives that had been under equal
nent no longer in the network. What is more (CGA) threshold of about 20 ms. consideration at the start of the SONET era.
appropriate is to look at the convergence As a marketing strategy the 50 ms issue
The CGA is a time threshold for persis-
required by the application running on the thus served as the “mesh killer” for the
tence of any alarm state (such as loss of
network and provide the required service 10s [..].
signal or frame synch loss) on the trans-
to them. In most cases, it is extremely dif-
mission line side, after which all trunk On the other hand, there was also real
ficult for service providers to offer that level
channels would be busied out. The 20 urgency in the early 10s to deploy some
of availability and equipment vendors are
quick to confuse end-to-end convergence
– which is what is really needed – with
simple and bounded failure scenarios.
There is a great misunderstanding about
what people mean when they talk about
“50 ms”. In order to lift the fog, we look
at where this figure comes from, what it
means, where it applies, and which applica-
tions really need it.
For example, in private line services, where
there is currently an SLA between the
SP and the customer for 50ms, the SP is
Figure 7 - Generic Metro Ethernet Network
IP NGN ARCHITECTURE THOUGHT LEADERSHIP JOURNAL - Q1 FY2010](https://image.slidesharecdn.com/q1fy201050ms-091203122348-phpapp01/85/Putting-50-ms-In-Perspective-1-320.jpg)
Putting 50-ms In Perspective
- 1. Putting 50-ms In Perspective by Lionel Florit Abstract not going to be able to increase this time easily based on a services argument. As a ms CGA threshold reinforced the need for 50 ms APS switches at the DS transmis- This article explores the historical reasons consequence, 50ms is going to remain the sion level, to allow for worst-case reframe behind the 50-ms requirement for con- benchmark for some SP service types re- times all the way down the DS, DS2, DS1 vergence. as well as the problems service gardless of the application’s requirements. hierarchy with suitable margin against the providers have in achieving it. This article 20-ms CGA deadline. However, it was also concludes that 50 ms, in itself, is not relevant to most of the applications running Where does 50 ms long since realized that a 20 ms CGA time over Metro Ethernet Networks. Further- come from? was far too short. Many minor line interrup- tions would trigger an associated switching more, as networks become more intel- “The 50 ms figure historically originated machine into mass call-dropping because ligent, other mechanisms can mitigate the from the specifications of APS (Automated of spurious CGA activations. As a result, effect of network outages. Protection Switching) subsystems in early the persistence time before call dropping digital transmission systems and was not Introduction actually based on any particular service was raised to 2.5 +/- 0.5 s by ITU recom- mendations in the 180s. Nevertheless, Subscribers demand reliable services – as requirement. (Section extracted from the requirement for 50-ms APS switching they perceive reliability. Historically, service the book: Mesh based survivable net- stayed in place, mainly because this was providers (SP) have built their networks works Author: Wayne Grover ) Early digital still technically quite feasible at no extra with as much redundancy as they can af- transmission systems embodied 1:N APS, cost in the design of APS subsystems. ford in order to be as close as possible to which required typically about 20 ms for a so-called 50-ms convergence time. On fault detection, 10 ms for signaling, and 10 The apparent sanctity of 50 ms was further the surface, everything looks consistent. ms for the operation of the tail-end transfer entrenched in the 10s by vendors who However, as we examine the problem more relay; consequently, the specification for promoted only ring-based transport solu- closely, we will see that subscribers don’t APS switching times was reasonably set tions and found it advantageous to insist really need the networks to converge in at 50 ms, allowing a 10 ms margin. Early on 50 ms as the requirement, effectively 50 ms. The 50 msec figure comes from generations of DS1 channel banks (from precluding distributed mesh restoration historical requirements of a voice compo- the 170s) also had a Carrier Group Alarm alternatives that had been under equal nent no longer in the network. What is more (CGA) threshold of about 20 ms. consideration at the start of the SONET era. appropriate is to look at the convergence As a marketing strategy the 50 ms issue The CGA is a time threshold for persis- required by the application running on the thus served as the “mesh killer” for the tence of any alarm state (such as loss of network and provide the required service 10s [..]. signal or frame synch loss) on the trans- to them. In most cases, it is extremely dif- mission line side, after which all trunk On the other hand, there was also real ficult for service providers to offer that level channels would be busied out. The 20 urgency in the early 10s to deploy some of availability and equipment vendors are quick to confuse end-to-end convergence – which is what is really needed – with simple and bounded failure scenarios. There is a great misunderstanding about what people mean when they talk about “50 ms”. In order to lift the fog, we look at where this figure comes from, what it means, where it applies, and which applica- tions really need it. For example, in private line services, where there is currently an SLA between the SP and the customer for 50ms, the SP is Figure 7 - Generic Metro Ethernet Network IP NGN ARCHITECTURE THOUGHT LEADERSHIP JOURNAL - Q1 FY2010
- 2. kind of fast automated restoration method. multipoint? Which type of failure does the is located at the HQ (right hand side of This lead to the quick adoption of ring- 50 ms figure covers, fiber cut, port down, the diagram). If UNI B fails, the customer based solutions which had only incremen- box down, POP down, CPE down? After equipment (CE) must make the decision to tal development requirements over 1+1 a failure, are we allowed to drop less impor- switch to UNI D. In the best case scenario APS transmission systems. However, once tant traffic in order to provide bandwidth for (as far as recovery speed is concerned), rings were deployed, the effect was to only more important traffic? the CE is a single piece of equipment, to reinforce further the cultural assump- either a router or switch. If it is a switch, These questions are very difficult to tion of 50 ms as the standard. Thus, as the CE is probably running Rapid Span- answer. There is no common view on all sometimes happens in engineering, what ning Tree Protocol (RSTP) to block one of of them and it is easy to claim “50 ms” in a was initially a performance capability in the two UNIs so a loop with the service context and be less than stellar in another. one specific context (APS switching time) provider is not created. Because the EVC Let’s tackle the questions above using a evolved into a perceived requirement in all is multipoint, RSTP has detected several generic example. Consider the network other contexts. “ neighbors and is now running with 0-s diagram in Figure 7. timers. If the CE is a router, the backup path What do we mean by 50 This diagram represents a company’s will be used after the routing protocol con- ms recovery? headquarters, HQ (right hand side), con- nected to two branches offices over a verges, which is likely longer than 50 ms. The CE can’t take advantage of the 50 ms. The expression “50 ms recovery” is overly multipoint EVC with UNIs A, B, C, and D used. What does it mean that “the network In the access and aggregation networks, in that EVC. Can this network offer 50 ms must converge in 50 ms”? Does it mean recovery is more complex. A frame protection? Let’s take a closer look. that a failure must be detected in less than coming from the branch offices going 50 ms and the recovery will take place The first thing we notice are the three loca- through Agg4 must now be directed to the later? If a port fails, must a backup link be tions. This is a multipoint situation, there- other edge device (UNI D). This means all brought up within 50 ms? Must end-to-end fore, one could use H-VPLS or Hierarchical headquarters’ MAC addresses mapped to service must be restored in 50 ms? Is a VPLS (with L2 or MPLS in the access part UNI B must first be forgotten before the for- service restored when the first frame of of the network) or L2 spanning tree end to warding tables can begin to be rebuilt. De- that application makes it through the back- end (unlikely). pending on the size of the access network, up path of the network or when the ap- this could easily take more than 50 ms. Failure at the UNI plication resumes its work? Does it apply In our example, the only UNI protected Failure in the access layer to all type of services, point-to-point and CISCO PUBLIC
- 3. perspective, convergence is more compli- cated than a simple link failover. Let’s now turn our attention to the application layer. Which application needs 50 ms protection? As mentioned previously, looking at protec- tion from the application’s point of view should be the ultimate goal. After all, net- works ultimately carry application’s traffic. If we thought the failure scenarios described in the Sect 0 could be complicated, the impact of a network failure on a user ap- plication is even more complex to evalu- ate. How the application behaves when packets are lost and how this translates into Figure 8 - A classification of applications user experience depends on many param- eters: protocol behavior, computer speed The failure of a link in the access layer of nodes would have to send their traffic to at each end, type of application (voice vs. the network is probably what most people Agg. This could be challenging, because data), type of user (stock trader vs. instant have in mind when they talk about 50 ms potentially a lot of addresses have to be messenger), and so on. Furthermore, it is recovery. This event is not likely to involve mapped to different ports on Agg2 and nearly impossible to measure down to the the CE and the failure can be dealt with Agg6. The EVC in our example is likely to millisecond the effect of a network outage locally, between the aggregation node and share this network with many other EVCs on an application. the edge node within the service provid- which will have to be moved away from Let’s adopt a commonly accepted sim- er’s domain. This is the classic “backhoe” Agg4. This is a typical example of why the plification to this problem. Let’s assume incident that causes a fiber cut. Equipment 50 ms context matters. It is one thing to ap- the application is considered to be fully vendors claiming 50 ms recovery as a fea- ply this ideal delay to a single segment, recovered as soon as the first packet of ture of their equipment are likely to protect quite another to apply it to an entire network. that application, after a failure, is transport- against this type of scenario. If the access ed across the service provider’s network. Some questions are left unanswered. In is a ring, technologies such as SONET Assume, furthermore that, after the failure, Ethernet technologies, flooding is a com- (Note that standard SONET performance it doesn’t matter whether the packets are mon mechanism for recovering from a figures are given in the context of a single out of order, or even if some of them are failure. After detecting a failure, network ring with 16 nodes, and with adequate un- missing . Finally, assume that there is only a elements flood unicasts until they learn used bandwidth reserved for protection.) single failure event. the location of the source unicast and stop or Cisco REP ( will switch traffic to the other flooding at that point. Because flooding With this in mind, let’s try to make sense of side of the ring very quickly. If the access causes frames to be multiplied, conges- the application space and find which appli- network is hub and spoke, then MPLS FRR tion may occur in other parts of the network cation really needs 50 ms. Figure 8 shows (MPLS FRR: MPLS fast reroute (MPLS-FRR) that were not affected by the failure in the a possible way to classify some applica- mechanisms deviate the traffic in case first place. However, traffic is indeed flow- tions. The leaves of this tree show applica- of network failures) will also switch traffic ing end to end. When do we decide the tions commonly thought to be candidates quickly. As long as the data path doesn’t network has converged? Can we stop the perceived to require a maximum 50-ms have to use a different aggregation node, clock that measures when we hit the “50 outage. The list is not exhaustive. recovery will be prompt. ms mark,” even if new congestion is intro- Failure at the aggregation Layer duced in other parts of the network? The application space can be divided into A failure of the aggregation node itself is three categories: data, voice and video. As we see, we are still far away from the The diagram shows “mission-critical” as a more involved. In our example, if Agg4 point-to-point context described in the first classification. A mission-critical application were to fail altogether, all other aggregation section. Observed from a network-wide IP NGN ARCHITECTURE THOUGHT LEADERSHIP JOURNAL - Q1 FY2010
- 4. is viewed by the subscribers as important tions. Clients using lower-quality clocks window of optimum trading opportunity enough that they would be willing to pay must poll more frequently than well-syn- is increasingly measured in milliseconds. more to the service provider in order to chronized clients. If a packet is lost, no This is the world of zero packet loss (and guarantee fast recovery. retransmission takes place, the stations big dollar loss). 50 ms outage is not accept- simply wait for the next update. Therefore, able. Uptime is essential. If a single packet Data oriented applications with NTP a network outage could last as is lost, a transaction may not take place, In the data space, unicast and multicast ap- long as several seconds, with the loss of money will be lost. plications are the two main subcategories. only a single timing packet, with little effect We will not talk about applications run- Most market feeds are point-to-point T1s. on accuracy. ning over TCP. These applications expect Metro Ethernet providers may be used packet loss and are adapted to retransmis- IEEE1588 offers much better accuracy as backup to T1 lines. Banks and trading sion mechanisms built into TCP. They will than NTP. Typically, 1588 will generate a companies will constantly monitor the not be affected by short network outages. few packets per second. During a network performance of their backup connections. outage, the client clock would drift for the However, these applications use other Unicast applications duration of the outage (depending on the networks than Metro Ethernet. The unicast applications of interest run precision and quality of the clock, the drift over a connectionless protocol. Therefore, Voice applications will vary). However, the short-term drift of a network outages have a greater impact be- Voice applications are very interactive. Stratum clock is less than .7 x 10-7 in 24 cause packets can’t be retransmitted. Let’s There are two broad categories: Voice over hours. This amounts to approximately 255 look at three categories: military, industrial IP and Circuit Emulation over Packet. Ethernet and network timing applications: Military applications As seen in RFC 167, the Navy’s High-Per- formance Network (HPN) working group has studied the requirements of mission- critical applications on Navy platforms. However, these applications are deployed on submarines, on aircrafts, on ships, and on bases – and the military owns the WAN. This is typically not a service provider play. Industrial Ethernet applications Very tight time synchronization among ma- Figure 9 - CESoP Loss of one frame chines is needed (below 1 ms, IEC 61850 part 5). This is not a WAN application, but rather a LAN application. It is out of the frame slips in 24 hours while the system Voice over IP scope of this document. is holding. An outage as long as 500 ms There are two classes of traffic: the voice – 10 times our archetypical 50-ms factor! signal itself (bearer traffic) and the out of Time synchronization – won’t introduce a significant drift. band call-signaling traffic. If an outage oc- protocols (NTP) Multicast and unicast data applications curs during a conversation, the end-user may lose contact for the duration of the The synchronization accuracy of a WAN There are a few data oriented applications outage. If the outage lasts less than a few using NTP is typically within the range of which have more stringent requirements. seconds, the call itself is not dropped. 10 to 100 ms; on a LAN, this is typically Two examples are: Real-Time Distributed If the outage occurs during a call setup a few milliseconds. A broadcast server Applications and trading applications. phase, it takes longer to set up the call, or sends out a packet about every 64 seconds. A non broadcast client/server Trading Applications: the user might simply have to dial again. VoIP deployments over networks designed requires 2 packets per transaction. When first started, the transactions occur about the race to the best price to converge in 800 ms or more are Automated order routing systems and the very common. once per minute, increasing gradually to dawn of algorithmic trading mean that the once per 17 minutes under normal condi- CISCO PUBLIC
- 5. Circuit Emulation over Packet interactive and unidirectional Let’s use a common MPEG2 stream for Of all the applications we have seen so far, this one is the most challenging. Circuit • Video on demand (VoD), which is non- the basis of the discussion. Video streams travel compressed over data networks. real-time, interactive (VCR-like controls), Emulation Service over Packet (CESoP) is A data packet contains a certain amount unidirectional defined by the IETF, Metro Ethernet Forum, of information describing a video frame. MPLS Forum, and ITU-T. The entire T1 • Near VoD, network personal video There are about 0 video frames (or im- (framing, signaling, and payload) is carried recorder (PVR) ages) per second. There are different kinds transparently across the Ethernet network. If a packet gets dropped (or excessively • Security applications – surveillance of video frames: I, B and P frames. I, P and B frames are assembled into a Group of delayed) in the network, its content in the • Internet streaming to PC desktop Pictures (GOP). A GOP is typically bound- egress data stream is replaced with a con- figurable idle pattern, as shown in Figure . • Broadcast contribution and TV produc- ed by I frames and 12-15 frames long but it can vary with frame rate, content com- tion networks plexity, and encoder implementation. An I When a lot of packets are lost, an alarm These applications have different re- picture is a reference picture containing all is sent towards the packet source and quirements and addressing all of them is the pixel information needed to represent the destination sees all 1. In the case of beyond the scope of this paper. However, accurately the picture. A P picture (also unframed service, all 1 is an alarm in itself. when people talk about video quality, called predicted picture) contains all the However in case of framed service the they often use the example of to the final motion vectors to describe the new posi- framing is preserved and only the payload touchdown of a Super Bowl game, the last tions of the macroblocks, along with the dif- is replaced by FFs. As a result, the destina- tion will not see an alarm at all. More than 50 ms (or even 200) will not cause calls to be dropped. The mobile backhaul applications add a little twist to this equation. Mobile opera- tors want an end-to-end delay of less than 10ms so phone calls are not dropped during tower site hand offs. This implies a de-jitter buffer size of less than 10 ms, which means a 10 ms network outage will cause an alarm. However, once the alarm is raised, nothing else should happen if the outage is less than 500 ms. From the user-experience perspective, there will be a glitch, more or less noticeable depending Figure 10 - Effect of a loss on the length of the outage. penalty kick of soccer’s World Cup final, ference data that must be added to those Note that running CES over IP allows more a brain surgeon using an HD video feed macroblocks. P pictures require approxi- flexibility in terms of packet loss and com- to stitch synapses on a patient located on mately half of the data of an I picture and mon implementations can accommodate a another continent, or the president having are based on the previous picture (I or P). 500ms outage. a video conference with his generals to The B pictures are based on past and order (or not) the launch of a nuclear strike. future I and P pictures and are not derived Video applications We can imagine the consequences of a from each other. Like P pictures, they Video applications over data networks can 500-ms glitch in these situations. “Did he contain vectors and difference data. They have many different forms: say launch or not?” We can’t say that these usually require about a quarter of the data situations will never happen, but one can’t • Video conferencing, which is real time, design a network based on these require- of an I picture. interactive, and bidirectional ments either! Nonetheless, let us closely Because video frames are linked to the • Broadcast content (TVoDSL) to living examine the effect of a network outage on a video stream. each other, the loss of consecutive packets translates into a bi-dimensional effect, in room TV which is near real-time, non- IP NGN ARCHITECTURE THOUGHT LEADERSHIP JOURNAL - Q1 FY2010
- 6. space and time, as shown in Figure 10 One of the most visible effects of a frame loss occurs when some data of an I frame is not received by the decoder. Typically, 20% of the packets of a video stream carry information used to construct an I frame. A network outage of less than 500 ms affects up to two consecutive GOPs. The user will see pixilation, but the video pro- gram resumes after the network recovers without the need for user intervention. It is important to note that the duration of the artifact on the video screen will be longer than the duration of the network outage itself. No matter how short the network out- age is, as long as frames are lost, there is a possibility to see an artifact on the screen. Several schemes exist to compensate for the network as the only requirement will not prevent artifacts due to a network failure. Error Repair The client waits for missing RTP sequence numbers. If packets have been dropped and remain uncorrected following FEC repair, the client requests retransmission of the packets from its designated VQE server . Before they are handed off to an MPEG demux, retransmitted packets are re-sequenced and de-jittered in the client’s network. A single RCTP message may request the retransmission of multiple contiguous or non-contiguous packets. Figure 11 - Retransmission of lost video packets Live-live protection packet loss: some of the most efficient are ed by the client, which uses received FEC In certain cases such as head-end redun- forward error correction, error repair and Protection Packets. Any missing RTP pack- dancy, a lossless delivery may be required. live-live protection. ets beyond FEC coverage are forwarded to In this situation, we need to achieve protec- an error repair function. Forward Error Correction (FEC) tion against a single network failure of any A FEC capable client receives FEC repair FEC is a good way to improve the quality length. A solution consists of sending two packets and searches for missing Real- of experience but introduces overhead, copies of the multicast video stream on two Time Transport Protocol (RTP) sequence latency, and subsequently cost. The more physically separated paths. The last core numbers encountered across a FEC-pro- we budget for overhead and latency, the edge router or a VQE element receives the tected block period of N-packets. FEC longer network outage we can absorb two copies but passes only one. Such a protection periods are determined at the without seeing an image artifact. FEC can design protects against a single failure of headend by the definition of the FEC block handle a network outage of 50 or 100 ms or any length and makes the 50 ms discus- size. Missing RTP packets within the FEC more. FEC budget and network design go sion irrelevant. block coverage are automatically correct- hand in hand. A 50 ms or 100 ms limit on CISCO PUBLIC
- 7. Summary of Video Applications As we saw at the introduction of this sec- Conclusion We have also encountered examples of applications that will be affected by a As the context in which 50-ms has been tion, there are numerous video applica- 50-ms outage. Trading applications can’t reflexively used has expanded, the defini- tions, each with its own requirements. accept any loss. For circuit emulation over tion of failure and recovery has become Some applications may require zero loss, Ethernet, a 10 ms loss could have an effect less meaningful. A common trap is to look others may accommodate some loss. A (alarms) but this doesn’t mean catastrophic at localized failure scenarios and claim to network outage of any length will cause consequences. Finally, packet loss will achieve “50 ms” for all cases. When we packets to be lost and a 50-ms or longer create video impairments. There are ef- look at failures across the entire network, network outage can create visible video fective mechanisms beyond fast network we see it is very difficult for a service pro- artifacts. The faster a network converges, convergence that can be used to comple- vider to design a network that will accom- the more satisfactory the user experience. ment a given network’s performance. Such modate 50-ms recovery for any possible However, aside from network convergence, mechanisms are FEC, VQE, live-live and failure. We have also established that the there are other mechanisms that improve time-shifted streams. It is up to the service concept of recovery itself, when viewed video delivery. These new tools should providers to balance the investment be- network wide, is not well defined. When be built into the video delivery solution tween fast network convergence and error exactly, after a failure, a network has recov- to improve overall the performance of the corrections based on the level of quality of ered is up to debate. network. Examples of such tools are FEC, experience they want to achieve. repair packets, live-live feeds, time-shifted We have reviewed a set of applications Service providers and vendors continue streams and so on. These mechanisms perceived to be very sensitive to packet to strive to provide solutions that recover can correct the degradation of quality loss. We have established that, in most from failures as quickly as possible. How- resulting from a network failure. Relying cases, these applications don’t mandate a ever, trying to achieve an artificial goal of solely on a 50-ms convergence require- hard 50-ms figure. Most of the time, they 50 ms is likely to affect the affordability, ment is not likely to lead to the most satis- can cope with much a longer outage. Ex- scalability and flexibility of the solution. factory solution. amples of such applications are Voice over Deciding when we reach the right balance, IP, time synchronization, real-time distrib- using sound technical and economic jus- uted systems. tification, will serve the interest of both the provider and the consumer. IP NGN ARCHITECTURE THOUGHT LEADERSHIP JOURNAL - Q1 FY2010
- 8. Americas Headquarters Asia Pacific Headquarters Europe Headquarters Cisco Systems, Inc. Cisco Systems (USA) Pte. Ltd. Cisco Systems International BV San Jose, CA Singapore Amsterdam, The Netherlands Cisco has more than 200 offices worldwide. Addresses, phone numbers, and fax numbers are listed on the Cisco Website at www.cisco.com/go/offices. CCDE, CCENT, CCSI, Cisco Eos, Cisco HealthPresence, the Cisco logo, Cisco Lumin, Cisco Nexus, Cisco Nurse Connect, Cisco Stackpower, Cisco StadiumVision, Cisco TelePresence, Cisco WebEx, DCE, and Welcome to the Human Network are trademarks; Changing the Way We Work, Live, Play, and Learn and Cisco Store are service marks; and Access Registrar, Aironet, AsyncOS, Bringing the Meeting To You, Catalyst, CCDA, CCDP, CCIE, CCIP, CCNA, CCNP, CCSP, CCVP, Cisco, the Cisco Certified Internetwork Expert logo, Cisco IOS, Cisco Press, Cisco Systems, Cisco Systems Capital, the Cisco Systems logo, Cisco Unity, Collaboration Without Limitation, EtherFast, EtherSwitch, Event Center, Fast Step, Follow Me Browsing, FormShare, GigaDrive, HomeLink, Internet Quotient, IOS, iPhone, iQuick Study, IronPort, the IronPort logo, LightStream, Linksys, MediaTone, MeetingPlace, MeetingPlace Chime Sound, MGX, Networkers, Networking Academy, Network Registrar, PCNow, PIX, PowerPanels, ProConnect, ScriptShare, SenderBase, SMARTnet, Spectrum Expert, StackWise, The Fastest Way to Increase Your Internet Quotient, TransPath, WebEx, and the WebEx logo are registered trademarks of Cisco Systems, Inc. and/or its affiliates in the United States and certain other countries. All other trademarks mentioned in this document or website are the property of their respective owners. The use of the word partner does not imply a partnership relationship between Cisco and any other company. (0903R) Americas Headquarters Asia Pacific Headquarters Europe Headquarters Cisco Systems, Inc. Cisco Systems (USA) Pte. Ltd. Cisco Systems International BV San Jose, CA Singapore Amsterdam, The Netherlands Cisco has more than 200 offices worldwide. Addresses, phone numbers, and fax numbers are listed on the Cisco Website at www.cisco.com/go/offices. CCDE, CCENT, CCSI, Cisco Eos, Cisco HealthPresence, the Cisco logo, Cisco Lumin, Cisco Nexus, Cisco Nurse Connect, Cisco Stackpower, Cisco StadiumVision, Cisco TelePresence, Cisco WebEx, DCE, and Welcome to the Human Network are trademarks; Changing the Way We Work, Live, Play, and Learn and Cisco Store are service marks; and Access Registrar, Aironet, AsyncOS, Bringing the Meeting To You, Catalyst, CCDA, CCDP, CCIE, CCIP, CCNA, CCNP, CCSP, CCVP, Cisco, the Cisco Certified Internetwork Expert logo, Cisco IOS, Cisco Press, Cisco Systems, Cisco Systems Capital, the Cisco Systems logo, Cisco Unity, Collaboration Without Limitation, EtherFast, EtherSwitch, Event Center, Fast Step, Follow Me Browsing, FormShare, GigaDrive, HomeLink, Internet Quotient, IOS, iPhone, iQuick Study, IronPort, the IronPort logo, LightStream, Linksys, MediaTone, MeetingPlace, MeetingPlace Chime Sound, MGX, Networkers, Networking Academy, Network Registrar, PCNow, PIX, PowerPanels, ProConnect, ScriptShare, SenderBase, SMARTnet, Spectrum Expert, StackWise, The Fastest Way to Increase Your Internet Quotient, TransPath, WebEx, and the WebEx logo are registered trademarks of Cisco Systems, Inc. and/or its affiliates in the United States and certain other countries. All other trademarks mentioned in this document or website are the property of their respective owners. The use of the word partner does not imply a partnership relationship between Cisco and any other company. (0903R)