PFRv3 – новое поколение технологии Performance Routing для интеллектуального управления трафиком
3 likes1,791 views
PFRv3 is Cisco's next generation performance routing technology designed for intelligent traffic management across hybrid WANs. It enables real-time monitoring and application-aware routing to optimize performance by dynamically adjusting paths based on network conditions. Key components include centralized management, policy enforcement, and support for significant scaling across multiple sites.
![© 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public
Exports to MC
Monitor2 Details
• Aggregate Bandwidth Per TC
• Bandwidth is measured egress direction.
• Using monitor #2
• Collect bandwidth per Traffic Class
Aggregate Bandwidth Per TC
31
PMI:
[Egress-‐Aggregate]
-‐
#2
Trigger NBAR: no/yes
Key Fields
• ipv4 destination prefix,
• ipv4 destination mask,
• pfr site destination prefix ipv4
• pfr site destination prefix mask ipv4
• [application id]
• ip dscp
• interface output
Non-Key Fields
• counter bytes long
• counter packet long
• timestamp absolute monitoring-interval start
SITE
2
Traffic Flow
Monitor-interval
30 sec default
BR BR
MC
PMI: Performance Monitor Instance](https://image.slidesharecdn.com/celcjune15enpfrv3-150702071312-lva1-app6892/85/PFRv3-Performance-Routing-31-320.jpg)
![© 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public
Monitor1 Details
• Site Prefix collected with Performance Monitor
• Using monitor #1
Site Prefix Learning
36
PMI:
[Egress-‐prefix-‐learn]
-‐
#2
Trigger NBAR: no
Key Fields
• ipv4 source prefix,
• ipv4 source mask,
• Routing vrf input
Non-Key Fields
• counter bytes long
• counter packet long
• timestamp absolute monitoring-interval start
Site10
10.1.10.0/24
1
Exports to MC
Traffic Flow
Monitor-interval
30 sec default
BR BR
MC](https://image.slidesharecdn.com/celcjune15enpfrv3-150702071312-lva1-app6892/85/PFRv3-Performance-Routing-36-320.jpg)
![© 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public
Monitor3 Details
• Performance is measured ingress direction:
– Actual Traffic
– Smart Probes if no traffic
• Using monitor #3
• Monitors performance metrics per DSCP and
per Site
• Export only when there is a performance issue
(performance threshold crossed)
Ingress Performance Monitor
42
PMI
[Ingress-‐per-‐DSCP]
-‐
#3
Key Fields
• pfr site source id ipv4
• pfr site destination id ipv4
• ip dscp
• Interface input
• policy performance-monitor classification hierarchy
Non-Key Fields
• transport packets lost rate
• transport bytes lost rate
• pfr one-way-delay
• network delay average
• transport rtp jitter inter arrival mean
• counter bytes long
• counter packets long
• timestamp absolute monitoring-interval start
SITE
3
Traffic Flow
Monitor-interval
30 sec default
BR1 R10](https://image.slidesharecdn.com/celcjune15enpfrv3-150702071312-lva1-app6892/85/PFRv3-Performance-Routing-42-320.jpg)
![© 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public
Smart Probing
• Without actual traffic
– BR sends 10 probes spaced 20ms apart in the first 500ms and another similar 10 probes in the next 500ms, thus
achieving 20pps for channels without traffic.
• With actual traffic
– Lower frequency when real traffic is observed over the channel
– Probes sent every 1/3 of [Monitor Interval], ie every 10 sec by default
• Measured by Unified Performance Monitoring just like other data traffic
Help for Measurement Over Channels
45
Traffic Flow
Site10
10.1.10.0/24 3
INET
MPLS
3
BR BR
MC MC
Site11
10.1.11.0/24](https://image.slidesharecdn.com/celcjune15enpfrv3-150702071312-lva1-app6892/85/PFRv3-Performance-Routing-45-320.jpg)
![© 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public
Unreachable
Channel State Examples
47
R84#sh domain IWAN border channels
Channel id: 1
Channel create time: 12:52:35 ago
Site id : 255.255.255.255
DSCP : default[0]
Service provider : MPLS
Number of Probes sent : 0
Number of Probes received : 0
Last Probe sent : 12:52:35 ago
Last Probe received : - ago
Channel state : Initiated and open
Channel next_hop : 0.0.0.0
RX Reachability : Initial State
TX Reachability : Reachable
Channel is sampling 0 flows
Supports Zero-SLA: Yes
Muted by Zero-SLA: No
Probe freq with traffic : 1 in 10000 ms
R84#sh domain IWAN border channels
Channel id: 3
Channel create time: 12:50:55 ago
Site id : 10.2.11.11
DSCP : default[0]
Service provider : MPLS
Number of Probes sent : 843019
Number of Probes received : 838980
Last Probe sent : 00:00:00 ago
Last Probe received : 00:00:00 ago
Channel state : Initiated and open
Channel next_hop : 10.0.100.11
RX Reachability : Reachable
TX Reachability : Reachable
Channel is sampling 0 flows
Supports Zero-SLA: Yes
Muted by Zero-SLA: No
Probe freq with traffic : 1 in 10000 ms](https://image.slidesharecdn.com/celcjune15enpfrv3-150702071312-lva1-app6892/85/PFRv3-Performance-Routing-47-320.jpg)
![© 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public
Deploying With VRF – Hub MC Policies
69
domain IWAN
vrf TEST1
master hub
load-balance
class VOICE sequence 10
match dscp ef policy voice
path-preference MPLS fallback INET
class VIDEO sequence 20
match dscp af41 policy voice
path-preference MPLS fallback INET
class CRITICAL sequence 30
match dscp af31 policy low-latency-data
[Cont’d]
vrf TEST2
master hub
load-balance
class VOICE sequence 10
match dscp ef policy voice
path-preference MPLS fallback INET
class CRITICAL sequence 30
match dscp af31 policy low-latency-data](https://image.slidesharecdn.com/celcjune15enpfrv3-150702071312-lva1-app6892/85/PFRv3-Performance-Routing-69-320.jpg)
![© 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public
Monitoring Operations
R83#sh domain one master traffic-classes summary
APP - APPLICATION, TC-ID - TRAFFIC-CLASS-ID, APP-ID - APPLICATION-ID
SP - SERVICE PROVIDER, PC = PRIMARY CHANNEL ID,
BC - BACKUP CHANNEL ID, BR - BORDER, EXIT - WAN INTERFACE
UC - UNCONTROLLED, PE - PICK-EXIT, CN - CONTROLLED, UK - UNKNOWN
Dst-Site-Pfx Dst-Site-Id APP DSCP TC-ID APP-ID State SP PC/BC BR/EXIT
10.1.13.0/24 10.2.13.13 N/A ef 11 N/A CN MPLS 25/27 10.8.4.4/Tunnel100
10.1.12.0/24 10.2.12.12 N/A ef 9 N/A CN MPLS 20/19 10.8.4.4/Tunnel100
10.1.11.0/24 10.2.11.11 N/A af31 6 N/A CN MPLS 24/22 10.8.4.4/Tunnel100
10.1.13.0/24 10.2.13.13 N/A af31 8 N/A CN MPLS 26/28 10.8.4.4/Tunnel100
10.1.12.0/24 10.2.12.12 N/A af31 4 N/A CN MPLS 11/12 10.8.4.4/Tunnel100
10.1.11.0/24 10.2.11.11 N/A default 5 N/A CN INET 8/NA 10.8.5.5/Tunnel200
[SNIP]
Total Traffic Classes: 12 Site: 6 Internet: 0
R83#
Traffic Class – Site11 - Critical TC Id Controlled Path Information - Channels](https://image.slidesharecdn.com/celcjune15enpfrv3-150702071312-lva1-app6892/85/PFRv3-Performance-Routing-76-320.jpg)
![© 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public
Check Traffic Classes Details
R83#sh domain one master traffic-classes
Dst-Site-Prefix: 10.1.10.0/24 DSCP: ef [46] Traffic class id:25
TC Learned: 00:12:44 ago
Present State: CONTROLLED
Current Performance Status: in-policy
Current Service Provider: INET since 00:06:01
Previous Service Provider: INET for 181 sec
(A fallback provider. Primary provider will be re-evaluated 00:00:01 later)
BW Used: 24 Kbps
Present WAN interface: Tunnel200 in Border 10.8.5.5
Present Channel (primary): 84
Backup Channel: 85
Destination Site ID: 10.2.10.10
Class-Sequence in use: 10
Class Name: VOICE using policy User-defined
priority 2 packet-loss-rate threshold 5.0 percent
priority 1 one-way-delay threshold 150 msec
priority 2 byte-loss-rate threshold 5.0 percent
BW Updated: 00:00:14 ago
Reason for Route Change: Delay
Check Traffic Class
Voice for site 10
Check Channels
used (Primary and
Backup)
Policies and
reason for last
change
Active Path used](https://image.slidesharecdn.com/celcjune15enpfrv3-150702071312-lva1-app6892/85/PFRv3-Performance-Routing-77-320.jpg)
![© 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public
Check Channel After TCA
78
MC1#sh domain IWAN master channels | beg 107
Channel Id: 107 Dst Site-Id: 10.2.11.11 Link Name: MPLS DSCP: ef [46] TCs: 1
Channel Created: 00:15:03 ago
Provisional State: Initiated and open
Operational state: Available
Interface Id: 11
Estimated Channel Egress Bandwidth: 0 Kbps
Immitigable Events Summary:
Total Performance Count: 0, Total BW Count: 0
ODE Stats Bucket Number: 1
Last Updated : 00:05:45 ago
Packet Count : 2
Byte Count : 116
One Way Delay : 254 msec*
Loss Rate Pkts: 0.66 %
Loss Rate Byte: 0.0 %
Jitter Mean : 38000 usec
Unreachable : FALSE
[SNIP]
TCA Statistics:
Received:1 ; Processed:1 ; Unreach_rcvd:0
Latest TCA Bucket
Last Updated : 00:05:45 ago
One Way Delay : 266 msec
Loss Rate Pkts: NA
Loss Rate Byte: NA
Jitter Mean : NA
Unreachability: FALSE
On Demand Export
(ODE)
Threshold Crossing
Alert (TCA)](https://image.slidesharecdn.com/celcjune15enpfrv3-150702071312-lva1-app6892/85/PFRv3-Performance-Routing-78-320.jpg)
PFRv3 – новое поколение технологии Performance Routing для интеллектуального управления трафиком
- 1. PfRv3 – новое поколение технологии Performance Routing для интеллектуального управления трафиком. Денис Коденцев Инженер-консультант, CCIE dkodents@cisco.com
- 2. BRKRST-2362 Cisco Public© 2015 Cisco and/or its affiliates. All rights reserved. Agenda • WAN Trends • PfRv3 Principles • Monitoring Details – The Life of a Packet • Path Enforcement – Route Override • Enterprise Deployment • IWAN Management 2
- 4. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public Hybrid WAN: Leveraging the Internet Secure WAN Transport and Internet Access Branch MPLS (IP-VPN) Internet Private Cloud Virtual Private Cloud Public Cloud Secure WAN transport for private and virtual private cloud access Leverage local Internet path for public cloud and Internet access Increased WAN transport capacity, cost effectively! Improve application performance (right flows to right places) Secure WAN Transport Direct Internet Access
- 5. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public Cisco Intelligent WAN (IWAN) MPLS Branch 3G/4G-LTE AVC Internet Private Cloud Virtual Private Cloud Public CloudWAAS Akamai PfRv3 Transport Independence Intelligent Path Control Application Optimisation Secure Connectivity ! IPSec WAN Overlay ! Consistent Operational Model ! Optimal application routing ! Efficient use of bandwidth ! Performance monitoring ! Optimisation and Caching ! NG Strong Encryption ! Threat Defence DMVPN Performance Routing AVC, WAAS, Akamai Suite-B, CWS, ZBFW Management & Orchestration ISR-AX ASR1000- AX 5
- 6. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public Intelligent Path Control with PfR Leveraging the Internet Branch MPLS Internet Virtual Private Cloud Private Cloud • PfR monitors network performance and routes applications based on application performance policies • PfR load balances traffic based upon link utilisation levels to efficiently utilise all available WAN bandwidth Other traffic is load balanced to maximise bandwidth Voice/Video/Critical will be rerouted if the current path degrades below policy thresholds Voice/Video/Critical take the best delay, jitter, and/or loss path 6
- 7. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public PfRv3 – How it Works 7 Path Enforcement Define path optimisation policies on the Hub MC load balancing, path preference, application metrics DSCP Based Policies Application Based Policies Traffic flowing through the Border Routers (BRs) that match a policy are learned Traffic Classes Unified Performance Monitor Report the measured TC performance metrics to the Master Controller for policy compliance Unified Performance Monitor Master Controller directs BR path changes to keep traffic within policy Route Enforcement module in feature path MeasurementLearn the TrafficDefine your Traffic Policy ISR G2 ASR1K MC BR BR MC BR BR Performance Measurements MC BR BR Learning Active TCs Traffic Classes TC Path
- 8. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public Intelligent Path Control 8 PfR/OER • Internet Edge • Basic WAN • Provisioning per site per policy • 1000s of lines of config PfRv2 • Policy simplification • App Path Selection • Blackout ~6s • Brownout ~9s • Scale 500 sites • 10s of lines of config PfRv3 • Centralised provisioning • AVC Infrastructure • VRF Awareness • Blackout ~ 2s • Brownout ~ 2s • Scale 2000 sites • Hub config only Available Now IWAN 2.0 Today Performance Routing Evolution IWAN 2.0
- 9. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public Configuration – Improvement and Simplification 9 pfr master policy-rules IWAN-pfr-map max-range-utilization percent 30 mc-peer domain 1 eigrp Loopback0 target-discovery logging ! border 10.0.0.30 key-chain PFR-KEYCHAIN interface GigabitEthernet0/0.32 internal interface GigabitEthernet0/0.31 internal interface GigabitEthernet0/0.30 internal interface Tunnel2 external link-group internet interface Tunnel1 external link-group mpls ! learn traffic-class filter access-list DENY_GLOBAL_LEARN_LIST list seq 10 refname LEARN_VOICE_VIDEO traffic-class access-list VOICE_VIDEO filter BRANCH_PREFIX count 2000 max 10000 throughput list seq 20 refname LEARN_CRITICAL traffic-class access-list CRITICAL filter BRANCH_PREFIX count 2000 max 10000 throughput list seq 30 refname LEARN_BEST_EFFORT traffic-class prefix-list BEST_EFFORT_PREFIX count 2000 max 10000 throughput periodic 90 probe packets 20 ! ! pfr-map IWAN-pfr-map 10 match pfr learn list LEARN_VOICE_VIDEO set periodic 90 set delay threshold 200 set mode monitor fast set resolve delay priority 1 variance 5 set resolve loss priority 2 variance 5 set resolve jitter priority 3 variance 5 set loss threshold 50000 set jitter threshold 30 set probe frequency 8 set link-group mpls fallback internet ! pfr-map IWAN-pfr-map 20 match pfr learn list LEARN_CRITICAL set periodic 90 set delay threshold 100 set mode monitor fast set resolve delay priority 1 variance 20 set resolve loss priority 5 variance 10 set probe frequency 8 set link-group mpls fallback internet ! pfr-map IWAN-pfr-map 30 match pfr learn list LEARN_BEST_EFFORT set periodic 90 set mode monitor passive ! router eigrp IWAN ! service-family ipv4 autonomous-system 1 ! sf-interface default shutdown exit-sf-interface ! sf-interface Loopback0 no shutdown hello-interval 200 hold-time 600 exit-sf-interface ! topology base exit-sf-topology neighbor 10.0.0.91 Loopback0 remote 100 exit-service-family ! ! ip prefix-list BEST_EFFORT_PREFIX seq 10 permit 0.0.0.0/0 ! ip prefix-list BEST_EFFORT_PREFIX seq 5 deny 10.0.0.0/16 ip prefix-list BEST_EFFORT_PREFIX seq 10 permit 0.0.0.0/0 ! !! ip access-list extended CRITICAL permit ip any any dscp af31 ! ip access-list extended VOICE_VIDEO permit ip any any dscp ef permit ip any any dscp af41 permit ip any any dscp cs4 ! domain <MYNAME> vrf <name> master hub source-interface Loopback0 password IWAN load-balance class VOICE sequence 10 match dscp ef policy voice path-preference mpls fallback inet class VIDEO sequence 20 match dscp af41 policy real-time-video match dscp cs4 policy real-time-video path-preference mpls fallback inet class APPLICATION sequence 30 match dscp af31 policy low-latency-data PfRv2 PfRv3 MC Configuration on Hub site only • 500 sites • ~90 lines of configuration • All MCs • 2000 sites • <20 lines of configuration, all under “domain” • On the hub only
- 10. PfRv3 Principles
- 11. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public IWAN Domain – Concept 11 Branch R10 R11 R12 R13 BR1 BR2 BR3 BR4 MC1 IWAN POP1 IWAN POP2 MC2 DMVPN MPLS DMVPN INET DCI WAN Core DC1 DCn • A collection of sites sharing the same set of policies • Each site runs Performance Routing components • They exchange services through the Enterprise Domain Peering framework • Centralised configuration from a Domain Controller • Overlay network per Transport for flexibility and simplification IWAN Domain Controller Branch Branch TransitTransit
- 12. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public PfRv3 Components 12 Transit MPLS INET BRANCH Single CPE BRANCH Dual CPE BRMC BR MC/BR MC/BR BR The Decision Maker: Master Controller (MC) § Apply policy, verification, reporting § No packet forwarding/ inspection required § Standalone of combined with a BR § VRF Aware The Forwarding Path: Border Router (BR) § Gain network visibility in forwarding path (Learn, measure) § Enforce MC’s decision (path enforcement) § VRF aware
- 13. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public PfRv3 Sites 13 TRANSIT MPLS INET BRANCH Single CPE BRANCH Dual CPE BRMC BR MC/BR MC/BR BR • Controlled by a local Master Controller (MC) • Site ID – the IP address of the MC loopback • One/Multiple BRs • Each BR one/multiple links BRANCH SITE Site11 – Site ID = 10.2.11.11 BRANCH SITE Site10 – Site ID = 10.2.10.10 TRANSIT SITE Hub – Site ID = 10.8.3.3 Site Type § Transit Sites – Enterprise POPs or Hubs § Branch Sites - Stub
- 14. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public Transit Site – Hub Master Controller 14 TRANSIT MPLS INET BRANCH Single CPE BRANCH Dual CPE BRDC/MC BR MC/BR MC/BR BR • One of the MC is assigned the Domain Controller role • Central point of provisioning for Domain policies • DC + MC = Hub Master Controller domain IWAN vrf default master hub source-interface Loopback0
- 15. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public Branch Site 15 MPLS INET BRANCH Single CPE BRANCH Dual CPE BRDC/MC BR MC/BR MC/BR BR • Service Exchange • Policies and Monitor configurations • Site Prefixes domain IWAN vrf default master branch source-interface Loopback0 hub 10.8.3.3 border source-interface Loopback0 master local TRANSIT EIGRP-SAF EIGRP-SAF
- 16. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public PfRv3 – Role Summary • Hub Master Controller: The hub MC is the MC at the hub-site, which is either a data centre or a head quarter. This is the device where all policies are configured. It also acts as MC for that site and makes optimisation decision • Hub Border Router: This is a BR at the hub-site. This is the device where WAN interface terminates. PfRv3 is enabled on these interfaces. There can be one or more WAN interface on the same device. There can be one or more hub BRs. • Branch Master Controller: The Branch MC is the MC at the branch-site. There is no policy configuration on this device. It receives policy from the Hub MC. This device acts as MC for that site for making optimisation decision. The configuration includes the IP address of the hub MC. • Branch Border Router: This is a BR at the branch-site. There is no configuration other than enabling of PfRv3 BR functionality on the device. The WAN interface that terminates on the device is detected automatically. 16
- 17. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public Policy/Monitor Distribution 17 MPLS INET BRANCH Single CPE BRANCH Dual CPE BRDC/MC BR MC/BR MC/BR BR • Domain policies and monitor instances are configured on the Hub MC. • Policies are defined per VRF • Then distributed to branch sites using the peering infrastructure Policies Monitors Policies Monitors TRANSIT
- 18. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public Performance Policies - DSCP or App Based 18 domain IWAN vrf default master hub load-balance class MEDIA sequence 10 match application telepresence-media policy real-time-video match application ms-lync policy real-time-video path-preference MPLS fallback INET class VOICE sequence 20 match dscp ef policy voice path-preference MPLS fallback INET class CRITICAL sequence 30 match dscp af31 policy low-latency-data • Policies: – DSCP or Application Based Policies (NBAR2) – DSCP marking can be used with NBAR2 on the LAN interface (ingress on BR) • Default Class is load balanced
- 19. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public Built-in Policy Templates 19 Pre-defined Template Threshold Definition Voice priority 1 one-way-delay threshold 150 (msec) priority 2 packet-loss-rate threshold 1 (%) priority 2 byte-loss-rate threshold 1 (%) priority 3 jitter 30 (msec) Real-time-video priority 1 packet-loss-rate threshold 1 (%) priority 1 byte-loss-rate threshold 1 (%) priority 2 one-way-delay threshold 150 (msec) priority 3 jitter 20 (msec) Low-latency-data priority 1 one-way-delay threshold 100 (msec) priority 2 byte-loss-rate threshold 5 (%) priority 2 packet-loss-rate threshold 5 (%) Pre-defined Template Threshold Definition Bulk-data priority 1 one-way-delay threshold 300 (msec) priority 2 byte-loss-rate threshold 5 (%) priority 2 packet-loss-rate threshold 5 (%) Best-effort priority 1 one-way-delay threshold 500 (msec) priority 2 byte-loss-rate threshold 10 (%) priority 2 packet-loss-rate threshold 10 (%) scavenger priority 1 one-way-delay threshold 500 (msec) priority 2 byte-loss-rate threshold 50 (%) priority 2 packet-loss-rate threshold 50 (%)
- 20. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public Discovery – WAN Interface 20 • Using Smart Probes • Hub generates Discovery packets over all available paths • Discovery packets intercepted on spokes • External interfaces and path names automatically discovered and added BRANCH Single CPE BRANCH Dual CPEMPLS INET Define the Path Names on the Hub Border Routers Branch BRs discover Path Names and External Interfaces Path MPLS Path INET TRANSIT HUB BRMC BR MC/BR MC/BR BR Hub Master MC
- 21. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public Performance Monitoring – User Traffic MPLS INET Bandwidth on egress Per Traffic Class (dest-prefix, DSCP, AppName) User traffic BRMC BR MC/BR MC/BR BR Passive Monitoring Performance Monitor • Collect Performance Metrics • Per Channel - Per DSCP - Per Source and Destination Site - Per Interface SITE3 Single CPE SITE2 Dual CPE SITE1
- 22. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public Performance Monitoring MPLS INET User traffic Integrated Smart Probes • Traffic driven – intelligent on/off • Site to site and per DSCP Performance Monitor • Collect Performance Metrics • Per Channel - Per DSCP - Per Source and Destination Site - Per Interface BRMC BR MC/BR MC/BR BR Smart Probing SITE3 Single CPE SITE2 Dual CPE SITE1
- 23. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public Performance Violation MPLS INET Threshold Crossing Alert (TCA) • Sent to source site • loss, delay, jitter, unreachable BRMC BR MC/BR MC/BR BR SITE3 Single CPE SITE2 Dual CPE SITE1
- 24. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public Performance Violation – NetFlow Export MPLS INET BRMC BR MC/BR MC/BR BR SITE3 Single CPE SITE2 Dual CPE SITE1
- 25. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public Policy Decision MPLS INET User traffic User traffic BRMC BR MC/BR MC/BR BR • Reroute Traffic to a Secondary Path SITE3 Single CPE SITE2 Dual CPE SITE1
- 26. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public Key Points and … Limitations • Key Points – DSCP or Application (NBAR2) based policies – Control traffic in the global Routing Table and per VRF – Passive Monitoring based on user traffic – Leverage smart probing when needed – Cooperation between pair of sites – Exports of PfR information with NetFlow v9 to collectors • Limitation: – No IPv6 support yet (Roadmap) – But infrastructure is IPv6 ready – NBAR2 with Asymmetric Routing is not yet supported 26
- 27. Monitoring Details – The Life of a Packet
- 28. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public PfRv3 Monitoring • PfRv3 defines multiple Performance Monitor Instances (PMI) • Applied on all External interfaces Based on Performance Monitor 28 1 Exports to MC 3 Site 2 1 3 2 Monitor1 – Site Prefix Learning (egress direction) Monitor2 – Aggregate Bandwidth per Traffic Class (egress direction) Monitor3 – Performance measurements (ingress direction) BR BR MC
- 29. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public BR Dest Prefix Dest. Mask Dest Site App Name DSCP Output I/F … Pkts 10.1.10.0 /24 10.2.10.10 Skype AF41 Tunnel100 … 1100 Performance Monitor Cache #1 Performance Monitor 1 Traffic Non-Key Fields Packets Bytes Timestamps Performance Monitor Cache #2 Performance Monitor 2 Performance Monitor Multiple Monitors with Unique Key Fields Key Fields Packet 1 ipv4 destination prefix 10.1.10.0 ipv4 destination mask /24 Destination Site ID 10.2.10.10 Application Name Skype DSCP AF41 Interface Output Tunnel100 Key Fields Packet 1 Source Prefix 10.8.0.0 Source Mask /16 Input VRF VRF1 Non-Key Fields Packets Bytes Timestamps Source Prefix Source Mask Input VRF … Pkts 10.8.0.0 /16 VRF1 … 11000 29
- 30. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public INETMPLS 10.1.10.0/24 10.1.11.0/24 10.1.12.0/24 10.1.13.0/24 IWAN POP MC1 BR1 BR2 R10 R11 R12 R13 Source Site – Egress Traffic • Traffic going outside a source site – Initially based on Routing Information • Captured by a Performance Monitor on the external interface on egress • BR reports to its local MC • Monitor interval is 30 sec Collecting Traffic Class 30 Hub MC 10.8.3.3/32 Source Destination DSCP App 10.8.1.200 10.1.10.200 AF41 APP1 2 MC1 Dst-Site- Pfx App DSCP Dst-Site- Id State BW BR Exit 10.1.10.0 APP1 AF41 ? UK 24 BR1 Tu10
- 31. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public Exports to MC Monitor2 Details • Aggregate Bandwidth Per TC • Bandwidth is measured egress direction. • Using monitor #2 • Collect bandwidth per Traffic Class Aggregate Bandwidth Per TC 31 PMI: [Egress-‐Aggregate] -‐ #2 Trigger NBAR: no/yes Key Fields • ipv4 destination prefix, • ipv4 destination mask, • pfr site destination prefix ipv4 • pfr site destination prefix mask ipv4 • [application id] • ip dscp • interface output Non-Key Fields • counter bytes long • counter packet long • timestamp absolute monitoring-interval start SITE 2 Traffic Flow Monitor-interval 30 sec default BR BR MC PMI: Performance Monitor Instance
- 32. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public What is a Traffic Class? 32 Dst-Site-Pfx App DSCP Dst-Site-id State BR Exit 10.1.10.0 APP1 AF41 ? UK ? ? 10.1.10.0 N/A EF ? UK ? ? 10.1.10.0 N/A AF31 ? UK ? ? 10.1.10.0 N/A 0 ? UK ? ? 10.1.11.0 N/A EF ? UK ? ? 10.1.11.0 N/A AF31 ? UK ? ? 10.1.11.0 N/A 0 ? UK ? ? 10.1.12.0 N/A 0 ? UK ? ? … • What is a Traffic Class (TC)? – Destination Site Prefix – DSCP value – Application Name • Each Master Controller owns a TC Database INETMPLS 10.1.10.0/24 10.1.11.0/24 10.1.12.0/24 10.1.13.0/24 MC1 BR1 BR2 R10 R11 R12 R13 Hub MC 10.8.3.3/32 IWAN POP
- 33. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public We Need The Destination Site ID – Why?? 33 • Performance Measurement is done on the destination site • Source site needs to get Performance Metrics from the destination site SITE MPLS INET ? Dst-Site-Pfx App DSCP Dst-Site-id State BR Exit 10.1.10.0 APP1 AF41 ? UK ? ? MC1 BR BR
- 34. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public INETMPLS 10.1.10.0/24 10.1.11.0/24 10.1.12.0/24 10.1.13.0/24 MC1 BR1 BR2 R10 R11 R12 R13 Hub MC 10.8.3.3/32 Site Prefix Exchange – Dynamic Based on User Traffic 34 1 Source Destination DSCP App 10.1.10.200 10.8.1.200 AF41 XYZ R10 MC Site-Pfx Mask 10.1.10.0 /24 Site 10 10.1.10.0/24 Site 10 10.1.10.0/24 Site 10 10.1.10.0/24 • Source Prefix and Mask collected from Performance Monitor • Monitor interval is 30 sec • BR send to its local MC • MC send information to all peers via Peering Site 10 10.1.10.0/24 IWAN POP
- 35. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public Site Prefix Exchange – Dynamic Based on User Traffic 35 Site Prefix List Hub 10.8.0.0/16 R10 10.1.10.0/24 R11 10.1.11.0/24 R12 10.1.12.0/24 R12 10.1.13.0/24 • Every MC in the domain owns a Site Prefix database • Gives the mapping between site and prefixes INETMPLS 10.1.10.0/24 10.1.11.0/24 10.1.12.0/24 10.1.13.0/24 MC1 BR1 BR2 R10 R11 R12 R13 Hub MC 10.8.3.3/32 IWAN POP
- 36. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public Monitor1 Details • Site Prefix collected with Performance Monitor • Using monitor #1 Site Prefix Learning 36 PMI: [Egress-‐prefix-‐learn] -‐ #2 Trigger NBAR: no Key Fields • ipv4 source prefix, • ipv4 source mask, • Routing vrf input Non-Key Fields • counter bytes long • counter packet long • timestamp absolute monitoring-interval start Site10 10.1.10.0/24 1 Exports to MC Traffic Flow Monitor-interval 30 sec default BR BR MC
- 37. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public Site Prefixes – Static Configuration • This allows configuring site-prefix manually instead of learning. • This configuration should be used at the site if the site is used for transit. – For example, Site A reaches Site B via Hub-Site, where Hub-Site is transit site. The configuration is used to prevent learning of Site A prefix as Hub-Site prefix when it is transiting from Hub. 37 domain IWAN vrf default master hub source-interface Loopback0 site-prefixes prefix-list DC1_PREFIX ! ip prefix-list DC1_PREFIX seq 10 permit 10.8.0.0/16 ! MC1 BR1 BR2 Hub MC 10.8.3.3/32 Source Destination DSCP App 10.1.10.200 10.1.11.200 AF41 AppXY TRANSIT SITE
- 38. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public Source Site – Egress Traffic Traffic Class Controlled 38 2 MC1 Dst-Site-Pfx App DSCP Dst- Site-id State BW BR Exit 10.1.10.0 APP1 AF41 R10 CN 24 BR1 Tu100 MC1 Site Prefix List Hub 10.8.0.0/16 R10 10.1.10.0/24 R11 10.1.11.0/24 R12 10.1.12.0/24 R12 10.1.13.0/24 • Now the MC has the destination site information • Traffic Class is controlled INETMPLS 10.1.10.0/24 10.1.11.0/24 10.1.12.0/24 10.1.13.0/24 MC1 BR1 BR2 R10 R11 R12 R13 Hub MC 10.8.3.3/32 IWAN POP Source Destination DSCP App 10.8.1.200 10.1.10.200 AF41 APP1
- 39. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public Source Site – Traffic Class Database 39 Dst-Site-Pfx Dst-Site-id App DSCP State BR Exit 10.1.10.0 R10 APP1 AF41 CN BR1 Tu100 10.1.10.0 R10 N/A EF CN BR1 Tu100 10.1.10.0 R10 N/A AF31 CN BR1 Tu100 10.1.10.0 R10 N/A 0 CN BR2 Tu200 10.1.11.0 R11 N/A EF CN BR1 Tu100 10.1.11.0 R11 N/A AF31 CN BR1 Tu100 10.1.11.0 R11 N/A 0 CN BR2 Tu200 10.1.12.0 R12 N/A 0 CN BR2 Tu200 • Same process for all traffic to between all sites. • Traffic Class database contains bandwidth, destination sites, BR and external path used INETMPLS 10.1.10.0/24 10.1.11.0/24 10.1.12.0/24 10.1.13.0/24 MC1 BR1 BR2 R10 R11 R12 R13 Hub MC 10.8.3.3/32 IWAN POP
- 40. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public INETMPLS 10.1.10.0/24 10.1.11.0/24 10.1.12.0/24 10.1.13.0/24 MC1 BR1 BR2 R10 R11 R12 R13 Hub MC 10.8.3.3/32 Destination Site – Ingress Traffic • Traffic flow captured on the destination site • Performance Monitor collects Performance Metrics • Per Channel • Default Monitor interval is 30 sec Channel Performance 40 R10 MC Channel Src-Site-id Path DSCP BW Delay Jitter Loss 5 Hub Tu1 AF41 24 51 2 1 3 IWAN POP Source Destination DSCP App 10.8.1.200 10.1.10.200 AF41 APP1
- 41. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public What is a Channel? • A Channel is a unique combination of: – Interface – Peer-Site id – DSCP • Created – Based on real traffic observed on the BRs – Added every time there is a new DSCP or a new interface or a new site added to the prefix database. – Smart Probe is received • On all exits (Present Channel, Backup Channel) Between Sites 41
- 42. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public Monitor3 Details • Performance is measured ingress direction: – Actual Traffic – Smart Probes if no traffic • Using monitor #3 • Monitors performance metrics per DSCP and per Site • Export only when there is a performance issue (performance threshold crossed) Ingress Performance Monitor 42 PMI [Ingress-‐per-‐DSCP] -‐ #3 Key Fields • pfr site source id ipv4 • pfr site destination id ipv4 • ip dscp • Interface input • policy performance-monitor classification hierarchy Non-Key Fields • transport packets lost rate • transport bytes lost rate • pfr one-way-delay • network delay average • transport rtp jitter inter arrival mean • counter bytes long • counter packets long • timestamp absolute monitoring-interval start SITE 3 Traffic Flow Monitor-interval 30 sec default BR1 R10
- 43. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public Channel Details INETMPLS 10.1.10.0/24 10.1.11.0/24 10.1.12.0/24 10.1.13.0/24 MC1 BR1 BR2 R10 R11 R12 R13 Hub MC 10.8.3.3/32 Present Channel 3 • Site 10 • DSCP AF41 • MPLS Backup Channel 4 • Site 10 • DSCP AF41 • Internet IWAN POP Backup channels Auto created
- 44. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public Channel Details INETMPLS 10.1.10.0/24 10.1.11.0/24 10.1.12.0/24 10.1.13.0/24 MC1 BR1 BR2 R10 R11 R12 R13 Hub MC 10.8.3.3/32 Present Channel 13 • Site 11 • DSCP EF • MPLS Backup Channel 14 • Site 11 • DSCP EF • Internet Between Any Pair of Sites that has traffic! IWAN POP
- 45. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public Smart Probing • Without actual traffic – BR sends 10 probes spaced 20ms apart in the first 500ms and another similar 10 probes in the next 500ms, thus achieving 20pps for channels without traffic. • With actual traffic – Lower frequency when real traffic is observed over the channel – Probes sent every 1/3 of [Monitor Interval], ie every 10 sec by default • Measured by Unified Performance Monitoring just like other data traffic Help for Measurement Over Channels 45 Traffic Flow Site10 10.1.10.0/24 3 INET MPLS 3 BR BR MC MC Site11 10.1.11.0/24
- 46. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public Channel Unreachable • PfRv3 considers a channel reachable as long as the site receives a PACKET on that channel • A channel is declared unreachable in both direction if – There is NO traffic on the Channel, probes are our only way of detecting unreachability. So if no probe is received within 1 sec, we detect unreachability. – When there IS traffic on the channel, if we don’t see any packet for more than a second on a channel we detect unreachability. • A channel is put to reachable if following happens – Traffic is received from the remote side. – Unreachable TCA not received for two monitor intervals 46 BR BR 1 Exports to MC 3 Channel DSCP EF
- 47. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public Unreachable Channel State Examples 47 R84#sh domain IWAN border channels Channel id: 1 Channel create time: 12:52:35 ago Site id : 255.255.255.255 DSCP : default[0] Service provider : MPLS Number of Probes sent : 0 Number of Probes received : 0 Last Probe sent : 12:52:35 ago Last Probe received : - ago Channel state : Initiated and open Channel next_hop : 0.0.0.0 RX Reachability : Initial State TX Reachability : Reachable Channel is sampling 0 flows Supports Zero-SLA: Yes Muted by Zero-SLA: No Probe freq with traffic : 1 in 10000 ms R84#sh domain IWAN border channels Channel id: 3 Channel create time: 12:50:55 ago Site id : 10.2.11.11 DSCP : default[0] Service provider : MPLS Number of Probes sent : 843019 Number of Probes received : 838980 Last Probe sent : 00:00:00 ago Last Probe received : 00:00:00 ago Channel state : Initiated and open Channel next_hop : 10.0.100.11 RX Reachability : Reachable TX Reachability : Reachable Channel is sampling 0 flows Supports Zero-SLA: Yes Muted by Zero-SLA: No Probe freq with traffic : 1 in 10000 ms
- 48. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public Smart Probing – Zero SLA Support 48 • No SLA on the secondary path, but still probing all channels (DSCPs) – 10 DSCP => 10 Smart Probes over the secondary path • Waste of bandwidth, especially for metered interfaces (4G/LTE) Traffic Flow Site10 10.1.10.0/24 3 INET MPLS 3 BR BR MC MC Site11 10.1.11.0/24
- 49. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public Zero-SLA • Zero-sla added on the WAN interface path configuration option. • PfR will only probe the default channel (DSCP 0). – It will mute all other smart-probes besides the default channel. – The default channel runs as DSCP 0, TCA and ODE are DSCP CS6. – Extrapolate metrics on this to all other DSCPs on this channel • Site Capability Exchange: – Site-capability exchange: allow for backwards compatibility and coexistence with legacy sites until they are upgraded. – The site-capability is fully extensible so that additional domain configuration features can be added, not just PfRv3 capabilities. Principles
- 50. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public Zero SLA Configuration 50 R83#sh domain IWAN master site-capability Device Capability ----------------------------------------------------------- | Capability | Major | Minor | ----------------------------------------------------------- | Domain | 2 | 0 | ----------------------------------------------------------- | Zero-SLA | 1 | 0 | ----------------------------------------------------------- Site id :10.2.10.10 ----------------------------------------------------------- | Capability | Major | Minor | ----------------------------------------------------------- | Domain | 2 | 0 | ----------------------------------------------------------- | Zero-SLA | 1 | 0 | ----------------------------------------------------------- Site id :10.2.12.12 [SNIP R83# interface Tunnel200 description --- INET --- domain IWAN path INET zero-sla ! BR2 INET DMVPN-2 IWAN POP Tunnel 200Borders: IP address: 10.8.4.4 Version: 2 Connection status: CONNECTED (Last Updated 1d00h ago ) Interfaces configured: Name: Tunnel100 | type: external | Service Provider: MPLS | Status: UP | Zero-SLA: NO Number of default Channels: 0 Tunnel if: Tunnel0 IP address: 10.8.5.5 Version: 2 Connection status: CONNECTED (Last Updated 00:04:59 ago ) Interfaces configured: Name: Tunnel200 | type: external | Service Provider: INET | Status: UP | Zero-SLA: YES Number of default Channels: 0 Tunnel if: Tunnel0
- 51. Path Enforcement – Route Override
- 52. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public INETMPLS 10.1.10.0/24 10.1.11.0/24 10.1.12.0/24 10.1.13.0/24 Enterprise HQ MC1 BR1 BR2 R10 R11 R12 R13 Hub MC 10.8.3.3/32 Performance Violation • Performance notification exported ONLY when there is a violation on a specific channel – Generated from ingress monitor attached on BRs to the source site MC – Based on Monitor interval (30 sec default, configurable) – Via all available external interfaces. 52 3 R10 Channel Dst-Site-id DSCP Path BW Delay Jitter Loss 5 Hub AF41 Tu1 24 250 2 1 R10 TCA Delay DSCP AF41 Path MPLS
- 53. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public Performance Violation – Detected on Dst Site 53 R10 TCA Delay DSCP AF41 Path MPLS Dst-Site-Pfx Dst-Site-id App DSCP State BR Exit 10.1.10.0 R10 APP1 AF41 CN BR1 Tu1 10.1.10.0 R10 N/A AF41 CN BR1 Tu1 10.1.10.0 R10 N/A AF31 CN BR1 Tu1 10.1.10.0 R10 N/A 0 CN BR2 Tu2 10.1.11.0 R11 N/A EF CN BR1 Tu1 10.1.11.0 R11 N/A AF31 CN BR1 Tu1 10.1.11.0 R11 N/A 0 CN BR2 Tu2 10.1.12.0 R12 N/A 0 CN BR2 Tu2 INETMPLS 10.1.10.0/24 10.1.11.0/24 10.1.12.0/24 10.1.13.0/24 Enterprise HQ MC1 BR1 BR2 R10 R11 R12 R13 Hub MC 10.8.3.3/32 3
- 54. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public INETMPLS 10.1.10.0/24 10.1.11.0/24 10.1.12.0/24 10.1.13.0/24 Enterprise HQ MC1 BR1 BR2 R10 R11 R12 R13 Hub MC 10.8.3.3/32 3 Policy Decision – Reroute TC • MC computes a new path for each impacted TC • MC tells the BRs to enforce the new path 54
- 55. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public INETMPLS 10.1.10.0/24 10.1.11.0/24 10.1.12.0/24 10.1.13.0/24 Enterprise HQ MC1 BR1 BR2 R10 R11 R12 R13 Hub MC 10.8.3.3/32 3 Reroute TC – Path Enforcement • Dataplane forwarding • Activated on all but external interfaces • Lookup per packet - output-if/next hop retrieved – Packet Forwarded – If no entry – Uses FIB entry • TC flows redirected to the new path over the auto mGRE tunnels between the BRs • No change in the routing table 55
- 57. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public Intelligent WAN Deployment Models Consistent VPN Overlay Enables Security Across Transition Expensive Highest SLA guarantees Tightly coupled to SP Internet Branch Public MPLS MPLS Branch Public MPLS + Internet More BW for key applications Moderately priced Balanced SLA guarantees Enterprise Dual MPLS Hybrid Branch Internet Best price/performance Enterprise responsible for SLAs Most SP flexibility Dual Internet
- 58. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public IWAN Layers MPLS Routing Internet Routing Overlay Routing Protocol (BGP, EIGRP) Transport Independent Design (DMVPN) PfRAVC QoS Infrastructure Routing Transport Overlay Overlay routing over tunnels Intelligent Path Selection ZBFW CWS
- 59. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public IWAN Routing Protocols Which protocol should I use? • IWAN Profiles are based upon BGP and EIGRP for scalability and optimal Intelligent Path Control • Scalability: – BGP (Path Vector) and EIGRP (Advanced Distance Vector) provide best scale over large hub-and-spoke topologies like DMVPN – OSPF (Link State) maintains a lot of network state which cannot be subdivided easily in large DMVPN networks • Intelligent Path Control: – PfR can be used with any routing protocols by relying on the routing table (RIB). • Requires all valid WAN paths be ECMP so that each valid path is in the RIB. – For BGP and EIGRP, PfR can look into protocol’s topology information to determine both best paths and secondary paths thus, ECMP is not required.
- 60. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public EIGRP IWAN Design Overview • Single EIGRP process for Branch, WAN and POP/hub sites • Extend Hello/Hold timers for WAN • Adjust tunnel interface “delay” to ensure WAN path preference • MPLS primary, INET secondary • Hubs – Route tag filtering to prevent routing loops across DMVPNs – Branch prefix summary route for spoke-to-spoke tunnels • Spokes – EIGRP Stub for scalability Principles 60 INET DMVPN-2 MPLS DMVPN-1 10.1.10.0/24 10.1.11.0/24 10.1.12.0/24 10.1.13.0/24 Delay 1000 Delay 2000 Delay 1000 Set Interface Delay to influence best path EIGRP Stub R10 R11 R12 R13 TAG DMVPN-1 TAG DMVPN-2 IWAN POP1 10.8.0.0/16Hub MC 10.8.3.3/32MC1 IWAN POP2 10.9.0.0/16 R84 R85
- 61. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public IWAN Deployment – BGP • A single iBGP routing domain is used • Extend Hello/Hold timers for WAN • Hub: – DMVPN hub routers function as BGP route- reflectors for the spokes – BGP dynamic peer feature configured on the route-reflectors – Summary route to spokes – Set Community for site local prefixes • Spokes: – peer to a redundant pair of route-reflectors in each DMVPN cloud – Inbound route-map to set local-preference based on community – Ensure that preferred path is MPLS INETMPLS 10.1.10.0/24 10.1.11.0/24 10.1.12.0/24 10.1.13.0/24 IWAN POP1 iBGPiBGP Community 10:84 Community 10:85 Local Pref 200 LP 50 R84 R85 R10 R11 R12 R13 61 10.6.36.0/2310.6.34.0/23 Hub MC 10.8.3.3/32MC1 IWAN POP2 10.8.0.0/16 10.9.0.0/16
- 62. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public INETMPLS 10.1.10.0/24 10.1.11.0/24 10.1.12.0/24 10.1.13.0/24 MC1 R84 R85 MC/BR MC/BR MC/BR BR Transit Site – Hub MC • IWAN POP is the central hub for the Enterprise Domain • MC1 – Hub MC • BR1 – Hub BR, DMVPN Hub for MPLS • BR2 – Hub BR, DMVPN Hub for INET domain IWAN vrf default master hub source-interface Loopback0 domain IWAN vrf default border master 10.8.3.3 source-interface Loopback0 ! interface Tunnel100 description -- Primary Path -- domain IWAN path MPLS domain IWAN vrf default border master 10.8.3.3 source-interface Loopback0 ! interface Tunnel200 description – Secondary Path -- domain IWAN path INET MC1 BR1 BR2 Policies Monitors path MPLS path INET TRANSIT SITE
- 63. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public Single CPE Branch Sites 63 • Stub Site • Combination of MC and BR on the same CPE domain IWAN vrf default master branch source-interface Loopback0 hub 10.8.3.3 border master local source-interface Loopback0 R10 R11 INETMPLS 10.1.10.0/24 10.1.11.0/24 10.1.12.0/24 10.1.13.0/24 MC1 R84 R85 R10 R11 R12 R13 Hub MC 10.8.3.3/32 TRANSIT SITE
- 64. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public Dual CPE Branch Sites 64 • Stub Site • One of the BR is also the MC domain IWAN vrf default border master 10.2.12.12 source-interface Loopback0 R13 domain IWAN vrf default master branch source-interface Loopback0 hub 10.8.3.3 border master local source-interface Loopback0 R12 INETMPLS 10.1.10.0/24 10.1.11.0/24 10.1.12.0/24 10.1.13.0/24 MC1 R84 R85 R10 R11 R12 R13 Hub MC 10.8.3.3/32 TRANSIT SITE
- 65. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public Enterprise Domain Policy – DSCP or App Based 65 • When load balancing is enabled, PfRv3 adds a “default class for match all DSCP (lowest priority compared to all the other classes)” and we influence this traffic. • When load balancing is disabled, PfRv3 deletes this “default class” and as a part of that frees up the TCs that was learnt as a part of LB – they follow the routing table domain IWAN vrf default master hub load-balance class VOICE sequence 10 match dscp ef policy voice path-preference MPLS fallback INET class VIDEO sequence 20 match dscp af41 policy voice path-preference MPLS fallback INET class CRITICAL sequence 30 match dscp af31 policy low-latency-data MC1
- 66. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public Policy – Monitor Intervals Monitoring Interval Fast Reaction Time 2 sec 66 • Advanced commands – Carefully choose monitor interval for critical applications domain IWAN vrf default master hub monitor-interval 2 dscp ef monitor-interval 2 dscp af41 monitor-interval 2 dscp cs4 monitor-interval 2 dscp af31 MC1
- 67. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public Deploying With User VRFs • DMVPN Tunnel per VRF • Over the top routing per VRF • SAF Peering per VRF 67 Enterprise Branch Sites INETMPLS 10.1.10.0/24 10.1.11.0/24 10.1.12.0/24 10.1.13.0/24 MC1 R84 R85 R10 R11 R12 R13 vrf definition TEST1 ! address-family ipv4 exit-address-family ! vrf definition TEST2 ! address-family ipv4 exit-address-family ! 1 2 1 2 1 2 1 2 1 2 1 2 interface Tunnel 101 vrf forwarding TEST1 tunnel key 101 tunnel vrf IWAN-TRANSPORT-1 ! interface Tunnel 102 vrf forwarding TEST2 tunnel key 102 tunnel vrf IWAN-TRANSPORT-1 TRANSIT SITE
- 68. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public Deploying With VRF – Hub MC 68 Enterprise Branch Sites INETMPLS 10.1.10.0/24 10.1.11.0/24 10.1.12.0/24 10.1.13.0/24 MC1 R84 R85 R10 R11 R12 R13 interface Loopback1 vrf forwarding TEST1 ! interface Loopback2 vrf forwarding TEST2 domain IWAN vrf TEST1 master hub source-interface Loopback1 ! vrf TEST2 master hub source-interface Loopback2 GLOBAL: 10.8.3.3 VRF TEST1: 11.8.3.3 VRF TEST2: 12.8.3.3 TRANSIT SITE
- 69. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public Deploying With VRF – Hub MC Policies 69 domain IWAN vrf TEST1 master hub load-balance class VOICE sequence 10 match dscp ef policy voice path-preference MPLS fallback INET class VIDEO sequence 20 match dscp af41 policy voice path-preference MPLS fallback INET class CRITICAL sequence 30 match dscp af31 policy low-latency-data [Cont’d] vrf TEST2 master hub load-balance class VOICE sequence 10 match dscp ef policy voice path-preference MPLS fallback INET class CRITICAL sequence 30 match dscp af31 policy low-latency-data
- 70. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public Deploying With VRF – Hub BR 70 Enterprise Branch Sites INETMPLS 10.1.10.0/24 10.1.11.0/24 10.1.12.0/24 10.1.13.0/24 MC1 R84 R85 R10 R11 R12 R13 domain IWAN vrf TEST1 border master 11.8.3.3 source-interface Loopback1 ! vrf TEST2 border master 12.8.3.3 source-interface Loopback2 ! interface Tunnel101 description -- Primary Path – vrf forwarding TEST1 domain IWAN path MPLS ! interface Tunnel102 description -- Primary Path – vrf forwarding TEST2 domain IWAN path MPLS Tu101 Tu102 GLOBAL: 10.8.3.3 VRF TEST1: 11.8.3.3 VRF TEST2: 12.8.3.3 TRANSIT SITE
- 71. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public Deploying With VRF – Branch MC/BR 71 Enterprise Branch Sites INETMPLS 10.1.10.0/24 10.1.11.0/24 10.1.12.0/24 10.1.13.0/24 MC1 R84 R85 R10 R11 R12 R13 Tu101 Tu102 domain IWAN vrf TEST1 master branch source-interface Loopback1 hub 11.8.3.3 border master local source-interface Loopback1 ! vrf TEST2 master branch source-interface Loopback2 hub 12.8.3.3 border master local source-interface Loopback2 R10 GLOBAL: 10.8.3.3 VRF TEST1: 11.8.3.3 VRF TEST2: 12.8.3.3 TRANSIT SITE
- 72. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public INETMPLS 10.1.10.0/24 10.1.11.0/24 10.1.12.0/24 10.1.13.0/24 MC1 R84 R85 R10 R11 R12 R13 Redundant MC – Anycast IP In IWAN POP 72 • What happens when a MC fails? • Traffic forwarded based on routing information – ie no drop • What happens when the Hub MC fails? • Branch MCs keep their configuration and policies • Continue to optimise traffic • A backup MC can be defined on the hub. • Using the same IP address as the primary • Routing Protocol is used to make sure BRs and branch MC connect to the primary • Stateless redundancy • Backup MC will re-learn the traffic Backup Hub MC 10.8.3.3/31MC2 Hub MC 10.8.3.3/32 TRANSIT SITE
- 73. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public Dual POPs – Different Prefixes • Requirements: – Separate data centres/POPs – Separate prefix advertised from each data centres to spokes • POP2 Hub MC – Configured as Branch 73 10.1.10.0/24 10.1.11.0/24 10.1.12.0/24 10.1.13.0/24 R10 R11 R12 R13 R84 R85 R94 R95 MC1 IWAN POP1 IWAN POP2 MC2 MPLS INET DCI WAN Core EIGRP/BGP 10.8.0.0/16 EIGRP/BGP 10.8.0.0/16 EIGRP/BGP 10.9.0.0/16 EIGRP/BGP 10.9.0.0/16 10.8.0.0/16 10.9.0.0/16
- 74. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public Horizontal Scaling Architecture • Limitations: – PfRv3 manages traffic between Tunnel Interfaces, not multiple tunnels within a single Tunnel Interface – Spokes have multiple next hops on the same DMVPN tunnel Interface – Channel definition: • local site id + remote site id + DSCP + colour(SP) • No differentiation for multiple channels within a colour(SP) • Solution: PfRv3 DMVPN Multiple Next Hop support – Need to add sub-colour to differentiate channels – New channel definition • local site id + remote site id + DSCP + colour(SP) + BR tag – BR1 with tag 1, BR2 with tag 2 • Targeted for April CY15 XE 3.15 / 15.5(2)T releases PfRv3 Multiple DMVPN Next Hop Limitation 74 INETMPLS 10.1.10.0/24 10.1.11.0/24 10.1.12.0/24 10.1.13.0/24 Hub MC 10.8.3.3/32 BR1 BR2 BR3 BR4 MC1 R10 R11 R12 R13 Next Hop 1 Next Hop 2 TRANSIT SITE
- 75. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public Multiple POPs – Common Prefixes • Requirements: – 2 (or more) Transit Sites advertise the very same set of prefixes – Data centre may not be collocated with the Transit Sites – DCs/DMZs are reachable across the WAN Core for each Transit Site – Branches can access any DC or DMZ across either POP(hub). And, DC/DMZs can reach any branch across multiple Transit Sites (hubs). – Multiple BRs per DMVPN per site may be required for crypto and bandwidth horizontal scaling • Targeted for April CY15 XE 3.15 / 15.5(2)T releases 10.1.10.0/24 10.1.11.0/24 10.1.12.0/24 10.1.13.0/24 R10 R11 R12 R13 BR1 BR2 BR3 BR4 MC1 TRANSIT SITE1 TRANSIT SITE2 MC2 DMVPN MPLS DMVPN INET DCI WAN Core 10.8.0.0/16 10.9.0.0/16 10.8.0.0/16 10.9.0.0/16 DC1
- 76. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public Monitoring Operations R83#sh domain one master traffic-classes summary APP - APPLICATION, TC-ID - TRAFFIC-CLASS-ID, APP-ID - APPLICATION-ID SP - SERVICE PROVIDER, PC = PRIMARY CHANNEL ID, BC - BACKUP CHANNEL ID, BR - BORDER, EXIT - WAN INTERFACE UC - UNCONTROLLED, PE - PICK-EXIT, CN - CONTROLLED, UK - UNKNOWN Dst-Site-Pfx Dst-Site-Id APP DSCP TC-ID APP-ID State SP PC/BC BR/EXIT 10.1.13.0/24 10.2.13.13 N/A ef 11 N/A CN MPLS 25/27 10.8.4.4/Tunnel100 10.1.12.0/24 10.2.12.12 N/A ef 9 N/A CN MPLS 20/19 10.8.4.4/Tunnel100 10.1.11.0/24 10.2.11.11 N/A af31 6 N/A CN MPLS 24/22 10.8.4.4/Tunnel100 10.1.13.0/24 10.2.13.13 N/A af31 8 N/A CN MPLS 26/28 10.8.4.4/Tunnel100 10.1.12.0/24 10.2.12.12 N/A af31 4 N/A CN MPLS 11/12 10.8.4.4/Tunnel100 10.1.11.0/24 10.2.11.11 N/A default 5 N/A CN INET 8/NA 10.8.5.5/Tunnel200 [SNIP] Total Traffic Classes: 12 Site: 6 Internet: 0 R83# Traffic Class – Site11 - Critical TC Id Controlled Path Information - Channels
- 77. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public Check Traffic Classes Details R83#sh domain one master traffic-classes Dst-Site-Prefix: 10.1.10.0/24 DSCP: ef [46] Traffic class id:25 TC Learned: 00:12:44 ago Present State: CONTROLLED Current Performance Status: in-policy Current Service Provider: INET since 00:06:01 Previous Service Provider: INET for 181 sec (A fallback provider. Primary provider will be re-evaluated 00:00:01 later) BW Used: 24 Kbps Present WAN interface: Tunnel200 in Border 10.8.5.5 Present Channel (primary): 84 Backup Channel: 85 Destination Site ID: 10.2.10.10 Class-Sequence in use: 10 Class Name: VOICE using policy User-defined priority 2 packet-loss-rate threshold 5.0 percent priority 1 one-way-delay threshold 150 msec priority 2 byte-loss-rate threshold 5.0 percent BW Updated: 00:00:14 ago Reason for Route Change: Delay Check Traffic Class Voice for site 10 Check Channels used (Primary and Backup) Policies and reason for last change Active Path used
- 78. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public Check Channel After TCA 78 MC1#sh domain IWAN master channels | beg 107 Channel Id: 107 Dst Site-Id: 10.2.11.11 Link Name: MPLS DSCP: ef [46] TCs: 1 Channel Created: 00:15:03 ago Provisional State: Initiated and open Operational state: Available Interface Id: 11 Estimated Channel Egress Bandwidth: 0 Kbps Immitigable Events Summary: Total Performance Count: 0, Total BW Count: 0 ODE Stats Bucket Number: 1 Last Updated : 00:05:45 ago Packet Count : 2 Byte Count : 116 One Way Delay : 254 msec* Loss Rate Pkts: 0.66 % Loss Rate Byte: 0.0 % Jitter Mean : 38000 usec Unreachable : FALSE [SNIP] TCA Statistics: Received:1 ; Processed:1 ; Unreach_rcvd:0 Latest TCA Bucket Last Updated : 00:05:45 ago One Way Delay : 266 msec Loss Rate Pkts: NA Loss Rate Byte: NA Jitter Mean : NA Unreachability: FALSE On Demand Export (ODE) Threshold Crossing Alert (TCA)
- 79. PfRv3 Management
- 80. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public PfRv3 Exporter Configuration • Enable exporter on the Hub MC – Collector IP address – Default UDP port 9995 • Distributed through SAF to all MCs and BRs in the domain 80 domain IWAN vrf default master hub collector 10.151.1.95 port 2055 MC1 INETMPLS 10.1.10.0/24 10.1.11.0/24 10.1.12.0/24 10.1.13.0/24 Hub MC 10.8.3.3/32 IWAN POP MC1 BR1 BR2 R10 R11 R12 R13
- 81. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public PfRv3 NetFlow Export • Exports from MC – TCA record – Route Change record – Immitigable Event Summary • Bandwidth – Exports from BRs – Egress Measurement Template – Ingress Measurement Template • All records available at: – http://docwiki.cisco.com/wiki/PfRv3:Reporting List of Templates 81
- 82. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public Specialised Management Cloud-Based Management • Eliminates manual building of WANs • Automated SD-WAN orchestration • Centralised hybrid WAN management • Quick config updates and IOS upgrades • Leverages onePK and REST APIs • Integrates with Cisco AVC and PfR • Monitor and analyse application traffic • End-to-end flow visualisation • Flow & App-based Troubleshooting • Fix and Verify in Realtime Cisco IWAN Management Automates Deployment and Lifecycle Management Application Aware Network Performance Management On-Prem Management Prime Infrastructure 2.2 • Single-pane view of IWAN • IWAN deployment workflows • Plug and Play • DMVPN, QoS, AVC deployment and monitoring • PfR v3 in Q1 2015 • License includes IWAN App and APIC- EM controller! End-to-End Assurance of Application Experience
- 83. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public 83 2. Alert and report on PfR Out of Policy events 1. PfR path change visualisation 3. Click on Alerts to get the details LiveAction 4.1 1. Alert Workflow
- 84. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public 84 PfRv3 TCA Alerts on DC1 Drill down to site pairs
- 85. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public 85 Alerts / performance by Site Alerts / performance by Application Group All Alerts 2. PfRv3 Dashboard
- 86. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public 86 Alerts / performance by Service Provider
- 87. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public APIC- EM Device Abstraction Layer REST APIs APIC-EM Services (Partial) CLIOnePK/OpenFlow PKI Svc NetFlow Svc PnP Svc Network Svc Events Svc Inventory Svc IWAN Automation and Orchestration Evolution Traditional Management Systems CiscoPrime Evolution Apps IWAN Transport PKI Automation Security Intelligent Path Control Cisco IWAN Apps Partners (future) Application Experience PnP Provisionin g Mid CY2015 Capacity Planning, Troubleshooting, Change controlPrime 87
- 88. Key Takeaways
- 89. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public Performance Routing – IOS and IOS-XE Releases 89 PfR/OER version 1 IOS 12.3(8)T, XE 2.6.1 PfR version 2 IOS 15.2(3)T, IOS-XE 3.6 PfR version 3 IOS 15.4(3)M, IOS-XE 3.13 Per Device provisioning Passive monitoring with Traditional NetFlow (TNF) Active monitoring with IP SLA Manual provisioning jitter probes 1000’s lines of configuration (pfr-map per site) Per Device provisioning Target Discovery (TD) Automatic provisioning of jitter probes Passive monitoring with Traditional NetFlow (TNF) Active monitoring with IP SLA 10’s lines of configuration PfR Domain One touch provisioning Auto Discovery of sites NBAR2 support Passive Monitoring (performance monitor) Smart Probing VRF Awareness IPv4/IPv6 (Future) <10 lines of configuration and centralized Blackout 6 seconds Brownout 9 seconds Limited scalability due to provisioning (~ tens of sites) Blackout 6 seconds Brownout 9 seconds Scale 500 sites Blackout ~ 2 sec Brownout ~ 2 sec Scale 2000 sites
- 90. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public Performance Routing – Platform Support Cisco ISR G2 family 3900-AX 2900-AX 1900-AX 890 Cisco ISR 4000 4400 4300 Cisco ASR-1000 Cisco CSR-1000 MC BR MC BR MC BR MC BR* * BR support coming
- 91. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public Key Takeaways • IWAN Intelligent Path Control pillar is based upon Performance Routing (PfR) – Maximises WAN bandwidth utilisation – Protects applications from performance degradation – Enables the Internet as a viable WAN transport – Provides multisite coordination to simplify network wide provisioning. – Application-based policy driven framework and is tightly integrated with existing AVC components. • PfRv3 is the 3rd generation Multi-Site aware Bandwidth and Path Control/Optimisation solution for WAN/Cloud based applications. 91
- 92. © 2015 Cisco and/or its affiliates. All rights reserved.BRKRST-2362 Cisco Public More Information • Cisco.com IWAN/PfR Page: – http://www.cisco.com/go/iwan – http://www.cisco.com/go/pfr • PfRv3 Home Page – http://docwiki.cisco.com/wiki/PfRv3:Home • Leverage dcloud.cisco.com virtual labs • LiveAction: – The New LiveAction 4.1: http://liveaction.com/new-liveaction-4-1/ – Download LiveAction v4.1.2: http://liveaction.com/download/links/ – LiveAction PfRv3 Demo: http://player.vimeo.com/video/103767237 – LiveAction 4.1 IWAN Webinar, October 2nd 2014: http://liveaction.com/webinars/oct-02-2014/?elq=d49b1ff6a68b43a7a4f8e81baa8bc801&elqCampaignId=172 92
- 93. Q & A