BGP FlowSpec experience and future developments

BGP FlowSpec in 2019: operational
experience and future developments
a.k.a. “yet another FlowSpec talk”
Alessandro Bulletti
Kirill Kasavchenko

Volumetric DDoS attacks 2018-2019
Source: Netscout
Threat Report 2018
Source: Rapporto Clusit
2019 sulla Sicurezza ICT in
Italia - Tipologie di attacchi
DDoS (Dati Fastweb
relativi all’anno 2018)

The need for collaboration
• Ideally, large DDoS attacks should be
mitigated as close to the source as possible.
• Practically, the good first step would be:
– Customer asks upstream provider to mitigate
– Peer asks another peers
• As time goes by and trust is built, we might
expect mitigation to naturally propagate closer
to the source.

Wait... This is not a new idea!
Some of initiatives in this area:
• Fingerprint Sharing Alliance (Arbor, 2005)
• BGP FlowSpec (IETF, 2009)
• M3AAWG DDoS Info Sharing (2017)
• I2NSF (IETF, 2017)
• Stellar: advanced blackholing (DE-CIX/TU
Berlin/Max Planck Institute, 2018)
• DOTS (IETF, work in progress)

Wait... This is not a new idea!
Some of initiatives in this area:
• Fingerprint Sharing Alliance (Arbor, 2005)
• BGP FlowSpec (IETF, 2009)
• M3AAWG DDoS Info Sharing (2017)
• I2NSF (IETF, 2017)
• Stellar: advanced blackholing (DE-CIX/TU
Berlin/Max Planck Institute, 2018)
• DOTS (IETF, work in progress)
• Vendor support
• Open-source tools
• Operational experience
• User groups

FlowSpec: vendors and open-source
Controllers and BGP FlowSpec speakers:
• Arbor (Netscout)
• ExaBGP
• FastNetMon
• Nokia
• Radware
Routers:
• Cisco IOS-XR (ASR9K, NCS5K) and IOS-XE
• Juniper MX/PTX
• Alcatel/Nokia 7x50
• Huawei NE routers
• 6WIND
• Arista
Telemetry FlowSpec

FlowSpec: operational experience & user groups
Deployments (based on public information):
• Cloudflare, Level3, Rostelecom, Orange Poland
DDoS Peering initiative:
• AT&T, Century Link, Charter
• See slides at NANOG 71 and 75 (links in the end of this presentation)
FlowSpec multi vendor tests by T-Mobile Austria and NextLayer :
• Cisco, Juniper, Huawei, Nokia
• Focus on Inter-AS applicability and operational needs

What works well
• FlowSpec is extensively used against DDoS Amplification attacks:
• We see operational community is automating FlowSpec generation to stop
attacks without human involvement:
– Not just hyperscalers
• We see adoption of FlowSpec as the way to redirect traffic on demand:
– Redirect to Route Target
– Redirect to Next-Hop

What could be improved
• Scale on router side:
– Number of concurrently supported FlowSpec rules
– Speed of FlowSpec rules import and processing
• Various „safety nets“:
– Prevent too many FlowSpec rules from impacting router forwarding performance
– Lack of granularity in import policies:
• It is impossible to allow/reject FlowSpec rules based on action
• Therefore, possible for customer to change DSCP of their traffic to „network-critical“
• These are implementation-specific improvements. They do not require industry
consensus in a form of IETF document or BCP.

What is impossible to do with FlowSpec today
• Match traffic based on random L3-L4 field or based on payload
– Example: IP TTL, TCP Window Size
• Report traffic statistics across administrative boundaries
– ISP1 sends a FlowSpec rule to ISP2
– How ISP1 gets visibility into dropped traffic to estimate the efficacy of FlowSpec?
• There is a lack of common approach to these problems.
– We need standards (think RFC documents) or some well-defined Best Current Practices

Flexible Payload Matching: problem statement
• Majority of volumetric DDoS attacks are amplification attacks. They can be
successfully mitigated using „standard“ FlowSpec, matching source UDP port:
• However, there exist more complex cases:
– Bittorent amplification
– SSDP diffraction
• These attacks result in packets with random source UDP port, so it‘s not possible
to create a classifier using FlowSpec.
• At the same time, these attacks generate specific patterns in payload.
– And some routers are able to match packets based on payload.

Flexible Payload Matching: solution (1/2)
• Extend FlowSpec with „flexible match conditions“ component type:
draft-khare-idr-bgp-flowspec-payload-match
Type X – Flexible Match Conditions

Flexible Payload Matching: solution (2/2)
• Use new component type to match
anywhere in the packet:
– In any field
– Across fields regardless of field boundaries
– In payload
• Match using:
– Bit pattern with bitmask
– Numeric pattern
– Regular expressions
draft-khare-idr-bgp-flowspec-payload-match

FlowSpec and Reporting: problem statement
• How do you know if FlowSpec rule is effective?
• No standardized way to report traffic that matches FlowSpec policy:
– SNMP counters
– XML/Netconf
– draft-wu-idr-flowspec-yang-cfg-02
• Good news: most routers generate flow telemetry for traffic, dropped by
FlowSpec:

FlowSpec and Reporting: solution
• Corellate FlowSpec NLRI and Flow records:
• Vendor-independent approach
– No need to parse proprietary SNMP or XML data models
FlagsToS
Output Intf =
0
Input IntfProtoDst PortSrc PortDest IPSrc IP
Flow Record
Packet
length
TCP
flag
s
ICMP
type
Dst
Port
Por
t
IP
Protoc
ol
Source
Prefix
Dest
Prefix
FlowSpec NLRI
Src
Port
ICMP
code
Etc…

FlowSpec and Reporting: Inter-AS?
• Reporting based on Netflow and FlowSpec correlation won‘t work between ASNs
– No way for ISP1 to receive flow telemetry from ISP2
• Other approaches (SNMP, Netconf, streaming telemetry) won‘t work either
• We would need something totally new:
– Most likely some mediation tool that makes sure only relevant information is reported
between ASNs
– Or some concept of „administrative domains“ in flow / SNMP / streaming telemetry
• Whatever the solution would be, having visibility into FlowSpec rules propagated
between ASNs is going to be an essential factor for adoption

FlowSpec evolution: RFC5575bis
• Defines all traffic action extended communities as transitive extended
communities. RFC5575 defined the traffic- rate action to be non-transitive and did
not define the transitivity of the other action communities at all.
• Introduces a new traffic filtering action traffic-rate-packets:
– Instead of ”rate-limit to 100 bps” you can now “rate-limit to 100 pps”
• Introduces rules how updates of Flow Specifications shall be handled in case
they contain interfering actions
• A full list of changes can be found in the references section of this slide deck
Major changes

Other interesting FlowSpec developments
• FlowSpec interface set (draft-ietf-idr-flowspec-interfaceset)
– Use BGP communities to signal to which interfaces (interface-set) should a FlowSpec
action be applied
– Supported in Cisco IOS-XE

Summary
• FlowSpec is a mature DDoS mitigation technology
• No, it‘s not perfect and alone is not enough to stop all types of DDoS attacks
– it is not applicable against TCP SYN floods and application layer DDoS
– but it mitigates the most impacting DDoS types in ISP networks – amplification attacks
• Check with $your_router_vendor what the platform limits are:
– Scale
– Policies
• Don‘t be afraid to automate FlowSpec. If you don‘t know how – talk to us.
• Work with IETF and vendors to get new FlowSpec ideas implemented
A.k.a Last Slide

References
• Stellar, advanced blackholing: https://www.de-
cix.net/Files/2731074c857497be3827ac9537b6e486f27aa57c/Research-paper-Stellar-Network-Attack-Mitigation-
using-Advanced-Blackholing.pdf
• NANOG71 DDoS peering: https://www.youtube.com/watch?v=Aj8EKYVSTtk
• NANOG75 eBGP FlowSpec peering: https://www.youtube.com/watch?v=rKEz8mXcC7o
• Interop FlowSpec tests by T-Mobile and Next Layer: https://www.nextlayer.at/wp-content/uploads/2018/06/loibl-bacher-
bgp-flowspec-interop-012017.pdf
• FlowSpec payload matching: https://datatracker.ietf.org/doc/draft-khare-idr-bgp-flowspec-payload-match/
• SSDP Diffraction Attack: https://www.netscout.com/blog/asert/new-twist-ssdp-attacks
• FlowSpec at CloudFlare: https://meetings.ripe.net/see4/files/RIPE%20SEE4%20Belgrade%20Serbia%20-
%20CloudFlare%20-%20DDoS%20mitigation%20-%20Martin%20Levy.pdf
• FlowSpec at Level3: https://www.capacitymedia.com/articles/3585228/Level-3-launches-BGP-Flowspec-on-global-
backbone
• FlowSpec at Rostelecom: https://habr.com/ru/company/rostelecom/blog/325138/
• FlowSpec in Orange Poland: https://cert.orange.pl/aktualnosci/ddos-w-naszej-sieci-coraz-silniej-coraz-wiecej
• RFC5575bis changes compared to RFC5575: https://tools.ietf.org/html/draft-ietf-idr-rfc5575bis-14#appendix-A
The last slide, really

Thank You.
www.netscout.com
alessandro.bulletti@netscout.com
kirill.kasavchenko@netscout.com

BGP FlowSpec experience and future developments

More Related Content

What's hot (20)

Similar to BGP FlowSpec experience and future developments (20)

More from Pavel Odintsov (20)

Recently uploaded (20)

BGP FlowSpec experience and future developments