IETF 113: IPv6 fragmentation and EH behaviours
0 likes348 views
The document discusses IPv6 fragmentation and the behaviors of extension headers, highlighting significant changes from IPv4. Initial tests revealed high drop rates for fragmented packets, attributed to firewalls and ICMPv6 blockages, with improvements noted in later experiments. Recent findings indicate that while the overall handling of fragmented IPv6 packets has improved, extension headers like destination and hop-by-hop options face severe support issues across public internet infrastructure.
IETF 113: IPv6 fragmentation and EH behaviours
- 1. IPv6 Fragmentation and EH behaviours Geoff Huston, Joao Damas APNIC Labs
- 2. Version IHL Total Length Flags Identification Fragment Offset Time To Live Source Address Destination Address Options Padding Protocol Header Checksum Type of Service Version Class Flow Payload Length Hop Limit Source Address Destination Address Next Header IPv4 Header IPv6 Header IPv6: Packet Header Changes These three fields in the IPv4 header were pushed into the Extension Header chain, and do not appear in every IPv6 packet This is a new field
- 3. IPv6: Packet Header Changes • Type of Service changed to Traffic Class • (yet to find a useful agreed role, even ECN!) • A Flow Label added • (but yet to find a useful role!) • Header Checksum becomes a media layer function • Options, Protocol fields replaced by chained Extension Headers • Packet ID and Fragmentation Controls become an Extension Header
- 4. IPv6: Packet Header Changes • Type of Service changed to Traffic Class • (yet to find a useful agreed role, even ECN!) • A Flow Label added • (but yet to find a useful role!) • Header Checksum becomes a media layer function • Options, Protocol fields replaced by chained Extension Headers • Packet ID and Fragmentation Controls become an Extension Header The substantive change with IPv6 is the handling of Fragmentation and the use of Extension Headers
- 5. Initial Tests: 2014 (RFC 7872) • August 2014 and June 2015 • Sent fragmented IPv6 packets towards “well known” IPv6 servers (Alexa 1M and World IPv6 Launch • Drop Rate: This is bad! Really bad!
- 6. Initial Tests: 2014 (RFC 7872) • August 2014 and June 2015 • Sent fragmented IPv6 packets towards “well known” IPv6 servers (Alexa 1M and World IPv6 Launch • Drop Rate: This is also bad!
- 7. Why is Frag Drop so high? Some possible reasons for the high drop rate: • Packet Fragments are often dropped by firewalls • IPv6 Path MTU measures rely on ICMPv6 (as there is no ability for the router to fragment on the fly), and ICMP messages are commonly blocked • Extension Header chains may either not be supported in router, or may only supported in the processor path (slow path)
- 8. IPv6 Fragmentation Extension Header Handling The extension header sits between the IPv6 packet header and the upper level protocol header for the leading fragged packet, and sits between the header and the trailing payload frags for the trailing packets Practically, this means that transport-protocol aware packet processors/switches need to decode the extension header chain, if its present, which can consume additional cycles to process/switch a packet – and the additional time is not predictable. For trailing frags there is no transport header! Or the transport-protocol aware unit can simply discard all IPv6 packets that contain extension headers! IPv6 header Payload TCP/UDP xtn header Fragmentation extn header
- 9. Are the effects of middleware symmetric? The RFC7872 experiments sent altered IPv6 packets towards well-known servers and observed whether the server received and reconstructed the altered packet by seeing whether the server responded (or not) Sending large fragmented queries towards servers is not all that common – the reverse situation of receiving big responses is more common
- 10. Are the effects of middleware symmetric? The RFC7872 experiments sent altered IPv6 packets towards well-known servers and observed whether the server received and reconstructed the altered packet by seeing whether the server responded (or not) Sending large fragmented queries towards servers is not all that common – the reverse situation of receiving big responses is more common What happens if we try to reproduce this experiment by looking at what happens when we send various forms of altered IPv6 packets back from servers – what’s the drop rate of this reverse case for packets from servers to end-clients? ?
- 11. IPv6 Fragmentation Extension Header Handling We used an ad-based measurement system, using a custom packet fragmentation wrangler as a front end to a DNS and Web server to test IPv6 fragmentation behaviour Client DNS Resolver IPv6 DNS Server IPv6 Web Server IPv6 ‘Fragmenter’ DNS Goo Fragmented packets
- 12. APNIC Test – August 2017 • Use APNIC IPv6 measurement platform to test the drop rate of IPv6 fragmented packets flowing in the opposite direction (server to client) Count % Tests 1,675,898 ACK Fragmented Packets 1,324,834 79% Fragmentation Loss 351,064 21% This is an improvement over the RFC7872 measurement, which reported a 28% drop rate client to server
- 13. APNIC Test – 2021 onward Re-work of the 2017 measurement experiment • Same server-to-client TCP session fragmentation mechanism • Uses a dedicated middlebox to fragment outgoing packets to improve packet handling capacity of the experiment • drop is detected by a hung TCP session that fails to ACK the sequence number in the fragmented packet • This time we randomly vary the initial fragmented packet size between 1,200 and 1,416 bytes • Performed as an ongoing measurement
- 14. IPv6 Packet Mangler Design • Easier said than done when we are limited to user-space code on standard cloud processing platforms • We needed a promiscuous capture mechanism that works in user space. We used the libpcap libraries to perform packet capture, and used IP packet filters to stop the kernel’s persistent desire to send RST packets • The problem is that the pcap libraries have no buffers so were are dropping packets under load. We resorted to using multiple processes and shared memory ring buffers to improve throughput • We also split the back end server and the packet mangling function to separate units, so we also implemented an IPv6 NAT function in the packet mangler again as a shared memory structure Input capture Input capture Shared Ring Buffer Shared Ring Buffer Shared NAT table IPv6 NAT IPv6 NAT + Packet Fragmenter External Client Server Packet Mangler
- 15. 2021 Fragmentation Drop Rate This is a significant improvement over 2017 data Since 2017 there are 10x the number of IPv6 users and the fragmentation drop rate has come down by 2/3 – we appear to be getting better at handling IPv6 fragments in the longer term!
- 16. 2021 Fragmentation Drop Rate More recent IPv6 deployments appear to be better at frag handling than more mature ones
- 17. Drop Rate by Size This is unexpected. Why does the drop rate increase so markedly when the fragmented packet size passes over the threshold of 1,360 octets?
- 18. Drop Size Profile by Region Americas Europe Asia
- 19. Why? • Drop patterns vary across service providers, so there are probably contributary factors from network equipment and configurations 4% Drop! 95% Drop! This was us!
- 20. Why? Other potential factors that could contribute: • Local security policies in user-facing edge devices • IPv6 EH may trigger “slow path” processing in network equipment that could lead to higher drop rates • IPv6 Path MTU woes! • TCP MSS settings interfere with the measurement
- 21. Frag Drop or EH Drop? We added a further test to try and see the difference between Fragmentation and Extension Headers • We used an ”atomic” fragment, which is a IPv6 packet with a Fragmentation Header where the fragment offset is 0 and the M (more) bit is also 0
- 22. 2021 Fragmentation Drop Rate Atomic Fragment drop rate is 6%
- 23. Frags vs Atomic Frags • Most of the time the Atomic Frag drop rate is ~3x lower than the Fragmented packet drop rate • Except when it’s not!
- 24. Frag Drop or EH Drop? • The Atomic Frag data isn’t really as informative as we would’ve liked • We thought that all hosts would accept incoming packets with an Atomic Frag header • After all the Atomic Frag header is a no-op for the packet! • So if there are Atomic Frag packet drops it should be a network effect • The Atomic Frag drop rate should always be less than the fragmented packet drop rate • But this is not always the case in our data • So we looked for other Extension Headers to test
- 25. EH DST Drop? • What’s another innocuous Extension Header we can use? • There is the PADN Destination Option • In this way can can compare the network treatment of extension headers to the treatment of fragmentation headers • Let’s use it! • Because padding does not relying on any particular functionality in the host, so hosts should accept it • In theory Destination Options should be handled by the network as a neutral option, as the option signalling is about the destination host, not the network elements that switch the packet in transit
- 26. Destination Option Drop Rate January 2022: 94.5% drop rate Wow! That’s awesomely bad! It seems that most hosts are dropping incoming packets with unexpected destination options, whether they contain directives of just padding or other directives, but we need to test this against various commonly used IPv6 protocol stacks to test this a little more
- 27. EH HBH Drop? • What’s another innocuous Extension Header we can use? • There is the PADN Hop-by-Hop Option • In this way can can compare the network treatment of extension headers to the treatment of fragmentation headers • Let’s try this • Because padding does not relying on any particular functionality in the host, so hosts should accept it • Again, this is a simple padding option so no special processing is being requested from the network’s switches
- 28. Hop-by-Hop Option Drop Rate February 2022: 99.5% drop rate Wow! That’s awesomely even badder! It seems that most hosts and routers are dropping incoming packets with destination and hop-by-hop options, but we need to test this some more
- 29. Summary • The network is slowly improving it’s handling of fragmented IPv6 packets • In 5 years its gone from unusably bad to tolerably poor • Recent IPv6 deployments appear to show more robust general handling of IPv6 packets • Destination Extension Headers and Hop-by-Hop Extension Headers are a completely lost cause - they are not usefully supported on public Internet infrastructure
- 30. Lessons Learned on Fragmentation • Don’t Fragment outgoing packets • Don’t rely on PMTUD • More generally, don’t rely on Extension Headers nor on ICMP6 integrity • Pick your TCP MSS setting carefully: • 1280 is a robust local MTU size – but possibly too conservative • 1350 is survivable as a MTU size – the Goldilocks choice • 1400 is risky – but survivable in most cases • 1500 is a poor choice – this leads to visible failure cases
- 31. Lessons Learned on HBH and Dst EHs • Don’t
- 33. That’s it!