Reconsider TCPdump for Modern Troubleshooting
Download as PPTX, PDF0 likes408 views
The document discusses modern troubleshooting techniques using tcpdump alongside advancements in networking technology, emphasizing the importance of capturing relevant packets and utilizing proper filters. It highlights the evolution of protocols like HTTP/2 and TLS 1.3, and the integration of artificial intelligence for anomaly detection to enhance network analysis in multi-cloud environments. The author suggests that while tcpdump remains a useful tool, transitioning to newer technologies and methodologies is essential for effective network management.
![TCPdump to the Rescue
7 6 5 4 3 2 1 0 0*2 + 0*2 + 0*2 + 1*2 + 0*2 + 0*2 + 1*2 + 0*2 = 18
Now we can't just use 'tcp[13] == 18' in the tcpdump filter expression, because that would select only those packets that have
SYN-ACK set, but not those with only SYN set. Remember that we don't care if ACK or any other control bit is set as long as
SYN is set.
In order to achieve our goal, we need to logically AND the binary value of octet 13 with some other value to preserve the
SYN bit. We know that we want SYN to be set in any case, so we'll logically AND the value in the 13th octet with the binary
value of a SYN:
00010010 SYN-ACK 00000010 SYN AND 00000010 (we want SYN) AND 00000010 (we want SYN) -------- -------- = 00000010 = 00000010
We see that this AND operation delivers the same result regardless whether ACK or another TCP control bit is set. The
decimal representation of the AND value as well as the result of this operation is 2 (binary 00000010), so we know that for
packets with SYN set the following relation must hold true:
( ( value of octet 13 ) AND ( 2 ) ) == ( 2 )
This points us to the tcpdump filter expression
tcpdump -i xl0 'tcp[13] & 2 == 2'
Some offsets and field values may be expressed as names rather than as numeric values. For example tcp[13] may be
replaced with tcp[tcpflags]. The following TCP flag field values are also available: tcp-fin, tcp-syn, tcp-rst, tcp-push, tcp-ack,
tcp-urg.
This can be demonstrated as:
tcpdump -i xl0 'tcp[tcpflags] & tcp-push != 0'
Note that you should use single quotes or a backslash in the expression to hide the AND ('&') special character from the
shell.
UDP Packets
UDP format is illustrated by this rwho packet:
https://www.tcpdump.org/manpages/tcpdump.1.html
For the past 30 years
• Powerful
• Detailed
• Specialized
For the next 30 years
• Speed and time
• Predictability
• Future-proofing](https://image.slidesharecdn.com/tcpdumpwebinarnetworkworld041219-190422201232/85/Reconsider-TCPdump-for-Modern-Troubleshooting-2-320.jpg)
Reconsider TCPdump for Modern Troubleshooting
- 1. Reconsider TCPdump and Move Forward with Modern Troubleshooting Chad Tripod, Avi Networks
- 2. TCPdump to the Rescue 7 6 5 4 3 2 1 0 0*2 + 0*2 + 0*2 + 1*2 + 0*2 + 0*2 + 1*2 + 0*2 = 18 Now we can't just use 'tcp[13] == 18' in the tcpdump filter expression, because that would select only those packets that have SYN-ACK set, but not those with only SYN set. Remember that we don't care if ACK or any other control bit is set as long as SYN is set. In order to achieve our goal, we need to logically AND the binary value of octet 13 with some other value to preserve the SYN bit. We know that we want SYN to be set in any case, so we'll logically AND the value in the 13th octet with the binary value of a SYN: 00010010 SYN-ACK 00000010 SYN AND 00000010 (we want SYN) AND 00000010 (we want SYN) -------- -------- = 00000010 = 00000010 We see that this AND operation delivers the same result regardless whether ACK or another TCP control bit is set. The decimal representation of the AND value as well as the result of this operation is 2 (binary 00000010), so we know that for packets with SYN set the following relation must hold true: ( ( value of octet 13 ) AND ( 2 ) ) == ( 2 ) This points us to the tcpdump filter expression tcpdump -i xl0 'tcp[13] & 2 == 2' Some offsets and field values may be expressed as names rather than as numeric values. For example tcp[13] may be replaced with tcp[tcpflags]. The following TCP flag field values are also available: tcp-fin, tcp-syn, tcp-rst, tcp-push, tcp-ack, tcp-urg. This can be demonstrated as: tcpdump -i xl0 'tcp[tcpflags] & tcp-push != 0' Note that you should use single quotes or a backslash in the expression to hide the AND ('&') special character from the shell. UDP Packets UDP format is illustrated by this rwho packet: https://www.tcpdump.org/manpages/tcpdump.1.html For the past 30 years • Powerful • Detailed • Specialized For the next 30 years • Speed and time • Predictability • Future-proofing
- 3. TCPdump Timeline State of the affairs • Real-time triage • Multi-cloud • Predictive analytics Immediate future • HTTP/2 • PFS impact: NPM / APM • TLS 1.3 Not-so-distant future • HTTP/3 • Kubernetes container clusters • Blue/Green deployments Copyright © 2018 Avi Networks
- 4. State of the affairs Copyright © 2018 Avi Networks
- 5. Triage Challenges • You must know an issue exists in order to find it with TCPdump • You must capture packets while the issue is occurring • The issue must be in the network segment you are capturing • You must have all the correct parameters, such as snaplen App has incorrect permissions set for an HTTP imageTap Tap
- 6. Do More with Less in a Multi-Cloud World TCPdump may be old, but does that matter? • Application proliferation as apps move from bare metal to virtual machine to containers • Network teams are asked to do more with the same number of people • Network analysis in public clouds and containers requires different tools
- 7. Technology Has Changed for the Better • Artificial Intelligence (applied Machine Learning) • Visibility / Analytics • Shift from data points to actionable information • With Avi, Analytics do not put pressure on the load balancer (service engine) because the data is being processed on the controller Troubleshoot transient issue? Side effects vs. root cause? Baseline for detection?
- 8. Technology Has Changed for the Better: Artificial Intelligence • Anomaly detection algorithms swiftly sift through data to provide intelligent insights • Automation detects and even correct problems before end users feel or report them Anomaly detection: A slow server is degrading end user experience of a virtual service
- 9. Immediate future Copyright © 2018 Avi Networks
- 10. Technology Has Changed: HTTP/2 • Multiple streams (requests) are multiplexed over a single connection • Headers are compressed with HPACK • Most browsers require HTTP/2 use modern TLS encryption • HTTP/2 has strict requirements for TLS cipher suites, preferring connections over PFS HTTP/2 connection to www.google.com
- 11. Technology Has Changed: Perfect Forward Secrecy • TCPdump and Wireshark can decrypt SSL with the private keys • Modern TLS is moving to ephemeral key exchange (PFS) – The private key is rotated, often every day – The client and server can still decrypt the connection and view clear text – Man in the middle devices, such as NPMs, aren’t able to view traffic Tap NPM
- 12. Technology Has Changed: TLS 1.3 • TLS 1.3 introduces SSL certificate encryption • Breaking “middlebox” scenarios • SNI fields will be encrypted and tunneled • MITM mitigation • Removal of RSA and Diffie-Hellman Ciphers Suites Copyright © 2018 Avi Networks
- 13. Not-so-distant future Copyright © 2018 Avi Networks
- 14. Real Game Changers: HTTP/3 • Soon HTTP / 3 (HTTP over QUIC) - UDP
- 15. Real Game Changers: Container and Kubernetes • Containers and Kubernetes require service mesh to network services • Dynamic scale of microservices • Sheer volume of microservices (adoption will increase) • Deployed in multi-cloud for no vendor lock-in • In 5 years, more applications will be written in micro services architecture
- 16. Real Game Changers: Microservices • Rapid CI/CD deployment • Blue/Green code version updates • Tracing • Ingress • East-West • Service mesh
- 17. BARE METAL VIRTUALIZED CONTAINERSON PREMISES PUBLIC CLOUDVIRTUALIZED CONTAINERS Modern, Scalable, Multi-Cloud Architecture CONTROLLER (SaaS / Customer-Managed) SERVICE ENGINE SEPARATE CONTROL & DATA PLANE ELASTICITY INTELLIGENCE AUTOMATION Copyright © 2019 Avi Networks MULTI-CLOUD
- 18. Demo
- 19. TCPdump: Tool of Last Resort • What if I still need to do TCPdump on Avi? • Available within the Avi Controller UI – Perform traffic capture on a virtual service – Traffic capture is executed on all Service Engines hosting the VS – A single PCAP file is created from traffic aggregated across all Service Engines CONTROLLER SERVICE ENGINE VS1 VS1 VS1
- 20. Network Analytics with TLS and Forward Secrecy • What if I still need to do capture traffic elsewhere on the network? • Avi Service Engines can mirror or clone traffic to NPM or network analytics tools • Cloned traffic may be clear text or re-encrypted with non-PFS TLS for wire to wire encryption • Traffic sent to public networks is encrypted with modern TLS encryption NPM
- 21. Modernize Your World • Troubleshoot faster • Do more with less… even as network and apps grow Happy birthday TCPdump… but it’s time to let go of those red balloons
- 22. Thank You! Chad Tripod, Avi Networks chad@avinetworks.com avinetworks.com Watch webinars: avinetworks.com/webinars-avi-tech-corner/ Try out Avi: Request a demo @ avinetworks.com Learn more: avinetworks.com/workshops
Editor's Notes
- #18: Next, I am going to talk about how Avi delivers these fundamental values. They need a solid foundation and 5 key building blocks. Avi Vantage Platform – 100% software defined, scalable and distributed modern architecture that best matches the new generation of applications Analytics – visibility is to see which is an important first step. The true value to businesses however comes from actionable insights that can help make better decisions Automation – again it’s more than REST APIs and Ecosystem integration. What makes automation increasingly important is that it’s the critical step to finally operationalize digital transformation – from vision to reality. Let me now go through each of these steps in details. Software defined principle – our architecture separates the control plane and data plane. It allows centralized management and policies with distributed data plane referred to as Ses. SEs can scale out and in automatically – seamless scaling based on workloads. Automation is really the invisible secret sauce that makes this possible. And to make full-stack automation, it’s key to integrate into a rich set of ecosystem with 100% clean REST APIs. SEs can be deployed across heterogeneous environment – x86 bare metal servers, in virtualized environments or along side containers in both on-prem data centers, private clouds and public clouds. Multi-cloud is important not only due to the flexibility and freedom it allows but also reduces your risk of being forced to put all your eggs in one basket. Ultimately, what’s driving all these is the intelligence Avi’s built in analytics brings. It helps your teams work smarter, your infrastructure to react to changing demands faster and enable you to make wiser decisions.