Automatic test packet generation
- 1. Automatic Test Packet Generation Tushar Jadhav
- 3. Introduction
- 5. Network Troubleshooting Downtime: 25%take over an hour to resolve. (estimated $60K-110K/hour ). Current tools: Ping, Traceroute, SNMP. 70%asked for better tools, automatic tests
- 6. Automatic Test Packet Generation (ATPG) generates packets : - To test the liveness of the underlying topology . - To test the compatibility between data plane state and configuration specifications. - To test performance assertions such as packet latency.
- 7. Problem Definition Most common causes of network failure 1) Hardware failures 2) Software bugs Others failure’s include reachability failures and throughput/latency degradation. Our goal is to automatically detect these types of failures.
- 8. Goals Automatically generate test packets to test the network state, faults before being noticed by application. Augment human wisdom and intuition. Reduce the downtime. Save money.
- 10. Roadmap Reachability Analysis Test Packet Generation & Selection Fault Localization Implementation & Evaluation
- 11. Reachability Analysis Header Space Analysis All-pairs reachability : Compute all classes of packets that can flow between every pair of ports.
- 13. Test Packet Generation & Selection All-pairs reachability
- 14. Test Packet Selection Packets in all-pairs reachability table are more than necessary. Goal : Select a minimum subset of packets whose histories cover the whole rule set.
- 15. Min Set Cover
- 16. Test Packet Selection Min Set Cover -Exercise all rules -Sent out Periodically - “Redundant” - Will be used in Fault Localization
- 18. Fault Localization In ATPG: we can choose packets! Step 1: Use results from regular test packets – F (potentially broken rules) = Union from all failing packets – P (known good rules) = Union from all passing packets – Suspect Set = F –P
- 19. Step 2 : Use reserved test packets - Pick packets that test only one rule in the suspect set, and send them out for testing - Passed: eliminate - Failed: label it as “broken” Step 3: Brute force - Continue with test packets that test two or more rules in the suspect set, until the set is small enough
- 20. Implementation Putting Them all together :
- 21. Cisco/Juniper Parsers -Translate router configuration files and forwarding tables (FIB) into Header space representation. Test Packet Generation/Selection - Hassel: A python header space library - Min-Set-Cover - Python’s multiprocess module to parallelize SDN can simplify the design
- 22. ATPG for Performance Testing ATPG can also be used for detecting and localizing performance problems. Intuition: generalize results of a test from success/failure to performance (e.g. latency) Users can inject performance errors
- 23. Limitations Dynamic/Non-deterministic boxes - e.g. NAT “Invisible” rules - e.g. backup rules Transient network states Ambiguous states (work in progress) - e.g. ECMP
- 24. Conclusion Test’s the forwarding state by generating minimal link, queue, rule cover automatically. Efficient liveness testing,reachability testing. Better tool than ping & traceroute. Brings more testing and coverage of networks
- 25. References P. Kazemian, G. Varghese, and N. McKeown, “Header space analysis : Static checking for networks,” in Proc. NSDI, 2012, pp. 9–9. Y. Bejerano and R. Rastogi, “Robust monitoring of link delays and faults in IP networks,” IEEE/ACM Trans. Netw., vol. 14, no. 5, pp.1092–1103, Oct. 2006 A. Dhamdhere, R. Teixeira, C. Dovrolis, and C. Diot, “Netdiagnoser:Troubleshooting network unreachabilities using end-to-end probes and routing data,” in Proc. ACM CoNEXT, 2007, pp. 18:1–18:12.