IPS Test Methodology

Rethink Intrusion Prevention System Testing

A Methodology to measure the performance, security, and stability of intrusion prevention systems
(IPS) under real-world conditions

www.breakingpoint.com
© 2005 - 2010. BreakingPoint Systems, Inc. All rights reserved. The BreakingPoint logo is a trademark of BreakingPoint Systems, Inc. 1
All other trademarks are the property of their respective owners.


Table of Contents
Introduction .................................................................................................................................................................................................................... 3

Baseline Application Performance: Maximum Connections ......................................................................................................................... 5

Baseline Application Performance: Throughput ............................................................................................................................................... 20

Baseline Attack Mitigation: SYN Flood .................................................................................................................................................................. 35

Baseline Attack Mitigation: Malicious Traffic ....................................................................................................................................................... 45

Application Traffic with SYN Flood ......................................................................................................................................................................... 55

Application Traffic with Malicious Traffic .............................................................................................................................................................. 65

Application Traffic with Malicious Traffic and SYN Flood................................................................................................................................ 76

Jumbo Frames ................................................................................................................................................................................................................ 88

IP, UDP and TCP Fuzzing ............................................................................................................................................................................................. 98

Protocol Fuzzing ............................................................................................................................................................................................................ 109

Evasion Techniques ...................................................................................................................................................................................................... 121

Negative Testing ............................................................................................................................................................................................................ 133

About BreakingPoint ................................................................................................................................................................................................... 147



Introduction
With more and more corporate data being placed on corporate networks, it is vitally important to protect that data from malicious activities.
An Intrusion Prevention System (IPS) is designed to detect malicious activities and drop or sanitize the packets while allowing legitimate
traffic to access the corporate network. Thoroughly testing IPS devices is essential to ensuring that they work properly. If the IPS device is
not working properly, malicious traffic containing viruses, worms and backdoors can easily gain access to the corporate network and cause
a great deal of problems, potentially bringing down the network.

Performing a series of measurements using the BreakingPoint Storm CTM on the IPS will help determine the actual performance, security
and stability of the IPS under real world conditions. For instance, the IPS device might be able to detect and mitigate malicious activity
when network traffic is light. However, when network traffic becomes heavy, the IPS device might detect significantly less malicious activity.
Using the BreakingPoint Storm CTM you can expose previously impossible to detect vulnerabilities in your IPS before they are exploited to
compromise your customer data, corporate assets, brand reputation and even nation security.

The test environment should emulate the actual deployment environment as closely as possible. Directly connected devices such as routers,
switches and firewalls will have an effect on packet loss, latency and data integrity. The number of advertised host IP and MAC addresses,
VLAN Tagging, and NAT will also affect the performance of an IPS.

If it is not feasible to fully recreate the deployment environment, the BreakingPoint Storm CTM should be connected directly to the IPS.

All IPS devices and builds being evaluated must use the same test environment to ensure consistent results.

Baseline Application Performance: Maximum Connections
Determine the number of connections per second that the IPS is able to handle. This will validate the performance of the IPS when
sending only good traffic with an “Allow All” policy. The TCP setup time will be analyzed to determine how a greater number of TCP
connections per second affect the time it takes to establish the TCP connection.

Baseline Application Performance: Throughput
Determine the throughput that the IPS is able to handle. This will validate the throughput performance the IPS is able to handle when
sending only good traffic with an “Allow All” policy. The overall throughput that the IPS is able to support will be determined.

Baseline Attack Mitigation Traffic: SYN Flood
Determine a baseline measurement for how the IPS performs when handling a SYN flood. Once a baseline has been established, it will
be compared with the results from the tests that blend both application and malicious traffic. The number of attempted sessions for the
SYN flood will be determined as well as the number of attempted sessions for the SYN flood that were blocked by the IPS.

Baseline Attack Mitigation Traffic: Malicious Traffic
Determine the ability of the IPS to remain stable while vulnerabilities, worms and backdoors are transmitted through it. To
perform this test, the BreakingPoint Storm CTM will be configured to use an Attack Series that includes high-risk vulnerabilities, worms and
backdoors. IPS devices have functionality that may block some of the attacks. The number of attacks blocked by the IPS will be determined
as well as the number of attacks that were able to pass through the IPS.

Application Traffic with SYN Flood
Determine a baseline measurement for how the IPS performs when handling a malicious SYN flood. Once a baseline has been
established, it will be compared with the results from the tests that blend both application and malicious traffic. The number of attempted
sessions for the SYN flood will be determined as well as the number of attempted sessions for the SYN flood that were blocked by the IPS.



Application Traffic with Malicious Traffic
Determine the ability of the IPS to remain stable while vulnerabilities, worms and backdoors are transmitted through it. To
perform this test, the BreakingPoint Storm CTM will be configured to use an Attack Series that includes high-risk vulnerabilities, worms and
backdoors.

Application Traffic with Malicious Traffic and SYN Flood
This test determines the ability of the IPS to handle application traffic, a SYN flood and malicious traffic. The results will be compared
to both the Throughput Test and the SYN Flood Test. Again, the IPS’s ability to detect and mitigate a SYN flood will be determined. Also, the
effect of the malicious traffic on the application traffic’s throughput, latency time-to-open, and time-to-close will be analyzed. Finally, the
IPS’s ability to detect and mitigate the same number of attacks as in the previous Security tests will be tested.

Jumbo Frames
This test uses the Throughput test, except the Maximum Segment Size (MMS) parameter will be increased. The maximum
transmission unit (MTU) size of the port will be verified and increased if needed. This test will determine if the IPS was able to perform
better, worse or the same when handling jumbo frames. These results will be compared to those from the Throughput Test.

IP, UDP and TCP Fuzzing
The BreakingPoint Storm CTM will be configured to use the Stack Scrambler component. This test component has the ability to
send malformed IP, UDP, TCP and Ethernet packets to the IPS. The fuzzing technique will modify parts of the packet, such as checksums
and protocol options, to generate the corrupted data. The IPS’s ability to handle malformed packets will be determined. Take notice if
the IPS crashes during the test, as this is the most important sign that the IPS is not able to appropriately handle the malformed packets.
Also, analyze the effects the malformed packets had on the application traffic and determine if the IPS’s attack detection and mitigation
capabilities were affected.

Protocol Fuzzing
This test will utilize the Security test component. This time the Security test component will fuzz application layer frames. The IPS’s ability
to handle malformed application layer frames will be determined.

Evasion Techniques
The Application Traffic with Malicious Traffic test will be used as a starting point for this test. The Security test component will have
changes made to its configuration. These changes will configure different evasion techniques that might create false negatives.

Negative Testing
The Maximum Connections test will be used as a starting point. Changes will then be made to a Super Flow. This Super Flow will then be
sent through the IPS. It will be determined how well the IPS unit was able to handle the negative testing.



Baseline Application Performance: Maximum Connections
RFC:
• RFC 793 – Transmission Control Protocol

Overview:
The specifications from the IPS data sheet will be used to determine if the IPS meets or exceeds the stated capacity. To determine the
capabilities, a Session Sender test component will be used to push the IPS beyond its stated supported limits.

Objective:
To evaluate the IPS’s ability to create and maintain sessions.

Setup:



1. Launch your favorite Web browser, and connect to the BreakingPoint Storm CTM. Click Start BreakingPoint Systems
Control Center once the page loads.

2. In the new window that appears, type your Login ID and Password. Click Login.



3. Reserve the required ports to run the test.

4. Select Control Center  Network Neighborhood.



5. Under the Network Neighborhoods heading, click the Create a new network neighborhood button.

6. In the Give the new network neighborhood a name box enter IPS Tests as the name and click OK.



7. Notice four Interface tabs are available for configuration. Only two are required for the tests. The first interface tab
should be selected; click the X to delete this interface. When prompted about removing the interface click Yes. The
remaining interfaces will be renamed. Repeat this process unitl only two interfaces are left.

8. With Interface 1 selected, configure the Network IP Address, Netmask, Gateway IP Address, Router IP Address, the
Minimum IP Address, and the Maximum IP Address. Click Apply Changes.



9. Select the Interface 2 tab. Configure the Network IP Address, Netmask and the Gateway IP Address. Using the Type
drop-down menu select Host. Finally the Minimum IP Address and the Maximum IP Address can be configured. Click
Apply Changes, then, click Save Network.

10. Now that the Network Neighborhood has been created, the test can be configured. Select Test  New Test.



11. Under the Test Quick Steps, click Select the DUT/Network.

12. In the Choose a device under test and network neighborhood window under the Device Under Test(s) section, verify
BreakingPoint Default is selected. Under Network Neighborhood(s), verify that the newly created one is selected. Click
Accept.



13. When prompted about switching Network Neighborhoods because the current setup contains more interfaces, click
Yes.

14. Under Test Quick Steps, click Add a Test Component.

15. In the Select a component type window, click Session Sender (L4).



16. Under the Information tab enter a name of Maximum Connections and click Apply Changes.

17. Select the Interfaces tab. Verify that only Interface 1 Client and Interface 2 Server are enabled.

18. Select the Parameters tab. Several parameters will be change in this section. The first parameter that needs to be
changed is the TCP Session Duration (segments) to a value of 4. Click Apply Changes.



19. Under the Data Rate section, change Minimum data rate to 90% of the total bandwidth possible, and click Apply
Changes.

20. Next under the Session Ramp Distribution tab, several parameters will be changed. First, using the Ramp Up Behavior
drop-down menu, select Full Open + Data + Close. Next, change Ramp Up Seconds to 30 and change Steady-State
Seconds to 120. Finally, change Ramp Down Duration to 30 and click Apply Changes. Scrolling may be required in
order to change some of the parameters.

21. The last parameters that need to be changed are in the Session Configuration section. The Maximum Simultaneous
Sessions should be changed to 33% of the IPS’s stated maximum. The Maximum Sessions Per Second should be
changed to 200% of the IPS’s ability. Click Apply Changes.



22. If desired, enter a description for the test under the Test Information section.

23. Verify that the Test Status has a green checkmark. If it does not have a green checkmark, click Test Status and make
the required changes.

24. Before running the test, the test component needs to be saved as a preset. This will allow for quicker and easier
configuration later. Right-click on the test component and select Save Component As Preset.

25. When prompted for a name to save the preset as, enter IPS Maximum Connections and click Save.



26. Under Test Quick Steps, click Save and Run.

27. When prompted for a name to save the test as, enter IPS Maximum Connections and click Save.

The Summary tab initially will be displayed. A great amount of information is seen on this screen from the TCP Connection Rate to the
Cumulative TCP Connections to the Bandwidth being used.



28. Select the TCP tab. This will display the TCP Connections per Second and allow the ability to determine the current
number of Attempted and Successful TCP Connection Rate. Using this view determine the maximum number of new
sessions per second open during the ramp-up phase, the maximum maintained during the steady-state phase and the
maximum opened during the steady-state phase.

29. Once the test completes, a window will appear, stating the test passed. Click Close to continue.



30. Next, select the View the report button.

31. Expand the Test Results for Maximum Connections folder, and select TCP Setup Time. The shorter the TCP setup
time, the better, as the DUT is able to quickly react and handle the incoming connection requests.

32. Next, select TCP Response Time. The shorter the response time, the better, as the DUT is able to quickly respond to
requests and continue normal operation.



33. Select TCP Close Time. The shorter the TCP Close Time the better, as the DUT is able to close out the current
connection quickly and free resources to be able to open a new connection.

34. Select Frame Latency. The shorter the frame latency, the better, as this means the frames are arriving quickly without
much delay in the network.

Other tests can also be performed. The following are some examples that can be run:

• Vary the TCP Segment size.
• Change the Distribution type to random.
• Change the TCP Session Duration (segments).
• Increase the test time for a longer test.
• If Hot Standby is going to be used, perform a test that shows how traffic is affected.



Baseline Application Performance: Throughput
RFC:
• RFC 768 – User Datagram Protocol
• RFC 791 – Internet Protocol

Overview:
A similar test setup as the previous one will be used. An Application Simulator test component will be used to generate, at maximum, 33%
of the effective session capacity of the IPS as determined in the previous test, while trying to maximize throughput.

Objective:
To evaluate the IPS’s ability to forward a wide variety of application traffic and the overall rate that it is able to do so.

Setup:




2. In the new window that appears, enter in your Login ID and Password. Click Login.




4. Select Test  New Test.

5. Under Test Quick Steps, click Select the DUT/Network.



6. In the Choose a device under test and network neighborhood window, make sure BreakingPoint Default is select under
Device Under Test(s) and IPS Tests is selected under Network Neighborhood(s). Once completed click Accept.

7. When prompted that the current test setup contains more interfaces, click Yes.




9. In the Select a component type, click Application Simulator (L7).

10. Under the Information tab enter a name of Maximum Throughput and click Apply Changes.



11. Select the Interfaces tab. Verify that Interface 1 Client is enabled and Interface 2 Server is enabled.

12. Select the Presets tab and select Enterprise Apps. Once completed, click Apply Changes.

13. Select the Parameters tab. Several parameters will need to be changed. The first parameter that needs to be changed
is in the Data Rate section. Change the Minimum data rate to 90% of the total available bandwidth, and click Apply
Changes.



14. Next under the Session Ramp Distribution section, several parameters will be changed. First, using the Ramp Up
Behavior drop-down menu, select Full Open + Data + Close. Next, change Ramp Up Seconds to 30 and change Steady-
state Seconds to 120. Finally, change Ramp Down Duration to 30 and click Apply Changes. Scrolling may be required
to change some of the parameters.

15. The next parameters that need to be changed are in the Session Configuration section. Change Maximum Simultaneous
Sessions to 33% of the session capacity of the DUT. Also, change the Maximum Sessions Per Second to 25% of the
ability of the DUT.

16. If desired, enter a description for the test under the Test Information section.

17. Verify that the Test Status has a green checkmark. If it does not have a green checkmark, click Test Status and make



18. Before running the test, the test component needs to be saved as a preset. This will allow for quicker and easier
configuration later. Right-click on the test component, and select Save Component As Preset.

19. Enter IPS Maximum Throughput as the name, and click Save.




21. When prompted to save the test, enter a name of IPS Maximum Throughput and click Save.

22. The Summary tab will initially be displayed. A great amount of information is seen on this screen: TCP Connection
Rate, Cumulative TCP Connections and Interface Bandwidth.



23. Select the TCP tab. This will display the TCP Connections per Second and allow the ability to determine the Attempted
TCP Connection Rate and Successful TCP Connection Rate.

.

24. Select the Application tab. Detailed results about each protocol may be viewed. Use the drop-down menus to select
different applications.



25. Once the test completes, a window will appear, stating the test passed. Click Close.

26. Next, select the View the report button.



27. Expand the Test Results for Maximum Throughput folder, and select Setup Time. The shorter the TCP setup time, the
better, as the DUT is able to quickly react and handle the incoming connection requests.

28. Next, select Response Time. The shorter the response time, the better, as the DUT is able to quickly respond to



29. Select TCP Close Time. The shorter the TCP close time, the better, as the DUT is able to close out the current
connection quickly and free resources to be able to open a new connection.

30. Select Frame Latency. The shorter the frame latency, the better, as this means the frames are arriving quickly without



31. Select Transmitted Frame Size. This provides a breakdown of frame sizes that were transmitted.

32. Next, expand the Detail folder and also expand the App Concurrent Flows: by protocol folder. Select the first item, App
Concurrent Flows: protocol aol, and determine how the different protocols were handles. View the entire list.

33. Repeat the previous step with App Transaction Rates: by protocol, App Response Time: by protocol, and App Failures:
by protocol. Determine how all the protocols were handled by the DUT.



34. Select Frame Data Rate and determine the maximum throughput the DUT was able to handle.

Other variations of this test can be run. The following are a few examples:

• Increase both Maximum Simultaneous Sessions and Maximum Sessions per Second by 10%, until 80% has been reached.
• Use different presets, such as the Service Provider App or a custom application profile.
• Increase the duration of the test time.



Baseline Attack Mitigation: SYN Flood
RFC:
• RFC 4987 – TCP SYN Flooding Attacks and Common Mitigations

Overview:
A SYN Flood is when a client starts a TCP connection but never sends an ACK and keeps trying to initiate TCP connections. This is harmful
to an IPS, as it has to provide resources to the TCP connection requests. The IPS likely has the ability to detect and prevent the SYN Flood. A
Session Sender test component will be used to create a SYN Flood to attack the IPS.

Objective:
To evaluate the IPS’s ability to detect and mitigate a SYN flood.

Setup:



6. In the Choose a device under test and network neighborhood window, make sure BreakingPoint Default is selected
under Device Under Test(s) and IPS Tests is selected under Network Neighborhood(s). Once completed, click Accept.

7. When prompted that the current test setup contains more interfaces click Yes.




9. In the Select a component type window click Session Sender (L4).

10. The Information tab should already be selected. Change the name of the test component to SYN Flood and click Apply
Changes.



11. Select the Parameters tab. Several parameters will be changed in this section. The first one that needs to be changed
is TCP Sessions Duration (segments) to 0. Click Apply Changes once completed.

12. In the Data Rate section, change the Minimum data rate to 10% of overall bandwidth, and click Apply Changes.

13. Next, in the Session Ramp Distribution section, use the Ramp Up Behavior drop-down menu and select SYN Only.
Change Ramp Up Seconds to 120, Steady-State Seconds to 0 and Ramp Down Seconds to 0. Scrolling down will be
required to update some of the parameters. Click Apply Changes once complete.



14. Finally, in the Session Configuration section, verify Maximum Simultaneous Sessions is set to 1,000,000. Change
Maximum Sessions Per Second to 45,000. Click Apply Changes once completed.

15. If desired, change the test Description under the Test Information section.

16. Verify that the Test Status has a green checkmark. If it does not, click Test Status and make the needed changes.

17. Before running the test the test component needs to be saved as a preset for use in later tests (saving as a preset allows
for quicker and easier configuration). Right-click on the test component, and select Save Component As Preset.



18. When prompted for a name to save the preset as, type IPS SYN Flood and click Save.

19. Finally, under Test Quick Steps, click Save and Run.

20. When prompted to save test, type IPS SYN Flood as a name.



21. Under the Summary tab it is possible to determine how the IPS is handling the SYN Flood attack. Under TCP
Connection Rate under Client, there should be a value only for Attempted. For Cumulative TCP Connections, a value
should be present only for Client Attempted. The Bandwidth for Rx should be very low, if not 0.

22. Select the TCP tab. No Successful connections should be present; this is another way of verifying that the IPS is
successfully handling the SYN Flood attack.



23. When the test finishes, a new window will appear, stating the test failed. This is expected, as no connections were
successfully made. Click Close.

24. Click the View the Report button.

25. Expand the Test Results for SYN Flood folder and select TCP Summary. Verify that Client attempted has a value and
that both Client established and Server established are 0. This means that the IPS was able to successfully handle the
SYN Flood.

Other test variations can also be run. The following are a couple of variations:

• Increase the test length for a longer SYN attack.



Baseline Attack Mitigation: Malicious Traffic
RFC:

Overview:
It is important to evaluate how malicious traffic will affect the performance of an IPS. A Security test component will be used in this test.
Five default attack series are available to use, but during this test only Security Level 1 will be used. Security Level 1 includes high-risk
vulnerabilities in services often exposed to the Internet.

Objective:
To evaluate the IPS’s ability to detect and mitigate vulnerabilities, worms and backdoors.

Setup:



6. In the Choose a device under test and network neighborhood window, make sure BreakingPoint Default is select under
Device Under Test(s) and IPS Tests is selected under Network Neighborhood(s). Once completed, click Accept.

7. When prompted that the current test setup contains more interfaces, click Yes.




9. In the Select a component type window, select the Security test component.

10. Under the Information tab, enter the name Malicious Traffic and click Apply Changes.



11. Select the Interfaces tab and verify Interface 1 Client is enabled and Interface 2 Server is enabled.

12. Select the Presets tab, and select Security Level 1. Click Apply Changes.

13. Select the Parameters tab. The defaults are all okay if repeatable strikes are required, change the RandomSeed to a
value higher than 0.





16. Before running the test, the test component needs to be saved as a preset for use in later tests (saving as a preset
allows for quicker and easier configuration). Right-click on the test component, and select Save Component As
Preset.

17. When prompted for a name to save the preset as, type IPS Malicious Traffic and click Save.



18. Finally, under Test Quick Steps, click Save and Run.

19. When prompted to save the test, type IPS Malicious Traffic as a name.

20. Select the Attacks tab. This provides a view that shows the number of blocked attacks and the number of attacks that
have been allowed to pass through the DUT.



21. When the test completes, a window will appear, stating that malicious traffic was able to pass through the DUT. Click
Close.

22. When the test completes, click the View the report button.

23. Expand the Test Results for Malicious Traffic folder and select Strike Results. Determine the number of strikes that
were allowed to pass through the DUT and the number that were blocked.



Other variations of this test can be performed. Below is a list of some of the other tests:

• Increase the test length for a longer malicious traffic attack.
• Change the Security Level.
• Use different presets, such as the Service Provider App or a custom application profile.
• Use a different random seed.



Application Traffic with SYN Flood
RFC:

Overview:
Since tests for application performance and a SYN Flood have already been configured and saved as presets, they will be used in this test.
Two test components will be used during this test, an Application Simulator and a Session Sender component.

Objective:
To combine application traffic with SYN flood traffic and compare the results against the results from the Throughput Test and the SYN
Flood Test.

Setup:




4. Use a previous test as a starting point for this test. Select Test  Open Recent Tests  IPS Maximum Throughput.

5. Before continuing with configuration of the test, click Save As.



6. When prompted for a name to save the test as, type App Traff with SYN Flood and click Save.

7. Under the Test Quick Steps, click Add a Test Component.

8. In the Select a component type window, select the Session Sender (L4).



9. The Information tab should be selected. Type the name SYN Flood and click Apply Changes.

10. Select the Presets tab, and select the IPS SYN Flood preset. Click Apply Changes once complete.






The Summary tab will be visible and provides a great deal of information about the current running test and results. The Summary
tab provides information about the application flows, TCP connections and overall bandwidth currently being utilized.



Detailed results about each protocol can be viewed under the Application tab. Use the drop down menus to display results from
different protocols.

14. Once the test completes, a new window will appear, stating that the test failed. This is expected, as the IPS should be
blocking a majority of the protocols being transmitted. Click Close to continue.



15. Select the View the report button. This will open more detailed results in a Web browser.

16. To determine the ability of the IPS to handle a SYN flood while also processing legit traffic, expand Test Results for SYN
Flood and select TCP Summary. Verify that no client was able to establish a connection and that no servers established
connections either. Once done viewing these results, for easier navigation minimize Test Results for SYN Flood.

17. Expand Test Results for Maximum Throughput and select TCP Setup Time. Again, the quicker the setup times, the
better, as the IPS is able to react and respond to the incoming request. Determine the effect the SYN flood had on the
TCP setup time of the application traffic.



18. Select TCP Response Time. Just as with TCP Setup Time, the quicker the response times, the better. Determine the
effect the SYN flood had on the TCP response time of the application traffic.

19. Next, select TCP Close Time. The quicker the IPS is able to close the TCP connection, the quicker it frees up those
resources and can use them to start a new connection. Determine the affect the SYN flood had on the TCP close time of
the application traffic.

20. Select Frame Latency, and determine how the SYN flood affects the latency of the application traffic.



21. Expand the Detail folder and also expand the App Throughput: by protocol folder. Select the first item, App

determine how each protocol was handled.
Throughput: protocol aol and determine if any traffic was able to pass through the IPS. View the entire list to

by protocol. Determine if transmitting blended traffic had an effect on any of the protocols.

23. Compare all the results collected from the current test with the baseline tests to determine any differences.

24. If any test variations were run with either the Baseline Application Perfromance: Throughput or the Baseline Attack
Mitigation: SYN Flood tests, make sure to run those variations on this test too.



Application Traffic with Malicious Traffic
RFC:

Overview:
Since tests for application performance and malicious traffic have already been configured and saved as presets, they will be used in this
test. Two test components will be used during this test, an Application Simulator and a Security component.

Objective:
To combine application traffic with malicious traffic and compare the results with the results from the security test.

Setup:




4. Use a previous test as a starting point for this test. Select Test  Open Recent Tests  IPS Maximum Throughput.

5. Before continuing with configuration of the test, click Save Test As.



6. When prompted for a name to save the test as, type App Traff Malicious Traffic and click Save.


8. In the Select a component type window, select the Security test component.



9. The Information tab should be selected. Type Malicious Traffic for the name, and click Apply Changes.

10. Select the Presets tab. Select IPS Malicious Traffic, and click Apply Changes.

11. If desired, enter a test Description under the Test Information section.



12. Verify that Test Status has a green checkmark next to it. If it does not have a green checkmark, click Test Status and
make the required changes.


tab provides information about the application flows, TCP connections and the overall bandwidth currently being utilized.



Detailed results about each protocol can be viewed under the Application tab. Use the drop-down menus to display results from

14. Select the Attacks tab. This will provide real-time information about how the IPS is performing with the malicious
traffic. As can be seen in the image below, some attacks have been allowed.



15. When the test completes, a window will appear saying the test failed. Click Close.

16. Select the View the report button. This will open up more detailed results in the browser.

17. Expand the Test results for Malicious Traffic folder and select Strike Results. Determine how well the DUT was able
to handle the different strikes and maintain blocking them while still transmitting regular traffic. Once completed,
collapse Test results for Malicious Traffic.



18. Expand the Test Results for Generic Traffic folder, and select TCP Setup Time. The quicker the IPS is able to react
and setup the TCP connection the better. Determine the effect the malicious traffic had on the TCP setup time.

19.
Next, select TCP Response Time. Again, the quicker the IPS is able to respond to the incoming connection, the better,
as the connection can be established quicker.

20. Select TCP Close Time. The ability of the IPS to quickly terminate a connection allows the IPS to quickly free those
resources.



21. Select Frame Latency, and determine the affect malicious traffic had on the overall latency.

22. Next, expand the Details folder and also expand the App Throughput: by protocol folder. Select the first item, App

Throughput: protocol aol and determine if any traffic was able to pass through the IPS. View the entire list to




24. Finally, select Frame Data Rate, and determine how the malicious traffic affects the data rate.


26. If any test variations were run with either the Baseline Application Performance Test: Throughput or the Baseline
Attack Mitigation: SYN Flood, make sure to run those variations on this test too.



Application Traffic with Malicious Traffic and SYN Flood
RFC:

Overview:
Since tests for application performance, malicious traffic and a SYN Flood have already been configured and saved as presets, they will be
used in this test. Three test components will be used during this test, an Application Simulator, a Security component and a Session Sender
component. This test will determine the ability of the IPS to handle malicious traffic while also having to deal with a SYN Flood and allowing
good traffic to pass through.

Objective:
To send a blend of application traffic with a SYN Flood and malicious traffic to the IPS and to compare the results of this test against the
results of the baseline tests.

Setup:




4. We will use a previous test as a starting point for this test. Select Test  Open Recent Tests  App Traff with
Malicious Traffic.




6. When prompted for a name to save the test as, type App Traff with Malicious Traffic and SYN Flood and click Save.


8. In the Select a component type window, select the Session Sender (L4) test component.



9. The Information tab should be selected. Type SYN Flood as the name and click Apply Changes.

10. Select the Presets tab. Locate IPS SYN Flood in the list, and click Apply Changes.

11. With the addition of the Session Sender test component, the interfaces have become oversubscribed. Select the

of the total available bandwidth, and click Apply Changes.
Maximum Throughput test component, and then select the Parameters tab. Change the Minimum data rate to 85%



12. Verify that the Test Status has a green checkmark. If not, click on Test Status and make the required changes.

13. If desired, edit the test Description under the Test Information section.

14. Under the Test Quick Steps, click Save and Run.





15. Select the Attacks tab. This provides a real-time look into how the IPS is performing with the malicious traffic. As can
be seen from the image below, some of the attacks are being allowed to pass through the IPS.



16. Once the test completes, a new window will appear, stating the test criteria failed. Click Close to continue.

17. Click the View the report button. This will open detailed results in a browser window.

18. Expand Test Results for SYN Flood and select TCP Summary. Verify that no TCP connections were established.
Collapse Test Results for SYN Flood once completed.



19. Expand Test Results for Malicious Traffic and select Strike Results. Determine how well the IPS was able to block and
not allow different strikes to pass through. Again, collapse Test Results for Malicious Traffic once completed.

20. Expand Test Results for Maximum Throughput and select TCP Setup Time. The quicker the IPS is able to react and set
up the TCP connection, the better. Determine the effect the malicious traffic had on the TCP setup time. The TCP setup
time has been affected and has increased.



21. Next, select TCP Response Time. Again, the quicker the IPS is able to respond to the incoming connection, the better
as the connection can be established quicker. Again, the time for TCP response time has increased.

22. Select TCP Close Time. The ability of the IPS to quickly terminate a connection allows the IPS to free those resources.
The TCP close time has also increased compared to the baseline tests.



23. Select Frame Latency and determine the affect malicious traffic and the SYN flood had on the overall latency.

24. Next, expand the Details folder. Also, expand the App Throughput: by protocol folder. Select the first item, App

Throughput: protocol aol, and determine if any traffic was able to pass through the IPS. View the entire list to




26. Finally, select Frame Data Rate and determine how the malicious traffic and SYN Flood affected the data rate.


28. If any test variations were run with either the Baseline Application Performance Test: Throughput, the Baseline Attack
Mitigation: Malicious Traffic or Baseline Attack Mitigation: SYN Flood, make sure to run those variations on this test
too.



Jumbo Frames
RFC
• RFC 894– A Standard for the Transmission of IP Datagrams over Ethernet

Overview:
The Throughput test will be used as a starting point in this test. Once the test is opened, the Maximum Segment size will be changed to
4,000 to send jumbo frames.

Objective:
To analyze how the IPS handles jumbo frames.

Setup:




4. We will use a previous test as a starting point for this test. Select Test  Open Recent Tests  IPS Maximum
Throughput.




6. When prompted for a name to save the test as, type IPS Jumbo Frames.

7. Select the Parameters tab and under the TCP Configuration section, change the Maximum Segment Size (MSS) to a
value greater than 1500 but less than 9142. In this example, a 4000-byte packet was used. Once the changes have been
completed, click Apply Changes.

8. Next, select Control Center  Device Status.



9. When prompted about saving the test due to changes, click Yes.

10. Right-click on a reserved port, and select Configure Port.

11. Verify that the MTU is large enough, and click Close. If needed, increase the MTU size, and click Apply. Repeat this
process for the other reserved port too.



12. To return to the test configuration, select Test  Open Recent Tests  IPS Jumbo Frames.

13. Under the Test Information section, edit the test Description.

14. Verify that the Test Status has a green checkmark. If it does not contain a green checkmark, click Test Status and make





16. Once the test completes, a new window will appear stating that the test either passed or failed. Click Close to continue.



17. Click the View the report button. This will open a Webpage containing more detailed results.

18. Expand the Test Results for Maximum Throughput folder, and select App Bytes Transmitted. This will display a byte
count that each protocol transmitted.



19. Expand the Details folder, and select TCP Setup Time. The shorter the TCP setup time, the better, as the DUT is able to
quickly handle the requests and continue operating as expected.

20. Select TCP Response Time. Again, the shorter the TCP response time, the better, as the DUT is able to quickly respond
to requests and continue operating.



21. Expand the Detail folder. Select the Frame Data Rate, and determine the maximum transmit and receive rate using the
graph and the table.

22. To determine how each protocol was handled by the IPS, five different results will be viewed. Under the Detail folder,
expand and analyze the results of the following: App Concurrent Flows: by protocol, App Throughput: by protocol, App
Transaction Rates: by protocol, App Response Time: by protocol and App Failures: by protocol.

23. Using the results from the current test and the results from the Throughput test, determine if the IPS performed better,
worse or the same when handling jumbo frames.

Other test variations can also be run. The following are some test variation examples:

• Test several different sizes of jumbo frames, specifically making sure to test the 9,000-byte frame.
• Increase the test duration.



IP, UDP and TCP Fuzzing
RFC:

Overview:
The Throughput test will be used as a starting point and a Stack Scrambler component will be used too. The Stack Scrambler tests the
integrity of different protocols by sending malformed IP, UDP, TCP and Ethernet packets to the IPS. The fuzzing technique will modify only a
single part of the packet to generate corrupt data.

Objective:
To send fuzzed traffic through the IPS and determine how it affects the IPS and other protocols.

Setup:



1. Open your favorite Web browser, and connect to the BreakingPoint Storm CTM. Once the page has loaded, click Start
BreakingPoint Systems Control Center.

2. Log in to the BreakingPoint Storm CTM by entering your Login ID and Password. Once done, click Login.



3. Once logged in, reserve the required ports to run the test.

Throughput.

5. In the lower left, click Save Test As.



6. A dialog box will appear asking for a name to save the test as. Type IPS Fuzzing and click Save.


8. From the Select a component type, choose the Stack Scrambler (Fuzzer) component.



9. Under the Information tab, change the name to IPS Fuzzer and click Apply Changes.

10. Select the Interfaces tab. Verify that only the Interface 1 Client and Interface 2 Server are enabled.

11. Select the Parameters tab. Define the percentages of traffic that will have malformed IP version, bad TCP options, Bad
Urgent Pointer and Bad IP Checksums. After each one, make sure to click Apply Changes.



12. If fuzzing through a stateful device such as an IPS unit, it is important that you set the Establish TCP Sessions
parameter to true. Otherwise, malformed TCP packets will be dropped.

13. With the addition of the Stack Scrambler, the interfaces have become oversubscribed. Select the Maximum

Data Rate section to 85% of the total available bandwidth, and click Apply Changes.
Throughput test component, and then select the Parameters tab. Change the Minimum data rate parameter in the



14. Before running the test, the test component needs to be saved as a preset for use in later tests. Saving as a preset
allows for quicker and easier configuration. Right-click on the test component, and select Save Component As Preset.

15. When prompted for a name to save the preset as, type IPS Fuzzer and click Save.

16. Verify that the Test Status has a green checkmark. If it does not, click Test Status and make the required changes.




The Summary tab will be visible and provides a great deal of information about the current running test and results. The Summary tab
provides information about the application flows, TCP connections and overall bandwidth currently being utilized.



18. When the test completes, a window will appear stating that the test failed. Click Close.

19. Next, click the View the report button. This will open detailed results in a new browser window.



20. Expand Test Results for Maximum Throughput and then expand the Details folder. Select the Frame Data Rate.
Determine how the fuzzing affected the overall data frame rate.

21. Next, expand the App Throughput: by protocol folder and select the first item, App Throughput: protocol aol.
Determine the Application data transmit and receive rate for each of the listed protocols.



22. Repeat the above process with the App transaction Rates: by protocol, App Response Time: by protocol and App
Failures: by protocol.

23. With the recently collected data, determine if the malformed packets had any effect on the application traffic. Also,
determine if the malformed packets caused any issues with the IPS, such as a crash.

24. If any variations were preformed with the Baseline Application Performance Test: Throughput, make sure to repeat
those variations with this test.



Protocol Fuzzing
RFC:

Overview:
The Application Traffic with Malicious Traffic and SYN Flood test will be used as a starting point, with the addition of the Security
component. The Security component will be used to fuzz the application level frames. This will determine if the IPS is able to handle fuzzed
application level frames and handle both malicious traffic and a SYN flood.

Objective:
To send fuzzed traffic at the application level through the IPS and determine how it affects the IPS and other protocols.

Setup:



1. Open your favorite Web browser, and connect to the BreakingPoint Storm CTM. Once the page has loaded, click Start
BreakingPoint Systems Control Center.

2. Log in to the BreakingPoint Storm CTM by entering your Login ID and Password. Once done, click Login.




Throughput.

5. In the lower left, click Save Test As.



6. A dialog box will appear, asking for a name to save the test as. Type Protocol Fuzzing and click Save.


8. From the Select a component type, select the Security component.



9. The Information tab should already be selected. Type the name Protocol Fuzzer and click Apply Changes.

10. Select the Parameters tab and set the Attack Series to BreakingPoint Protocol Fuzzers. Click Apply Changes once
completed.

11. If desired, change the test Description under Test Information.



12. Verify that the Test Status has a green checkmark. If it does not, click Test Status and make the required changes.






14. When the test completes, a window will appear stating the test failed. Click Close.




16. Expand Test Results for Protocol Fuzzer and select Strike Results. Determine the number of strikes blocked. For more
details about the strike detection, expand the Detail folder and view the different results.

17. Expand Test Results for Malicious Traffic and select Strike Results. Determine how well the IPS was able to block and
not allow different strikes to pass through. Again, collapse Test Results for Malicious Traffic once completed.



18. Expand Test Results for Maximum Throughput and select TCP Setup Time. The quicker an IPS is able to react and set
up the TCP connection, the better. Determine the effect the malicious traffic had on the TCP setup time. The TCP setup
time has been affected and has increased.

19. Next, select TCP Response Time. Again, the quicker the IPS is able to respond to the incoming connection, the better,
as the connection can be established quicker. Again, the TCP response time has increased.



The TCP close time has also increased compared to the baseline tests.

21. Select Frame Latency and determine the effect malicious traffic and the SYN flood had on the overall latency.



22. Next, expand the Details folder and also expand the App Throughput: by protocol folder. Select the first item, App

Throughput: protocol aol, and determine if any traffic was able to pass through the IPS. View the entire list to

23. Repeat the previous step with App Transaction Rates: by protocol, App Response Time: by protocol and App Failures:



24. Finally, select Frame Data Rate and determine how the malicious traffic and SYN Flood affects the data rate.


26. If any variations were performed with the Application Traffic with Malicious Traffic and SYN Flood test, make sure to
repeat those variations with this test.



Evasion Techniques
RFC:

Overview:
The Application Traffic with Malicious Traffic test will be used as a starting point in this test. The Security test component will have changes
made to parameters in the Override tab. These changes will configure evasion techniques that will attempt to be transmitted through the
IPS.

Objective:
To add evasion techniques to disguise the attacks so that they can pass through the IPS undetected.

Setup:




4. We will use a previous test as a starting point for this test. Select Test  Open Recent Tests  App Traff Malicious
Traffic.




6. When prompted for a name to save the test as, type IPS Evasion.



7. Select the Malicious Traffic test component and the Overrides tab. Different parameters can be changed in this
section, depending on the evasion techniques desired. Change the necessary parameters, and click Apply Changes.

8. If desired, edit the test Description under Test Information.

9. Verify that Test Status has a green checkmark. If it does not, click Test Status and make the required changes.




11. Select the Attacks tab. This will provide real-time information about how the IPS is performing with the malicious
traffic. As the image below shows, some attacks have been allowed.



12. When the test completes, a window will appear, saying the test failed. Click Close.

13. Select View the report button. This will open up more detailed results in the browser.



14. Expand Test results for Malicious Traffic and select Strike Results. Determine how well the DUT was able to handle
the different strikes and maintain blocking them while still transmitting regular traffic. Once completed, collapse Test
Results for Malicious Traffic.

15. Expand Test Results for Maximum Throughput, and select TCP Setup Time. The quicker the IPS is able to react and
set up the TCP connection, the better. Determine the affect the malicious traffic had on the TCP set up time.



16. Next, select TCP Response Time. Again, the quicker the IPS is able to respond to the incoming connection, the better,
as the connection can be established quicker.




18. Select Frame Latency, and determine the effect malicious traffic had on the overall latency.

19. Next, expand the Details folder and the App Throughput: by protocol folder. Select the first item, App Throughput:

protocol was handled.
protocol aol, and determine if any traffic was able to pass through the IPS. View the entire list to determine how each




21. Finally, select Frame Data Rate and determine how the malicious traffic affects the data rate.

22. With all the results collected from the current test, compare them with the baseline tests to determine any differences.

23. If any variations were preformed with the Application Traffic with Malicious Traffic test, make sure to repeat those
variations with this test.



Negative Testing
RFC:

Overview:
The Throughput test will be used as a starting point. One of the default provided Super Flows will be changed in the Application Manager.
The actions of the Super Flow either will be rearranged and/or have parameters changed. This newly created Super Flow will then be added
to a new Application Profile and then be transmitted through the IPS.

Objective:
Send a mix a negative traffic through the IPS and see how it is handled.

Setup:




2. In the new window that appears enter in your Login ID and Password. Click Login.




Throughput.



5. Before continuing with configuration of the test click Save Test As.

6. When prompted for a name to save the test as, type IPS Negative Testing.

7. Select Managers  Application Manager.



8. Select the Super Flows tab, and then locate BreakingPoint HTTP Text. Click Save As to create a copy of this Super Flow.

9. When prompted for a name to save the Super Flow as, type IPS HTTP Negative Test and click OK.



10. Under the Define Actions section, modify any of the actions by changing the action parameters or rearranging them.
Click Save Super Flow once completed. In this example, the actions were rearranged.

11. Select the App Profiles, tab and click the Create new application profile button.



12. When prompted for a new name, type IPS Negative Test.

13. Locate the newly created Super Flow, and click the Add the Super Flow to the profile button. Click Save App Profile
once completed.

14. Click the Return to previous screen button.



15. Select the Parameters tab, and locate the Application Profile parameter. Use the drop-down menu to select the newly
created application profile.




17. When the test completes, a window will appear. Click Close.




19. Expand the Test Results for Maximum Throughput folder and select TCP Setup Time. The shorter the TCP setup time,
the better, as the DUT is able to quickly react and handle the incoming connection requests.

20. Next, select TCP Response Time. The shorter the response time, the better, as the DUT is able to quickly respond to



21. Select TCP Close Time. The shorter the TCP close time, the better, as the DUT is able to close out the current
connection quickly and free resources to open a new connection.

22. Select Frame Latency. The smaller the frame latency, the better, as this means the frames are arriving quickly without



23. Select Transmitted Frame Size. This provides a breakdown of frame sizes that were transmitted.

24. Next, expand the Details folder. Also, expand the App Throughput: by protocol folder. Select the second item, App
Throughput: protocol httpadv, and determine how the different protocol was handled.



by protocol. Determine how all the httpadv was handled by the DUT.

26. Select Frame Data Rate, and determine the maximum throughput the DUT was able to handle.

If any variations were performed with the Baseline Application Performance: Throughput test, make sure to repeat those
variations with this test.



About BreakingPoint
BreakingPoint pioneered the first and only Cyber Tomography Machine Contact BreakingPoint
(CTM) to expose previously impossible-to-detect stress fractures within Learn more about BreakingPoint
cyber infrastructure components before they are exploited to compromise products and services by contacting a
customer data, corporate assets, brand reputation and even national security. representative in your area.
BreakingPoint products are the standard by which the world’s governments, 1.866.352.6691 U.S. Toll Free
enterprises, and service providers optimize the resiliency of their cyber www.breakingpoint.com
infrastructures. For more information, visit www.breakingpoint.com.
BreakingPoint Global Headquarters
BreakingPoint Storm CTM 3900 North Capital of Texas Highway
BreakingPoint has pioneered Cyber Tomography with the introduction of Austin, TX 78746
the BreakingPoint Storm CTM, enabling users to see for the first time the email: salesinfo@breakingpoint.com
virtual stress fractures lurking within their cyber infrastructure through the tel: 512.821.6000
simulation of crippling attacks, high-stress traffic load and millions of users. toll-free: 866.352.6691

BreakingPoint Storm CTM is a three-slot chassis that provides the equivalent BreakingPoint EMEA Sales Office
performance and simulation of racks and racks of servers, including: Paris, France
email: emea_sales@breakingpoint.com
• 40 Gigabits per second of blended stateful application traffic tel: + 33 6 08 40 43 93
• 30 million concurrent TCP sessions
BreakingPoint APAC Sales Office
• 1.5 million TCP sessions per second
Suite 2901, Building #5, Wanda Plaza
• 600,000+ complete TCP sessions per second No. 93 Jianguo Road
• 80,000+ SSL sessions per second Chaoyang District, Beijing, 100022, China
• 100+ stateful applications email: apac_sales@breakingpoint.com
• 4,500+ live security strikes tel: + 86 10 5960 3162

BreakingPoint Resources
Hardening cyber infrastructure is not easy work, but nothing that is this
important has ever been easy. Enterprises, service providers, government
agencies and equipment vendors are under pressure to establish a cyber
infrastructure that can not only repel attack but is resilient to application
sprawl and maximum load. BreakingPoint’s Cyber Tomography Machine
(CTM) provides the technology and solutions that allow these organizations
to create a hardened and resilient cyber infrastructure. BreakingPoint also
provides the very latest industry resources to make this process that much
easier, including Resiliency Methodologies, How-to Guides, white papers,
webcasts, and a newsletter. To learn more, visit
www.breakingpoint.com/resources.

BreakingPoint Labs Community
Join discussions on the latest developments in hardening cyber
infrastructure. BreakingPoint Labs brings together a diverse community of
people leveraging the most current insight to harden cyber infrastructure to
withstand crippling attack and high-stress application load.
Visit www.breakingpointlabs.com.


IPS Test Methodology

More Related Content

What's hot (20)

Viewers also liked (20)

Similar to IPS Test Methodology (20)

More from Ixia (11)

Recently uploaded (20)

IPS Test Methodology