SlideShare a Scribd company logo
Wireless Controller
Comparative Performance:
Cisco 5520 and 8540
Aruba 7210 and 7240
DR150602D
June 2015
Miercom
www.miercom.com
Cisco vs Aruba Wireless Controllers 2 DR150602D
Copyright © Miercom 2015 18 June 2015
Contents
1 - Executive Summary.............................................................................................................................3
2 - About the Products Tested ...............................................................................................................4
Cisco ................................................................................................................................................................... 4
Aruba.................................................................................................................................................................. 5
3 - Test Setup.............................................................................................................................................6
4 - Throughput ..........................................................................................................................................7
5 - User Authentication Rate...................................................................................................................9
6 – Radio Resource Management........................................................................................................11
7 - Summary.............................................................................................................................................19
8 - Independent Evaluation...................................................................................................................20
9 - About Miercom..................................................................................................................................20
10 - Use of This Report...........................................................................................................................20
Cisco vs Aruba Wireless Controllers 3 DR150602D
Copyright © Miercom 2015 18 June 2015
1 - Executive Summary
Miercom was engaged to perform independent, hands-on, comparative testing of performance and
features of mid-range and high-end Wireless Controllers from Cisco Systems and Aruba Networks.
This report summarizes the results of the Wireless Controller testing in these areas:
 Data plane: The Wireless Controller’s comparative throughput for varying packet sizes
 Control plane: The Wireless Controller’s capacity and rate for client authentication
 RF (radio frequency) spectrum management: The effect on throughput of channel-bandwidth
selection, and the extent that Cisco's Dynamic Bandwidth Selection (DBS) makes a difference.
Key Findings:
Significantly higher
throughput than Aruba
The Cisco 8540 delivers more than twice as much throughput than the
high-end Aruba 7240 Wireless Controller with small and medium packet
sizes, and with an IMIX real-world mixture of traffic packet sizes.
Cisco uses most of 40
Gbps bandwidth at all
packet sizes
Both the Cisco 8540 and the Aruba 7240 Wireless Controllers support 40
Gbps of network-connectivity bandwidth. However, with small-packet and
IMIX test traffic Aruba effectively uses only about 30 percent, while the
Cisco 8540 can fill most of this bandwidth at all packet sizes.
Cisco uses most of 20
Gbps bandwidth,
compared to Aruba
The Cisco 5520 and the Aruba 7210 both support 20 Gbps of network
bandwidth. Cisco can fill 85 to 95 percent of the 20-Gbps bandwidth, while
Aruba achieves less than 25 percent at most packet sizes.
Latest Cisco 5520
delivers much faster
client authentication rate
Testing found that the latest Cisco 5520 controller can handle 764 WiFi
client authentications (IEEE 802.1X) per second – more than three times the
rate supported by the predecessor Cisco 5508 controller.
Better TCP throughput
over all channel
bandwidths
TCP throughput performance is notably better with Cisco than Aruba for all
channel bandwidths – 50 percent better for 20-MHz channels and 116
percent better for 40-MHz channels. The highest throughput was achieved
with Cisco's Dynamic Bandwidth Selection, which Aruba doesn't support.
Miercom independently verified key performance and feature differences between the Cisco
5520 and 8540 Wireless Controllers, and comparable Wireless Controller models from Aruba
Networks. With better throughput, faster authentication and improved Radio Resource
Management (RRM), we present the Miercom Performance Verified certification to the Cisco
5520 and 8540 Wireless Controllers.
Robert Smithers
CEO
Miercom
Cisco vs Aruba Wireless Controllers 4 DR150602D
Copyright © Miercom 2015 18 June 2015
2 - About the Products Tested
The Wireless Controllers that were tested represent the current mid-range and high-end
offerings from Cisco Systems, Inc. and Aruba Networks, Inc. The specific models tested are
described in more detail below.
Cisco
The Cisco 8540 Wireless Controller is the current top-of-the-line model, designed to handle the
wireless infrastructure of medium to large enterprises, campus and Service Provider deployments.
This controller is designed to manage Cisco Aironet access points (APs). The Cisco 8540 supports
40-Gbps of bidirectional throughput via four 10GE interfaces.
The Cisco 8540, supports up to 6,000 APs and up to 64,000 clients, which can be subdivided into
4,096 VLANs. The unit ships with solid state drive and redundant power supplies.
The Cisco 5520 wireless controller addresses wireless-network control and management for
medium-sized enterprises, branches and campuses. This controller handles up to 20,000 clients
and up to 1,500 APs. The 5520 supports 20 Gbps of network-connectivity bandwidth via two
10GE interfaces. The unit ships with solid state drive and optional redundant power supplies.
Both Cisco Wireless Controllers ran latest software release version 8.1.102.0 and were tested with
Cisco Aironet 2702i APs.
Cisco 8540
Cisco 5520
Cisco vs Aruba Wireless Controllers 5 DR150602D
Copyright © Miercom 2015 18 June 2015
Aruba
The Aruba Wireless Controllers tested represent the equivalent models to the Cisco units,
including their network-connectivity bandwidth. The 7210 and 7240 are shown below: Both
models look the same from the front but support substantially different capacities. They also
support dual redundant hot-swappable power supplies.
The mid-range 7210, tested against the Cisco 5520, supports up to 512 APs and up to 16,384
clients. And while the controller comes with four 10GE interfaces (SFP+), it supports up to 20
Gbps of network-connectivity bandwidth.
The Aruba 7240 is the vendor's current high-end model, which was tested against the Cisco 8540.
Like the Cisco 8540, up to 40 Gbps of network-connectivity bandwidth is supported via the unit's
four 10GE interfaces. The 7240 supports up to 2,048 APs and up to 32,768 clients. Both
controllers were tested with Aruba AP 225 access points.
The below table summarizes the Wireless Controller models tested:
High-end, 40-Gbps
Wireless Controllers
Mid-range, 20-Gbps
Wireless Controllers
Cisco 8540 Aruba 7240 Cisco 5520 Aruba 7210
Number of clients/
users concurrently
supported
64,000 32,768 20,000 16,384
Number of APs
supported
6,000 2,048 1,500 512
Net-connectivity
bandwidth
40 Gbps 40 Gbps 20 Gbps 20 Gbps
Front view of the Aruba
7210 and 7240 (both models
look the same)
Cisco vs Aruba Wireless Controllers 6 DR150602D
Copyright © Miercom 2015 18 June 2015
3 - Test Setup
A diverse set of tests were applied to the Wireless Controllers. Subsequently, the test beds were
equally diverse and fairly complicated. Three discrete test beds were assembled for the three
areas of testing:
1. Data Plane Scale. This test bed was designed to ascertain and compare the aggregate
throughput supported by the vendors' Wireless Controller. As noted earlier, the 20-
Gbps-supporting Cisco 5520 was matched against the comparable 20-Gbps Aruba 7210,
and the 40-Gbps-supporting Cisco 8540 was pitted against the 40-Gbps Aruba 7240.
2. Control Plane Scale. This test bed was designed to ascertain the maximum user-
authentication rate of the Wireless Controller. Because custom test equipment was
employed, in the absence of any off-the-shelf test tools, and due to authentication-
protocol differences, this test could not be performed with Aruba. Instead, the user-
authentication rate of Cisco's latest 5520 was measured, and compared with the user-
authentication rate measured for the predecessor Cisco 5508 Wireless Controller, which
was tested in exactly the same manner.
3. Radio Resource Management (RRM): This test bed compared the throughputs achieved
using the different channel widths – 20-MHz, 40-MHz and 80-MHz. In addition,
throughput was measured for Cisco's Dynamic Bandwidth Selection (DBS), which applies
additional criteria provided by the APs in selecting the optimum channel width for each
AP. The Aruba equipment tested did not offer such a best or auto channel-bandwidth
selection.
All APs and controllers were configured using similar configurations (same channels, SSID, Cat-6A
cable length) with the individual vendors' best practices applied. The latest publicly available
controller codes at the time of testing were deployed: Cisco 8.1.102.0, Aruba 6.4.3.1.
Miercom recognizes IxChariot by Ixia (www.ixiacom.com) and Spirent Test Center
(www.spirent.com) as a leading test tools for simulating real-world applications for predicting
device and system performance under practical load conditions. IxChariot and Spirent were used
to accurately access the performance characteristics of any application running on wired and
wireless networks. For all the RRM Test cases, IxChariot v7.30 server was used with TCP High-
Throughput Script for each pair. The most recent available client IxChariot endpoints were
installed on respective client devices.
Each test bed is discussed in more detail in the following sections.
Cisco vs Aruba Wireless Controllers 7 DR150602D
Copyright © Miercom 2015 18 June 2015
4 - Throughput
Test Objective
To compare the throughputs supported by the Cisco and Aruba Wireless Controllers, with each
other and with their network-connectivity bandwidths.
How We Did It
We compared the throughput of two Wireless Controllers with 20 Gbps of network-connectivity
bandwidth – the Cisco 5520 and the Aruba 7210 – and then two Wireless Controllers with 40
Gbps of network-connectivity bandwidth – the Cisco 8540 and the Aruba 7240.
A Spirent test system was used to deliver traffic simulating the flows of multiple simultaneous
wireless clients. The Spirent sent the test data through a Nexus 7000 switch to the Cisco
controller being tested over multiple interfaces. The controller then sent the data back through
the switch to eight simulated APs and out to their simulated clients, which returned it through
the Cisco controller back to the Spirent test system where throughput was carefully measured.
We lacked the facility to simulate APs and clients on the Aruba controllers, as this protocol is
proprietary. Subsequently, raw throughput traffic for the given number of clients was generated
by the Spirent system and sent through the switch to the Aruba controller being tested, which
returned the data back to the Spirent system.
The Spirent system adjusted traffic levels to determine maximum throughput for a specific packet
size, before loss occurred. A more real-world mixture of packets, called IMIX, which comprised
60 percent of 64-byte packets, 25 percent of 594-byte packets and 15 percent of 1,518-byte
packets. The test bed for the throughput tests is shown below.
Source: Miercom June 2015
Cisco vs Aruba Wireless Controllers 8 DR150602D
Copyright © Miercom 2015 18 June 2015
The following charts show the results of the throughput tests.
Cisco fills available bandwidth. The Cisco 5520 can effectively utilize available 20-Gbps network
bandwidth at packet sizes from 256 to 1,374 bytes. The Aruba 7210 fills 10 to 25 percent of available
bandwidth at packet sizes up to 516 bytes.
Unaffected by number of clients. The Cisco 5520 controller proved better overall throughput with client
tests for a wide range of packet sizes.
Cisco outperforms Aruba. At small packets sizes up to 1,374 bytes as well as with the IMIX packet-size
mixture, Cisco's latest 8540 controller roundly outperforms the Aruba 7240.
Aruba throughput depends on big packets. The Aruba 7240 can make good use of its 40 Gbps of network
bandwidth only with large packet sizes. The Controller datapath CPU hits 90-95% limiting its ability to pass
more traffic.
17 17
18
19
2 3
5
12
0
5
10
15
20
256 IMIX 516 1374
Troughput(Gbps)
Packet Size (bytes)
Cisco 5520 vs Aruba 7210, 20-Gbps
Wireless Conrtollers Throughput
Cisco 55202
Aruba 7210
Theoretical
Maximum
25
30
36 38
12 12
23
36
0
10
20
30
40
256 IMIX 516 1374
Throughput(Gbps)
Packet Size (bytes)
Cisco 8540 vs Aruba 7240, 40-Gbps
Wireless Controllers Throughput
Cisco 8540
Aruba 7240
Theoretical
Maximum
Source: Miercom June 2015
Source: Miercom June 2015
Cisco 5520
Aruba 7210
Cisco vs Aruba Wireless Controllers 9 DR150602D
Copyright © Miercom 2015 18 June 2015
5 - User Authentication Rate
Test Objective
To determine the maximum rate that new WiFi client authentications can be processed.
How We Did It
Our research was unable to find any known test tools on the market for conducting high-volume
WiFi client-authentication testing. So to quantify client-authentication performance, we acquired
a custom test platform originally developed for this purpose by Cisco.
The client-authentication test package would work with Cisco wireless controllers, but not with
Aruba. Subsequently, we decided to conduct this comparative performance testing on two Cisco
models the latest Cisco 5520 controller, supporting 20 Gbps of network bandwidth, and the older
Cisco 5508 Wireless Controller, which supports 8 Gbps of network connectivity.
We decided to test most commonly used and one of the most secured authentication methods
based on IEEE 802.1X WPA2 Protected Access 2 specification. IEEE802.1X authentications like
PEAP, EAP-FAST , EAP-SIM, EAP-TLC are commonly used in the deployments like enterprises,
campuses, hotspot deployments.
This test simulates a campus spanning wireless environment, where classes change on a college
campus, or a stadium game starts or restarts, or a corporation starts up in the morning. In these
cases large numbers of users with wireless devices all want to connect, or disconnect and re-
connect to, the wireless network or connection at approximately the same time. It is important in
such environments for the WiFi system to be able to handle these requests quickly the first time.
As shown in the test bed configuration diagram on the next page, the Cisco 5520 and the older
5508 were connected through a wire speed switch to five AAA/RADIUS (Remote Authentication
Dial In User Service, a networking protocol that provides centralized Authentication,
Authorization and Accounting (AAA)).
The controllers were connected through the switch to five simulators, which were set up to
simulate 1,500 APs (300 APs/simulator), each AP with 13 simulated clients – a total of 19,500
simulated clients. The simulated clients are spread across all five of the simulators.
The authentication was done by roaming each simulated client from one AP to the next, at a rate
of 800 roams per second. This is designed to roll all the clients through all 1,500 APs. The test
was run for 5 minutes, using the 802.1X authentication.
Cisco vs Aruba Wireless Controllers 10 DR150602D
Copyright © Miercom 2015 18 June 2015
WiFi client-authentication test bed. The configuration that was used to test the controller’s
ability to handle high volumes of WiFi client authentications, is shown below.
Three times faster authentication. The results below show that the Cisco 5520 can authenticate new WiFi
clients using the 802.1X authentication 3.25 times faster than the predecessor Cisco 5508.
235
764
0
200
400
600
800
IEEE 802.1X Authentications
Authentications/Second
Authentication Rates: Cisco 5508 vs Cisco 5520
5508
5520
Source: Miercom June 2015
Source: Miercom June 2015
Cisco vs Aruba Wireless Controllers 11 DR150602D
Copyright © Miercom 2015 18 June 2015
6 – Radio Resource Management
Test Objective
To determine how aggregate throughput varies for a set of 5 APs with Cisco 5520 and Aruba
7210 environments with a diverse mix of clients, for the various channel widths and channel-
selection options offered by Cisco (RRM) and Aruba (ARM).
How We Did It
An elaborate test bed was set up with the Cisco 5520 and the Aruba 7210 side by side along with
five sets of co-located Access Points – Cisco AP 2702i and Aruba AP 225.
APs mounted in the office area for the Radio Resource Management Tests
Source: Miercom June 2015
All RRM tests were performed in an
office building dedicated for
interference-free wireless testing
where the tests can be easily
reproduced and aggregated over
multiple test runs. Five AP locations
are located around the building on
the same floor, as indicated on the
floor plan on the left. The office
space had a typical style of cubicle
areas, and open areas with each AP
neighbor about 60 to 80 feet apart.
Source: Miercom June 2015
Cisco vs Aruba Wireless Controllers 12 DR150602D
Copyright © Miercom 2015 18 June 2015
The Ixia Chariot Performance tool was used to assess the overall network performance under
realistic load conditions. The tool supported high-speed TCP connections with the chariot
endpoints installed on every client associated to the network – a total of 100 paired TCP
connections, four per wireless client during the active tests. The Ixia Chariot meticulously
measured the TCP throughput for each paired connection and provided an average throughput
number for each test run.
Clients connected to the network
The floor plan above is to scale. Each AP was connected to the five diverse clients as shown in the
chart above. Clients connected to each AP were located at distance of about 15 to 35 feet in both
line of sight and non-line of sight locations.
Total
25 Clients
5 x Macbook Pro
OS X, v10.10.10.3
802.11ac
3x3 ss
5 x Samsung
Galaxy S6
Android v5.0.2
802.11ac
2x2 2ss
5 x Apple
iPhone 6
iOS v8.3
802.11ac
1x1 1ss
5 x Dell
Latitude E6430
Windows 7, SP2
802.11n
3x3 3ss
5 x Apple
iPad Mini
iOS v8.3
802.11n
1x1 1ss
Source: Miercom June 2015
Cisco vs Aruba Wireless Controllers 13 DR150602D
Copyright © Miercom 2015 18 June 2015
Channels Available for APs
The RRM tests were evaluated for the 5GHz Channels. Available channels for both the vendors
were limited to UNII-1 (36, 40, 44, and 48) and UNII-3 (149, 153, 157, 161, and 165) channels,
allowing a total of nine 20MHz non-overlapping channels to select and bond with. The DFS
Channels (UNII-2 and UNII-2 extended) were excluded from the AP Channel list to create an
environment with an increased chance of channel overlapping especially when the five APs are
bonding on 40-MHz and 80-MHz channel widths. In addition, since Aruba ships with all DFS
channels disabled, the same configuration was mirrored on the Cisco controller to ensure an
apples-to-apples comparison. Restricting the number of channels for this test also helps to
determine at a system level how the radio manages the channels and widths when there is
limited spectrum for Wi-Fi, which again is a common occurrence in many wireless deployments
across different regulatory domains.
Selection for Available Channel Widths on the Cisco and Aruba Controllers at Time of Testing
Bonded channel widths are achieved by using multiple 20-MHz channels. By bonding two 20MHz
channels together you get a 40-MHz wide channel, and bonding four 20-MHz channels together
forms a single 80-MHz wide channel.
With the DFS Channels disabled, both Cisco and Aruba had a choice of:
 Nine 20 MHz non-overlapping channels
 Four 40 MHz non-overlapping channels
 Two 80 MHz non- overlapping channels
Conduct of RRM Tests
The RF spectrum in each case was managed by the respective Cisco and Aruba wireless
controllers, interacting with their respective access points. The tests exercised the choice of
channels, the channel-width choices, and the overlap of channels affect the overall throughput in
a typical office configuration.
Cisco and Aruba tests were run separately for each channel-width choice, with sufficient time
allowed for the configurations to settle down following each configuration change. All 25 clients
Source: Miercom June 2015
Cisco vs Aruba Wireless Controllers 14 DR150602D
Copyright © Miercom 2015 18 June 2015
were associated to the APs during these settling periods. Different volumes of test traffic were
run during the settling periods to simulate a typical office's busy daytime environment, and then
single ping streams for inactive periods.
Sequence of Bandwidth Management Testing
1. 20-MHz channel width: First the Cisco controller was set for 20-MHz channels and the network
allowed to settle for six hours while clients connected to the APs. Then the IxChariot traffic
was run to get an aggregate throughput performance for all 25 clients from a total of three
runs. Then the Aruba controller was set for 20-MHz channels and likewise settled for six hours
while clients connected to the APs. Then the IxChariot traffic was run to get an aggregate
throughput performance for all 25 Clients from a total of three runs.
2. 40-MHz channel width: Similar to the same sequence as above but channel width selected as
40-MHz.
3. 80-MHz channel width: Similar to the same sequence as above but channel width selected as
80-MHz.
4. Best (Auto channel width) selection: The Cisco controller was set for best channel-selection
mode to dynamically select the optimum channel widths for each individual AP and the
wireless network allowed to settle for six hours while clients connected. Then the IxChariot
traffic was run to get an aggregate throughput performance for all 25 Clients from a total of
three runs. Aruba did not offer a comparable auto channel-selection setting for their ARM
configuration.
RRM Channel Selection Results
The channel selection assignments for these tests are summarized in the below table:
AP Channel Selection Plan
Channel
Width
Cisco Aruba
AP-1 AP-2 AP-3 AP-4 AP-5 AP-1 AP-2 AP-3 AP-4 AP-5
20 MHz 40 149 48 165 36 149 161 48 165 161
40 MHz 36-40 149-153 48-44 161-157 36-40 36-40 36-40 149-153 149-153 40-36
80 MHz
161-157-
149-153
48-44-
36-40
48-44-
36-40
161-157-
149-153
48-44-
36-40
36-40-
44-48
48-44-
40-36
157-161-
149-153
153-149-
157-161
161-157-
153-149
Auto 165 149-153 48-44 161-157 36-40 n/a n/a n/a n/a n/a
In our testing we would change the AP configurations and then let them settle on the channels to
be used for their 5 clients for ~6+ hours. It was observed that the Aruba’s ARM would tend to
settle with more overlapping channels than Cisco’s RRM in almost all test cases. Aruba’s situation
would improve if the controller and APs were rebooted before starting each test. This behavior
was not present on the Cisco Infrastructure, where Cisco’s RRM would quickly configure to the
selected channel widths with least overlapping channels within the first hour itself. In the test
cases, where Aruba ARM’s channel selection had heavy overlapping, the controller was rebooted
and allowed to settle for another 6 hour period before the performance tests were run.
Cisco vs Aruba Wireless Controllers 15 DR150602D
Copyright © Miercom 2015 18 June 2015
Channel Overlapping
Comparing the final channel selections for both the vendors, it was observed that Cisco 5520 did
a better job of assessing the environment and providing a better RRM configuration by selecting
least overlapping channels in all test cases, thus minimizing the problem of co-channel
interference. With less co-channel interference, the APs would have a better shot in delivering the
traffic to the wireless clients and directly improving the overall system performance. The Aruba
7210 tended to select channels for the five APs that overlapped more than Cisco. Also, as noted,
the Aruba controller did better, in terms of avoiding overlapping channels, if the controller was
rebooted before starting a new test.
Shown below are the visual channel selections by each controller for 20 and 40 MHz (The 80-
MHz option is left out for brevity).
The x-axis shows the channel numbers, and the y-axis shows the signal strength in decibels: from
0 dB (the strongest) to -90 dBm. Each mountain slope represents an AP.
20-MHz Channel Selection Visual Representation
20-MHZ CHANNEL WIDTH SELECTION TEST
VENDOR NUMBER OF OVERLAPPING APS CHANNELS OVERLAP PERCENTAGE
CISCO 0 0%
ARUBA 2 40%
CISCO RRM 20-MHz
ARUBA ARM 20-MHz
Source: Miercom June 2015
Source: Miercom June 2015
Cisco vs Aruba Wireless Controllers 16 DR150602D
Copyright © Miercom 2015 18 June 2015
Wi-Fi is a shared medium. When APs select an overlapping channel, the APs and clients have to
share the airtime on that overlapping channel. This means that when one AP is transmitting data
to the client, the other APs on the same channel will have to pause their traffic communication
and wait till the first AP has finished the transmission. This goes in a round-robin fashion with
each AP getting an airtime to send data to the respective clients. More the overlapping, more the
waiting period for the APs. As the overlapping channel gets more crowded, the APs tend to have
higher retries for a successful data transmission.
With heavy overlapping of channels as seen in Aruba’s 40-MHz channel selection below, the
overall performance of the network tends to reduce as the co-channel interference and the
number of retransmissions goes up.
40-MHz Channel Selection Visual Representation
40-MHZ CHANNEL WIDTH SELECTION TEST
VENDOR NUMBER OF OVERLAPPING APS CHANNELS OVERLAP PERCENTAGE
CISCO 2 40%
ARUBA 5 100%
CISCO RRM 40-MHz
ARUBA ARM 40-MHz
Source: Miercom June 2015
Source: Miercom June 2015
Cisco vs Aruba Wireless Controllers 17 DR150602D
Copyright © Miercom 2015 18 June 2015
For the 40-MHz tests, channel overlapping was observed on both the vendors but varied in
comparison. Cisco chose to overlap 2 of the 5 APs, while in Aruba's case all the 5 APs ended up
sharing channels amongst each other.
For the 80-MHz channel width tests, as there were only two channels to select from, both Cisco
and Aruba had no choice but overlap the APs on shared channels. The AP distribution was similar
for both vendors (3 on one channel, 2 on other channel) but the difference was in the overall
power levels. Cisco APs had lowered the power levels more than Aruba APs to create smaller cell
sizes in order to compensate for the heavy overlapping. This was determined as one of the
reasons why even with the heavy overlapping for the 80-MHz channels, Cisco APs performed
slightly better in terms of overall system throughput as seen in the performance comparison
chart below.
80 MHZ CHANNEL WIDTH SELECTION TEST
VENDOR NUMBER OF OVERLAPPING APS CHANNELS OVERLAP PERCENTAGE
CISCO 5 100%
ARUBA 5 100%
Cisco’s Dynamic Bandwidth Channel Selection is based on the Auto channel-width select feature.
As shown, four 40-MHz channels and one 20-MHz channel were used by the five APs, with
minimal overlap. The result: less interference and better throughput.
Cisco’s Auto Channel-Width Selection Visual Representation
AUTO CHANNEL WIDTH SELECTION TEST
VENDOR NUMBER OF OVERLAPPING APS CHANNELS OVERLAP PERCENTAGE
CISCO 0 0%
ARUBA N/A N/A
Source: Miercom June 2015
CISCO RRM DBS AUTO
Cisco vs Aruba Wireless Controllers 18 DR150602D
Copyright © Miercom 2015 18 June 2015
We observed that the Aruba controller would tend to settle with more overlapping channels than
the Cisco controller. All performance tests were run three times. The below graph shows the
average performance of the three test runs for each channel width test case.
Aruba offered no similar option for "auto" or "best" channel-bandwidth selection.
As noted, Cisco offers a best channel-width selection option, where the controller selects the best
channel width for each AP, based on the AP’s neighbor information, which each AP sends to the
controller. We found that this option, called Dynamic Bandwidth Selection (DBS) yielded the best
throughput in our test environment. With DBS, the controller selects the channel bandwidth for each
AP – based on more reported frequency and interference data – rather than the operator, who
selects the channel width on a more-or-less "one-channel-size-fits-all" basis.
452
649
455
730
304 308 343
0
100
200
300
400
500
600
700
800
20 MHz 40 MHz 80 MHz Auto
Throughput(Mbps)
Aggregate TCP Performance
Across 5 APs with 25 Total Clients
Cisco
Aruba
Source: Miercom June 2015
Cisco vs Aruba Wireless Controllers 19 DR150602D
Copyright © Miercom 2015 18 June 2015
7 - Summary
Cisco 5520 and 8540 Wireless Controllers have successfully proved superior performance through
testing to offer higher data plane throughput of 20 Gbps and 40 Gbps respectively and improved
control plane performance for higher scale deployments.
Cisco's RRM and Dynamic Bandwidth Selection together exhibited the most ideal channel &
channel-width selection, resulting in the highest performing RF configuration among all the RF
performance tests. Real-world testing proved that Cisco's RRM with DBS enhances the spectrum
management for any typical wireless environment, thus earning the Miercom Performance
Verified Certification.
Cisco vs Aruba Wireless Controllers 20 DR150602D
Copyright © Miercom 2015 18 June 2015
8 - Independent Evaluation
This report was sponsored by Cisco Systems, Inc. The data was obtained completely and
independently as part of Miercom's competitive analysis.
9 - About Miercom
Miercom has published hundreds of network-product-comparison analyses – many made public,
appearing in leading trade periodicals and other publications, and many confidential, for internal
use only. Miercom’s reputation as the leading, independent product test center is undisputed.
Private test services available from Miercom include competitive product analyses, as well as
individual product evaluations. Miercom test methodologies are generally developed
collaboratively with the client, and feature comprehensive certification and test programs
including: Certified Interoperable, Certified Reliable, Certified Secure and Certified Green.
Products may also be evaluated under the Performance Verified program, the industry’s most
thorough and trusted assessment for product usability and performance.
10 - Use of This Report
Every effort was made to ensure the accuracy of the data in this report. However, errors and/or
oversights can nevertheless occur. The information documented in this report may depend on
various test tools, the accuracy of which is beyond our control. Furthermore, the document may
rely on certain representations by the vendors that were reasonably verified by Miercom, but are
beyond our control to verify with 100-percent certainty.
This document is provided “as is” by Miercom, which gives no warranty, representation or
undertaking, whether express or implied, and accepts no legal responsibility, whether direct or
indirect, for the accuracy, completeness, usefulness or suitability of any information contained
herein. Miercom is not liable for damages arising out of or related to the information contained
in this report.
No part of any document may be reproduced, in whole or in part, without the specific written
permission of Miercom or Cisco Systems, Inc. All trademarks used in the document are owned by
their respective owners. You agree not to use any trademark in or as the whole or part of your
own trademarks in connection with any activities, products or services, which are not yours. You
also agree not to use any trademarks in a manner which may be confusing, misleading or
deceptive or in a manner that disparages Miercom or its information, projects or developments.

More Related Content

PDF
PLNOG 8: Przemysław Dziel - NG - PON - Lights The Future Way for Broadband
PPT
Core 2 processors
PPTX
Five building blocks of digital transformation
PDF
OTT Distribution (Over The Top)
PDF
Austere Systems-IT Staff Augmentation- Recruitment services
PDF
PFE Swap INWI 2G 3G LTE
DOCX
Mobile processors
PPTX
IT Staff Augmentation Services
PLNOG 8: Przemysław Dziel - NG - PON - Lights The Future Way for Broadband
Core 2 processors
Five building blocks of digital transformation
OTT Distribution (Over The Top)
Austere Systems-IT Staff Augmentation- Recruitment services
PFE Swap INWI 2G 3G LTE
Mobile processors
IT Staff Augmentation Services

What's hot (20)

PDF
Grayscale Metaverse Report November 2021
PDF
Enabling Multi-access Edge Computing (MEC) Platform-as-a-Service for Enterprises
PDF
Singapore 3D Map for a Smart Nation
PPTX
Sécurité d’une plateforme VoIP Open Source « Elastix »
PPTX
Technology Enhanced Learning Futuristic Approach.pptx
PDF
Data Driven Assessment.pdf
PDF
Manifeste pour l'internet physique fr version 1.11.1 2012-11-28
PDF
Fintech Strategic Roadmap for the UK’s Largest Banks: our MIT Fintech course ...
PDF
Les Séniors et l'achat sur Internet : des freins générationnels?
PPTX
Trunk IAX et Conférence sur Asterisk
PPSX
CCTV Installation Plan - CEO Residence
PPTX
Mavenir: Monetizing RCS through Innovation on Cloud Native Network​
PPTX
Future of mobile technology
PPTX
APIdays Open Banking & Fintech: Workshop - Financial Services Use Cases for APIs
PPT
Sécurité asterisk web
PDF
apidays LIVE Singapore - Open Banking: A foundation for the new world by Bhar...
PDF
05c reseaux-sans-fil
PPTX
Réseau de capteurs sans fils wsn
PPTX
Accenture Media & Entertainment Industry 2021 - The Aggregator Value Play
PDF
Comment réaliser une soudure fibre optique dans les règles de l'art ?
Grayscale Metaverse Report November 2021
Enabling Multi-access Edge Computing (MEC) Platform-as-a-Service for Enterprises
Singapore 3D Map for a Smart Nation
Sécurité d’une plateforme VoIP Open Source « Elastix »
Technology Enhanced Learning Futuristic Approach.pptx
Data Driven Assessment.pdf
Manifeste pour l'internet physique fr version 1.11.1 2012-11-28
Fintech Strategic Roadmap for the UK’s Largest Banks: our MIT Fintech course ...
Les Séniors et l'achat sur Internet : des freins générationnels?
Trunk IAX et Conférence sur Asterisk
CCTV Installation Plan - CEO Residence
Mavenir: Monetizing RCS through Innovation on Cloud Native Network​
Future of mobile technology
APIdays Open Banking & Fintech: Workshop - Financial Services Use Cases for APIs
Sécurité asterisk web
apidays LIVE Singapore - Open Banking: A foundation for the new world by Bhar...
05c reseaux-sans-fil
Réseau de capteurs sans fils wsn
Accenture Media & Entertainment Industry 2021 - The Aggregator Value Play
Comment réaliser une soudure fibre optique dans les règles de l'art ?
Ad

Viewers also liked (20)

PDF
Wireless LAN Security, Policy, and Deployment Best Practices
PDF
Cisco and illinois school district make learning fun case study
PPTX
Beyond BYOD: Uncompromised Experience for Any Workspace
PPTX
iPads on Your Network? Real, Secure Mobile Solutions
PDF
Katy Independent School District Makes a Difference with Cisco Mobility: Case...
PDF
Punahou school maintains a secure and open campus with cisco - case study
PPTX
Beyond BYOD
PDF
Managing the Mobile Device Wave for Enterpise Wireless Networks: Best Practices
PDF
Secure Mobility in Cisco Unified WLAN Networks for Mobile Devices
PDF
Understanding CleanAir Technology to improve enterprise WLAN spectrum management
PDF
IT Guide for Mobility: Making the case for Security leaders
PDF
Conquering the 802.11ac Shift
PDF
Internet of Everything Case Study: Punahou School
PDF
Mobilize employees with the cisco mobile workspace solution
PDF
Preparing Your Network for Wave 2 of 802.11ac
PDF
Technical overview of new cisco catalyst multigigabit switches
PDF
Wireless Branch Office Network Architecture
PDF
Hyatt Regency Transforms Guest Experiences and Bottom Line with Cisco Wi-Fi a...
PDF
Design and Deployment of Outdoor Mesh Wireless Networks
PDF
Deploying the Cisco Mobility Services Engine for Advanced Wireless Services
Wireless LAN Security, Policy, and Deployment Best Practices
Cisco and illinois school district make learning fun case study
Beyond BYOD: Uncompromised Experience for Any Workspace
iPads on Your Network? Real, Secure Mobile Solutions
Katy Independent School District Makes a Difference with Cisco Mobility: Case...
Punahou school maintains a secure and open campus with cisco - case study
Beyond BYOD
Managing the Mobile Device Wave for Enterpise Wireless Networks: Best Practices
Secure Mobility in Cisco Unified WLAN Networks for Mobile Devices
Understanding CleanAir Technology to improve enterprise WLAN spectrum management
IT Guide for Mobility: Making the case for Security leaders
Conquering the 802.11ac Shift
Internet of Everything Case Study: Punahou School
Mobilize employees with the cisco mobile workspace solution
Preparing Your Network for Wave 2 of 802.11ac
Technical overview of new cisco catalyst multigigabit switches
Wireless Branch Office Network Architecture
Hyatt Regency Transforms Guest Experiences and Bottom Line with Cisco Wi-Fi a...
Design and Deployment of Outdoor Mesh Wireless Networks
Deploying the Cisco Mobility Services Engine for Advanced Wireless Services
Ad

Similar to Wireless Controller Comparative Performance Cisco vs Aruba Miercom Report (20)

DOCX
Mobility express even better than before
PDF
2500 controller
PDF
Migrating to the 7200 controller george anderson marcus christensen
PDF
Cisco Unified Wireless Network and Converged access – Design session
PDF
Unified Access Update - 11AC and Switching Platform
PPTX
Cisco AIRACCRWM1400
PDF
cisco-air-cap2702i-n-k9-datasheet.pdf
PPS
Iuwne10 S02 L08
PPTX
Innovations in Mobility
DOCX
New cisco aironet 1850 series access points focus on wave 2 wifi
PDF
cisco-air-ap2802i-g-k9c-datasheet.pdf
PDF
cisco-air-cap2702e-n-k9-datasheet.pdf
PPTX
Cisco aironet 1140 access point overview
PDF
cisco-air-ap2802i-c-k9c-datasheet.pdf
PDF
cisco-air-ap2802i-i-k9c-datasheet.pdf
PDF
cisco-air-cap2702i-d-k9-datasheet.pdf
PDF
Wireless lan controller
PDF
Outdoor ap datasheet c78-737416
PDF
cisco-air-cap2702i-r-k9-datasheet.pdf
PDF
Car net testresults_may2015_presentation_final
Mobility express even better than before
2500 controller
Migrating to the 7200 controller george anderson marcus christensen
Cisco Unified Wireless Network and Converged access – Design session
Unified Access Update - 11AC and Switching Platform
Cisco AIRACCRWM1400
cisco-air-cap2702i-n-k9-datasheet.pdf
Iuwne10 S02 L08
Innovations in Mobility
New cisco aironet 1850 series access points focus on wave 2 wifi
cisco-air-ap2802i-g-k9c-datasheet.pdf
cisco-air-cap2702e-n-k9-datasheet.pdf
Cisco aironet 1140 access point overview
cisco-air-ap2802i-c-k9c-datasheet.pdf
cisco-air-ap2802i-i-k9c-datasheet.pdf
cisco-air-cap2702i-d-k9-datasheet.pdf
Wireless lan controller
Outdoor ap datasheet c78-737416
cisco-air-cap2702i-r-k9-datasheet.pdf
Car net testresults_may2015_presentation_final

More from Cisco Mobility (7)

PDF
Troubleshooting Wireless LANs with Centralized Controllers
PDF
Understanding RF Fundamentals and the Radio Design of Wireless Networks
PDF
Managing an Enterprise WLAN with Cisco Prime NCS & WCS
PDF
Design and Deployment of Enterprise Wirlesss Networks
PDF
Wireless LAN Design and Deployment of Rich Media Networks
PPTX
Cisco OfficeExtend: Secure, High-Performance Wireless for Teleworkers
PPTX
iPads on your network? Take Control with Unified Policy and Management
Troubleshooting Wireless LANs with Centralized Controllers
Understanding RF Fundamentals and the Radio Design of Wireless Networks
Managing an Enterprise WLAN with Cisco Prime NCS & WCS
Design and Deployment of Enterprise Wirlesss Networks
Wireless LAN Design and Deployment of Rich Media Networks
Cisco OfficeExtend: Secure, High-Performance Wireless for Teleworkers
iPads on your network? Take Control with Unified Policy and Management

Recently uploaded (20)

PDF
Encapsulation_ Review paper, used for researhc scholars
PDF
NewMind AI Monthly Chronicles - July 2025
PDF
cuic standard and advanced reporting.pdf
PDF
Mobile App Security Testing_ A Comprehensive Guide.pdf
PDF
Dropbox Q2 2025 Financial Results & Investor Presentation
PDF
NewMind AI Weekly Chronicles - August'25 Week I
PPTX
Effective Security Operations Center (SOC) A Modern, Strategic, and Threat-In...
PPTX
20250228 LYD VKU AI Blended-Learning.pptx
PDF
Reach Out and Touch Someone: Haptics and Empathic Computing
PDF
Machine learning based COVID-19 study performance prediction
PDF
Approach and Philosophy of On baking technology
PDF
CIFDAQ's Market Insight: SEC Turns Pro Crypto
DOCX
The AUB Centre for AI in Media Proposal.docx
PPTX
Understanding_Digital_Forensics_Presentation.pptx
PDF
Electronic commerce courselecture one. Pdf
PPTX
PA Analog/Digital System: The Backbone of Modern Surveillance and Communication
PDF
Unlocking AI with Model Context Protocol (MCP)
PPTX
Big Data Technologies - Introduction.pptx
PPTX
VMware vSphere Foundation How to Sell Presentation-Ver1.4-2-14-2024.pptx
PPT
Teaching material agriculture food technology
Encapsulation_ Review paper, used for researhc scholars
NewMind AI Monthly Chronicles - July 2025
cuic standard and advanced reporting.pdf
Mobile App Security Testing_ A Comprehensive Guide.pdf
Dropbox Q2 2025 Financial Results & Investor Presentation
NewMind AI Weekly Chronicles - August'25 Week I
Effective Security Operations Center (SOC) A Modern, Strategic, and Threat-In...
20250228 LYD VKU AI Blended-Learning.pptx
Reach Out and Touch Someone: Haptics and Empathic Computing
Machine learning based COVID-19 study performance prediction
Approach and Philosophy of On baking technology
CIFDAQ's Market Insight: SEC Turns Pro Crypto
The AUB Centre for AI in Media Proposal.docx
Understanding_Digital_Forensics_Presentation.pptx
Electronic commerce courselecture one. Pdf
PA Analog/Digital System: The Backbone of Modern Surveillance and Communication
Unlocking AI with Model Context Protocol (MCP)
Big Data Technologies - Introduction.pptx
VMware vSphere Foundation How to Sell Presentation-Ver1.4-2-14-2024.pptx
Teaching material agriculture food technology

Wireless Controller Comparative Performance Cisco vs Aruba Miercom Report

  • 1. Wireless Controller Comparative Performance: Cisco 5520 and 8540 Aruba 7210 and 7240 DR150602D June 2015 Miercom www.miercom.com
  • 2. Cisco vs Aruba Wireless Controllers 2 DR150602D Copyright © Miercom 2015 18 June 2015 Contents 1 - Executive Summary.............................................................................................................................3 2 - About the Products Tested ...............................................................................................................4 Cisco ................................................................................................................................................................... 4 Aruba.................................................................................................................................................................. 5 3 - Test Setup.............................................................................................................................................6 4 - Throughput ..........................................................................................................................................7 5 - User Authentication Rate...................................................................................................................9 6 – Radio Resource Management........................................................................................................11 7 - Summary.............................................................................................................................................19 8 - Independent Evaluation...................................................................................................................20 9 - About Miercom..................................................................................................................................20 10 - Use of This Report...........................................................................................................................20
  • 3. Cisco vs Aruba Wireless Controllers 3 DR150602D Copyright © Miercom 2015 18 June 2015 1 - Executive Summary Miercom was engaged to perform independent, hands-on, comparative testing of performance and features of mid-range and high-end Wireless Controllers from Cisco Systems and Aruba Networks. This report summarizes the results of the Wireless Controller testing in these areas:  Data plane: The Wireless Controller’s comparative throughput for varying packet sizes  Control plane: The Wireless Controller’s capacity and rate for client authentication  RF (radio frequency) spectrum management: The effect on throughput of channel-bandwidth selection, and the extent that Cisco's Dynamic Bandwidth Selection (DBS) makes a difference. Key Findings: Significantly higher throughput than Aruba The Cisco 8540 delivers more than twice as much throughput than the high-end Aruba 7240 Wireless Controller with small and medium packet sizes, and with an IMIX real-world mixture of traffic packet sizes. Cisco uses most of 40 Gbps bandwidth at all packet sizes Both the Cisco 8540 and the Aruba 7240 Wireless Controllers support 40 Gbps of network-connectivity bandwidth. However, with small-packet and IMIX test traffic Aruba effectively uses only about 30 percent, while the Cisco 8540 can fill most of this bandwidth at all packet sizes. Cisco uses most of 20 Gbps bandwidth, compared to Aruba The Cisco 5520 and the Aruba 7210 both support 20 Gbps of network bandwidth. Cisco can fill 85 to 95 percent of the 20-Gbps bandwidth, while Aruba achieves less than 25 percent at most packet sizes. Latest Cisco 5520 delivers much faster client authentication rate Testing found that the latest Cisco 5520 controller can handle 764 WiFi client authentications (IEEE 802.1X) per second – more than three times the rate supported by the predecessor Cisco 5508 controller. Better TCP throughput over all channel bandwidths TCP throughput performance is notably better with Cisco than Aruba for all channel bandwidths – 50 percent better for 20-MHz channels and 116 percent better for 40-MHz channels. The highest throughput was achieved with Cisco's Dynamic Bandwidth Selection, which Aruba doesn't support. Miercom independently verified key performance and feature differences between the Cisco 5520 and 8540 Wireless Controllers, and comparable Wireless Controller models from Aruba Networks. With better throughput, faster authentication and improved Radio Resource Management (RRM), we present the Miercom Performance Verified certification to the Cisco 5520 and 8540 Wireless Controllers. Robert Smithers CEO Miercom
  • 4. Cisco vs Aruba Wireless Controllers 4 DR150602D Copyright © Miercom 2015 18 June 2015 2 - About the Products Tested The Wireless Controllers that were tested represent the current mid-range and high-end offerings from Cisco Systems, Inc. and Aruba Networks, Inc. The specific models tested are described in more detail below. Cisco The Cisco 8540 Wireless Controller is the current top-of-the-line model, designed to handle the wireless infrastructure of medium to large enterprises, campus and Service Provider deployments. This controller is designed to manage Cisco Aironet access points (APs). The Cisco 8540 supports 40-Gbps of bidirectional throughput via four 10GE interfaces. The Cisco 8540, supports up to 6,000 APs and up to 64,000 clients, which can be subdivided into 4,096 VLANs. The unit ships with solid state drive and redundant power supplies. The Cisco 5520 wireless controller addresses wireless-network control and management for medium-sized enterprises, branches and campuses. This controller handles up to 20,000 clients and up to 1,500 APs. The 5520 supports 20 Gbps of network-connectivity bandwidth via two 10GE interfaces. The unit ships with solid state drive and optional redundant power supplies. Both Cisco Wireless Controllers ran latest software release version 8.1.102.0 and were tested with Cisco Aironet 2702i APs. Cisco 8540 Cisco 5520
  • 5. Cisco vs Aruba Wireless Controllers 5 DR150602D Copyright © Miercom 2015 18 June 2015 Aruba The Aruba Wireless Controllers tested represent the equivalent models to the Cisco units, including their network-connectivity bandwidth. The 7210 and 7240 are shown below: Both models look the same from the front but support substantially different capacities. They also support dual redundant hot-swappable power supplies. The mid-range 7210, tested against the Cisco 5520, supports up to 512 APs and up to 16,384 clients. And while the controller comes with four 10GE interfaces (SFP+), it supports up to 20 Gbps of network-connectivity bandwidth. The Aruba 7240 is the vendor's current high-end model, which was tested against the Cisco 8540. Like the Cisco 8540, up to 40 Gbps of network-connectivity bandwidth is supported via the unit's four 10GE interfaces. The 7240 supports up to 2,048 APs and up to 32,768 clients. Both controllers were tested with Aruba AP 225 access points. The below table summarizes the Wireless Controller models tested: High-end, 40-Gbps Wireless Controllers Mid-range, 20-Gbps Wireless Controllers Cisco 8540 Aruba 7240 Cisco 5520 Aruba 7210 Number of clients/ users concurrently supported 64,000 32,768 20,000 16,384 Number of APs supported 6,000 2,048 1,500 512 Net-connectivity bandwidth 40 Gbps 40 Gbps 20 Gbps 20 Gbps Front view of the Aruba 7210 and 7240 (both models look the same)
  • 6. Cisco vs Aruba Wireless Controllers 6 DR150602D Copyright © Miercom 2015 18 June 2015 3 - Test Setup A diverse set of tests were applied to the Wireless Controllers. Subsequently, the test beds were equally diverse and fairly complicated. Three discrete test beds were assembled for the three areas of testing: 1. Data Plane Scale. This test bed was designed to ascertain and compare the aggregate throughput supported by the vendors' Wireless Controller. As noted earlier, the 20- Gbps-supporting Cisco 5520 was matched against the comparable 20-Gbps Aruba 7210, and the 40-Gbps-supporting Cisco 8540 was pitted against the 40-Gbps Aruba 7240. 2. Control Plane Scale. This test bed was designed to ascertain the maximum user- authentication rate of the Wireless Controller. Because custom test equipment was employed, in the absence of any off-the-shelf test tools, and due to authentication- protocol differences, this test could not be performed with Aruba. Instead, the user- authentication rate of Cisco's latest 5520 was measured, and compared with the user- authentication rate measured for the predecessor Cisco 5508 Wireless Controller, which was tested in exactly the same manner. 3. Radio Resource Management (RRM): This test bed compared the throughputs achieved using the different channel widths – 20-MHz, 40-MHz and 80-MHz. In addition, throughput was measured for Cisco's Dynamic Bandwidth Selection (DBS), which applies additional criteria provided by the APs in selecting the optimum channel width for each AP. The Aruba equipment tested did not offer such a best or auto channel-bandwidth selection. All APs and controllers were configured using similar configurations (same channels, SSID, Cat-6A cable length) with the individual vendors' best practices applied. The latest publicly available controller codes at the time of testing were deployed: Cisco 8.1.102.0, Aruba 6.4.3.1. Miercom recognizes IxChariot by Ixia (www.ixiacom.com) and Spirent Test Center (www.spirent.com) as a leading test tools for simulating real-world applications for predicting device and system performance under practical load conditions. IxChariot and Spirent were used to accurately access the performance characteristics of any application running on wired and wireless networks. For all the RRM Test cases, IxChariot v7.30 server was used with TCP High- Throughput Script for each pair. The most recent available client IxChariot endpoints were installed on respective client devices. Each test bed is discussed in more detail in the following sections.
  • 7. Cisco vs Aruba Wireless Controllers 7 DR150602D Copyright © Miercom 2015 18 June 2015 4 - Throughput Test Objective To compare the throughputs supported by the Cisco and Aruba Wireless Controllers, with each other and with their network-connectivity bandwidths. How We Did It We compared the throughput of two Wireless Controllers with 20 Gbps of network-connectivity bandwidth – the Cisco 5520 and the Aruba 7210 – and then two Wireless Controllers with 40 Gbps of network-connectivity bandwidth – the Cisco 8540 and the Aruba 7240. A Spirent test system was used to deliver traffic simulating the flows of multiple simultaneous wireless clients. The Spirent sent the test data through a Nexus 7000 switch to the Cisco controller being tested over multiple interfaces. The controller then sent the data back through the switch to eight simulated APs and out to their simulated clients, which returned it through the Cisco controller back to the Spirent test system where throughput was carefully measured. We lacked the facility to simulate APs and clients on the Aruba controllers, as this protocol is proprietary. Subsequently, raw throughput traffic for the given number of clients was generated by the Spirent system and sent through the switch to the Aruba controller being tested, which returned the data back to the Spirent system. The Spirent system adjusted traffic levels to determine maximum throughput for a specific packet size, before loss occurred. A more real-world mixture of packets, called IMIX, which comprised 60 percent of 64-byte packets, 25 percent of 594-byte packets and 15 percent of 1,518-byte packets. The test bed for the throughput tests is shown below. Source: Miercom June 2015
  • 8. Cisco vs Aruba Wireless Controllers 8 DR150602D Copyright © Miercom 2015 18 June 2015 The following charts show the results of the throughput tests. Cisco fills available bandwidth. The Cisco 5520 can effectively utilize available 20-Gbps network bandwidth at packet sizes from 256 to 1,374 bytes. The Aruba 7210 fills 10 to 25 percent of available bandwidth at packet sizes up to 516 bytes. Unaffected by number of clients. The Cisco 5520 controller proved better overall throughput with client tests for a wide range of packet sizes. Cisco outperforms Aruba. At small packets sizes up to 1,374 bytes as well as with the IMIX packet-size mixture, Cisco's latest 8540 controller roundly outperforms the Aruba 7240. Aruba throughput depends on big packets. The Aruba 7240 can make good use of its 40 Gbps of network bandwidth only with large packet sizes. The Controller datapath CPU hits 90-95% limiting its ability to pass more traffic. 17 17 18 19 2 3 5 12 0 5 10 15 20 256 IMIX 516 1374 Troughput(Gbps) Packet Size (bytes) Cisco 5520 vs Aruba 7210, 20-Gbps Wireless Conrtollers Throughput Cisco 55202 Aruba 7210 Theoretical Maximum 25 30 36 38 12 12 23 36 0 10 20 30 40 256 IMIX 516 1374 Throughput(Gbps) Packet Size (bytes) Cisco 8540 vs Aruba 7240, 40-Gbps Wireless Controllers Throughput Cisco 8540 Aruba 7240 Theoretical Maximum Source: Miercom June 2015 Source: Miercom June 2015 Cisco 5520 Aruba 7210
  • 9. Cisco vs Aruba Wireless Controllers 9 DR150602D Copyright © Miercom 2015 18 June 2015 5 - User Authentication Rate Test Objective To determine the maximum rate that new WiFi client authentications can be processed. How We Did It Our research was unable to find any known test tools on the market for conducting high-volume WiFi client-authentication testing. So to quantify client-authentication performance, we acquired a custom test platform originally developed for this purpose by Cisco. The client-authentication test package would work with Cisco wireless controllers, but not with Aruba. Subsequently, we decided to conduct this comparative performance testing on two Cisco models the latest Cisco 5520 controller, supporting 20 Gbps of network bandwidth, and the older Cisco 5508 Wireless Controller, which supports 8 Gbps of network connectivity. We decided to test most commonly used and one of the most secured authentication methods based on IEEE 802.1X WPA2 Protected Access 2 specification. IEEE802.1X authentications like PEAP, EAP-FAST , EAP-SIM, EAP-TLC are commonly used in the deployments like enterprises, campuses, hotspot deployments. This test simulates a campus spanning wireless environment, where classes change on a college campus, or a stadium game starts or restarts, or a corporation starts up in the morning. In these cases large numbers of users with wireless devices all want to connect, or disconnect and re- connect to, the wireless network or connection at approximately the same time. It is important in such environments for the WiFi system to be able to handle these requests quickly the first time. As shown in the test bed configuration diagram on the next page, the Cisco 5520 and the older 5508 were connected through a wire speed switch to five AAA/RADIUS (Remote Authentication Dial In User Service, a networking protocol that provides centralized Authentication, Authorization and Accounting (AAA)). The controllers were connected through the switch to five simulators, which were set up to simulate 1,500 APs (300 APs/simulator), each AP with 13 simulated clients – a total of 19,500 simulated clients. The simulated clients are spread across all five of the simulators. The authentication was done by roaming each simulated client from one AP to the next, at a rate of 800 roams per second. This is designed to roll all the clients through all 1,500 APs. The test was run for 5 minutes, using the 802.1X authentication.
  • 10. Cisco vs Aruba Wireless Controllers 10 DR150602D Copyright © Miercom 2015 18 June 2015 WiFi client-authentication test bed. The configuration that was used to test the controller’s ability to handle high volumes of WiFi client authentications, is shown below. Three times faster authentication. The results below show that the Cisco 5520 can authenticate new WiFi clients using the 802.1X authentication 3.25 times faster than the predecessor Cisco 5508. 235 764 0 200 400 600 800 IEEE 802.1X Authentications Authentications/Second Authentication Rates: Cisco 5508 vs Cisco 5520 5508 5520 Source: Miercom June 2015 Source: Miercom June 2015
  • 11. Cisco vs Aruba Wireless Controllers 11 DR150602D Copyright © Miercom 2015 18 June 2015 6 – Radio Resource Management Test Objective To determine how aggregate throughput varies for a set of 5 APs with Cisco 5520 and Aruba 7210 environments with a diverse mix of clients, for the various channel widths and channel- selection options offered by Cisco (RRM) and Aruba (ARM). How We Did It An elaborate test bed was set up with the Cisco 5520 and the Aruba 7210 side by side along with five sets of co-located Access Points – Cisco AP 2702i and Aruba AP 225. APs mounted in the office area for the Radio Resource Management Tests Source: Miercom June 2015 All RRM tests were performed in an office building dedicated for interference-free wireless testing where the tests can be easily reproduced and aggregated over multiple test runs. Five AP locations are located around the building on the same floor, as indicated on the floor plan on the left. The office space had a typical style of cubicle areas, and open areas with each AP neighbor about 60 to 80 feet apart. Source: Miercom June 2015
  • 12. Cisco vs Aruba Wireless Controllers 12 DR150602D Copyright © Miercom 2015 18 June 2015 The Ixia Chariot Performance tool was used to assess the overall network performance under realistic load conditions. The tool supported high-speed TCP connections with the chariot endpoints installed on every client associated to the network – a total of 100 paired TCP connections, four per wireless client during the active tests. The Ixia Chariot meticulously measured the TCP throughput for each paired connection and provided an average throughput number for each test run. Clients connected to the network The floor plan above is to scale. Each AP was connected to the five diverse clients as shown in the chart above. Clients connected to each AP were located at distance of about 15 to 35 feet in both line of sight and non-line of sight locations. Total 25 Clients 5 x Macbook Pro OS X, v10.10.10.3 802.11ac 3x3 ss 5 x Samsung Galaxy S6 Android v5.0.2 802.11ac 2x2 2ss 5 x Apple iPhone 6 iOS v8.3 802.11ac 1x1 1ss 5 x Dell Latitude E6430 Windows 7, SP2 802.11n 3x3 3ss 5 x Apple iPad Mini iOS v8.3 802.11n 1x1 1ss Source: Miercom June 2015
  • 13. Cisco vs Aruba Wireless Controllers 13 DR150602D Copyright © Miercom 2015 18 June 2015 Channels Available for APs The RRM tests were evaluated for the 5GHz Channels. Available channels for both the vendors were limited to UNII-1 (36, 40, 44, and 48) and UNII-3 (149, 153, 157, 161, and 165) channels, allowing a total of nine 20MHz non-overlapping channels to select and bond with. The DFS Channels (UNII-2 and UNII-2 extended) were excluded from the AP Channel list to create an environment with an increased chance of channel overlapping especially when the five APs are bonding on 40-MHz and 80-MHz channel widths. In addition, since Aruba ships with all DFS channels disabled, the same configuration was mirrored on the Cisco controller to ensure an apples-to-apples comparison. Restricting the number of channels for this test also helps to determine at a system level how the radio manages the channels and widths when there is limited spectrum for Wi-Fi, which again is a common occurrence in many wireless deployments across different regulatory domains. Selection for Available Channel Widths on the Cisco and Aruba Controllers at Time of Testing Bonded channel widths are achieved by using multiple 20-MHz channels. By bonding two 20MHz channels together you get a 40-MHz wide channel, and bonding four 20-MHz channels together forms a single 80-MHz wide channel. With the DFS Channels disabled, both Cisco and Aruba had a choice of:  Nine 20 MHz non-overlapping channels  Four 40 MHz non-overlapping channels  Two 80 MHz non- overlapping channels Conduct of RRM Tests The RF spectrum in each case was managed by the respective Cisco and Aruba wireless controllers, interacting with their respective access points. The tests exercised the choice of channels, the channel-width choices, and the overlap of channels affect the overall throughput in a typical office configuration. Cisco and Aruba tests were run separately for each channel-width choice, with sufficient time allowed for the configurations to settle down following each configuration change. All 25 clients Source: Miercom June 2015
  • 14. Cisco vs Aruba Wireless Controllers 14 DR150602D Copyright © Miercom 2015 18 June 2015 were associated to the APs during these settling periods. Different volumes of test traffic were run during the settling periods to simulate a typical office's busy daytime environment, and then single ping streams for inactive periods. Sequence of Bandwidth Management Testing 1. 20-MHz channel width: First the Cisco controller was set for 20-MHz channels and the network allowed to settle for six hours while clients connected to the APs. Then the IxChariot traffic was run to get an aggregate throughput performance for all 25 clients from a total of three runs. Then the Aruba controller was set for 20-MHz channels and likewise settled for six hours while clients connected to the APs. Then the IxChariot traffic was run to get an aggregate throughput performance for all 25 Clients from a total of three runs. 2. 40-MHz channel width: Similar to the same sequence as above but channel width selected as 40-MHz. 3. 80-MHz channel width: Similar to the same sequence as above but channel width selected as 80-MHz. 4. Best (Auto channel width) selection: The Cisco controller was set for best channel-selection mode to dynamically select the optimum channel widths for each individual AP and the wireless network allowed to settle for six hours while clients connected. Then the IxChariot traffic was run to get an aggregate throughput performance for all 25 Clients from a total of three runs. Aruba did not offer a comparable auto channel-selection setting for their ARM configuration. RRM Channel Selection Results The channel selection assignments for these tests are summarized in the below table: AP Channel Selection Plan Channel Width Cisco Aruba AP-1 AP-2 AP-3 AP-4 AP-5 AP-1 AP-2 AP-3 AP-4 AP-5 20 MHz 40 149 48 165 36 149 161 48 165 161 40 MHz 36-40 149-153 48-44 161-157 36-40 36-40 36-40 149-153 149-153 40-36 80 MHz 161-157- 149-153 48-44- 36-40 48-44- 36-40 161-157- 149-153 48-44- 36-40 36-40- 44-48 48-44- 40-36 157-161- 149-153 153-149- 157-161 161-157- 153-149 Auto 165 149-153 48-44 161-157 36-40 n/a n/a n/a n/a n/a In our testing we would change the AP configurations and then let them settle on the channels to be used for their 5 clients for ~6+ hours. It was observed that the Aruba’s ARM would tend to settle with more overlapping channels than Cisco’s RRM in almost all test cases. Aruba’s situation would improve if the controller and APs were rebooted before starting each test. This behavior was not present on the Cisco Infrastructure, where Cisco’s RRM would quickly configure to the selected channel widths with least overlapping channels within the first hour itself. In the test cases, where Aruba ARM’s channel selection had heavy overlapping, the controller was rebooted and allowed to settle for another 6 hour period before the performance tests were run.
  • 15. Cisco vs Aruba Wireless Controllers 15 DR150602D Copyright © Miercom 2015 18 June 2015 Channel Overlapping Comparing the final channel selections for both the vendors, it was observed that Cisco 5520 did a better job of assessing the environment and providing a better RRM configuration by selecting least overlapping channels in all test cases, thus minimizing the problem of co-channel interference. With less co-channel interference, the APs would have a better shot in delivering the traffic to the wireless clients and directly improving the overall system performance. The Aruba 7210 tended to select channels for the five APs that overlapped more than Cisco. Also, as noted, the Aruba controller did better, in terms of avoiding overlapping channels, if the controller was rebooted before starting a new test. Shown below are the visual channel selections by each controller for 20 and 40 MHz (The 80- MHz option is left out for brevity). The x-axis shows the channel numbers, and the y-axis shows the signal strength in decibels: from 0 dB (the strongest) to -90 dBm. Each mountain slope represents an AP. 20-MHz Channel Selection Visual Representation 20-MHZ CHANNEL WIDTH SELECTION TEST VENDOR NUMBER OF OVERLAPPING APS CHANNELS OVERLAP PERCENTAGE CISCO 0 0% ARUBA 2 40% CISCO RRM 20-MHz ARUBA ARM 20-MHz Source: Miercom June 2015 Source: Miercom June 2015
  • 16. Cisco vs Aruba Wireless Controllers 16 DR150602D Copyright © Miercom 2015 18 June 2015 Wi-Fi is a shared medium. When APs select an overlapping channel, the APs and clients have to share the airtime on that overlapping channel. This means that when one AP is transmitting data to the client, the other APs on the same channel will have to pause their traffic communication and wait till the first AP has finished the transmission. This goes in a round-robin fashion with each AP getting an airtime to send data to the respective clients. More the overlapping, more the waiting period for the APs. As the overlapping channel gets more crowded, the APs tend to have higher retries for a successful data transmission. With heavy overlapping of channels as seen in Aruba’s 40-MHz channel selection below, the overall performance of the network tends to reduce as the co-channel interference and the number of retransmissions goes up. 40-MHz Channel Selection Visual Representation 40-MHZ CHANNEL WIDTH SELECTION TEST VENDOR NUMBER OF OVERLAPPING APS CHANNELS OVERLAP PERCENTAGE CISCO 2 40% ARUBA 5 100% CISCO RRM 40-MHz ARUBA ARM 40-MHz Source: Miercom June 2015 Source: Miercom June 2015
  • 17. Cisco vs Aruba Wireless Controllers 17 DR150602D Copyright © Miercom 2015 18 June 2015 For the 40-MHz tests, channel overlapping was observed on both the vendors but varied in comparison. Cisco chose to overlap 2 of the 5 APs, while in Aruba's case all the 5 APs ended up sharing channels amongst each other. For the 80-MHz channel width tests, as there were only two channels to select from, both Cisco and Aruba had no choice but overlap the APs on shared channels. The AP distribution was similar for both vendors (3 on one channel, 2 on other channel) but the difference was in the overall power levels. Cisco APs had lowered the power levels more than Aruba APs to create smaller cell sizes in order to compensate for the heavy overlapping. This was determined as one of the reasons why even with the heavy overlapping for the 80-MHz channels, Cisco APs performed slightly better in terms of overall system throughput as seen in the performance comparison chart below. 80 MHZ CHANNEL WIDTH SELECTION TEST VENDOR NUMBER OF OVERLAPPING APS CHANNELS OVERLAP PERCENTAGE CISCO 5 100% ARUBA 5 100% Cisco’s Dynamic Bandwidth Channel Selection is based on the Auto channel-width select feature. As shown, four 40-MHz channels and one 20-MHz channel were used by the five APs, with minimal overlap. The result: less interference and better throughput. Cisco’s Auto Channel-Width Selection Visual Representation AUTO CHANNEL WIDTH SELECTION TEST VENDOR NUMBER OF OVERLAPPING APS CHANNELS OVERLAP PERCENTAGE CISCO 0 0% ARUBA N/A N/A Source: Miercom June 2015 CISCO RRM DBS AUTO
  • 18. Cisco vs Aruba Wireless Controllers 18 DR150602D Copyright © Miercom 2015 18 June 2015 We observed that the Aruba controller would tend to settle with more overlapping channels than the Cisco controller. All performance tests were run three times. The below graph shows the average performance of the three test runs for each channel width test case. Aruba offered no similar option for "auto" or "best" channel-bandwidth selection. As noted, Cisco offers a best channel-width selection option, where the controller selects the best channel width for each AP, based on the AP’s neighbor information, which each AP sends to the controller. We found that this option, called Dynamic Bandwidth Selection (DBS) yielded the best throughput in our test environment. With DBS, the controller selects the channel bandwidth for each AP – based on more reported frequency and interference data – rather than the operator, who selects the channel width on a more-or-less "one-channel-size-fits-all" basis. 452 649 455 730 304 308 343 0 100 200 300 400 500 600 700 800 20 MHz 40 MHz 80 MHz Auto Throughput(Mbps) Aggregate TCP Performance Across 5 APs with 25 Total Clients Cisco Aruba Source: Miercom June 2015
  • 19. Cisco vs Aruba Wireless Controllers 19 DR150602D Copyright © Miercom 2015 18 June 2015 7 - Summary Cisco 5520 and 8540 Wireless Controllers have successfully proved superior performance through testing to offer higher data plane throughput of 20 Gbps and 40 Gbps respectively and improved control plane performance for higher scale deployments. Cisco's RRM and Dynamic Bandwidth Selection together exhibited the most ideal channel & channel-width selection, resulting in the highest performing RF configuration among all the RF performance tests. Real-world testing proved that Cisco's RRM with DBS enhances the spectrum management for any typical wireless environment, thus earning the Miercom Performance Verified Certification.
  • 20. Cisco vs Aruba Wireless Controllers 20 DR150602D Copyright © Miercom 2015 18 June 2015 8 - Independent Evaluation This report was sponsored by Cisco Systems, Inc. The data was obtained completely and independently as part of Miercom's competitive analysis. 9 - About Miercom Miercom has published hundreds of network-product-comparison analyses – many made public, appearing in leading trade periodicals and other publications, and many confidential, for internal use only. Miercom’s reputation as the leading, independent product test center is undisputed. Private test services available from Miercom include competitive product analyses, as well as individual product evaluations. Miercom test methodologies are generally developed collaboratively with the client, and feature comprehensive certification and test programs including: Certified Interoperable, Certified Reliable, Certified Secure and Certified Green. Products may also be evaluated under the Performance Verified program, the industry’s most thorough and trusted assessment for product usability and performance. 10 - Use of This Report Every effort was made to ensure the accuracy of the data in this report. However, errors and/or oversights can nevertheless occur. The information documented in this report may depend on various test tools, the accuracy of which is beyond our control. Furthermore, the document may rely on certain representations by the vendors that were reasonably verified by Miercom, but are beyond our control to verify with 100-percent certainty. This document is provided “as is” by Miercom, which gives no warranty, representation or undertaking, whether express or implied, and accepts no legal responsibility, whether direct or indirect, for the accuracy, completeness, usefulness or suitability of any information contained herein. Miercom is not liable for damages arising out of or related to the information contained in this report. No part of any document may be reproduced, in whole or in part, without the specific written permission of Miercom or Cisco Systems, Inc. All trademarks used in the document are owned by their respective owners. You agree not to use any trademark in or as the whole or part of your own trademarks in connection with any activities, products or services, which are not yours. You also agree not to use any trademarks in a manner which may be confusing, misleading or deceptive or in a manner that disparages Miercom or its information, projects or developments.