IBM Flex System IB6132D 2-port FDR InfiniBand Adapter

®

IBM Flex System IB6132D 2-port FDR InfiniBand
Adapter
IBM Redbooks Product Guide
The IBM® Flex System™ IB6132D 2-port FDR InfiniBand Adapter is a two-port mid-mezzanine card for
the IBM Flex System x222 Compute Node. It delivers low latency and high bandwidth for
performance-driven server clustering applications in enterprise data centers, high-performance computing
(HPC), and embedded environments. The adapter is designed to operate at InfiniBand FDR speeds (56
Gbps or 14 Gbps per lane) and provides one 56 Gbps port to each of the independent servers in the x222
Compute Node.
The following figure shows the IBM Flex System IB6132D 2-port FDR InfiniBand Adapter.

Figure 1. IBM Flex System IB6132D 2-port FDR InfiniBand Adapter

Did you know?
Mellanox InfiniBand adapters deliver industry-leading bandwidth with ultra-low, submicrosecond latency
for performance-driven server clustering applications. Combined with the IB6131 InfiniBand Switch, your
organization can achieve efficient computing by offloading, from the CPU, protocol processing, and data
movement impact, such as Remote Direct Memory Access (RDMA) and Send/Receive semantics,
allowing more processor power for the application. Advanced acceleration technology enables more than
90,000,000 Message Passing Interface (MPI) messages per second, making it a highly scalable adapter
that delivers cluster efficiency and scalability to tens-of-thousands of nodes.

IBM Flex System IB6132D 2-port FDR InfiniBand Adapter

1

Part number information
The following table shows the part number to order this card.
Table 1. Part number and feature code for ordering
Description

Part number

Feature code
(x-config)

Feature code
(e-config)


90Y3486

A365

A365

The part number includes the following items:




One IBM Flex System IB6132D 2-port FDR InfiniBand Adapter
A documentation CD containing the adapter user’s guide
The IBM Important Notices document

Features
The IBM Flex System IB6132D 2-port FDR InfiniBand Adapter has the following features.
Performance
Based on Mellanox ConnectX-3 technology, the IB6132D 2-port FDR InfiniBand Adapter provides a high
level of throughput performance for all network environments by removing I/O bottlenecks in mainstream
servers that are limiting application performance. Servers can achieve up to 56 Gbps transmit and receive
bandwidth. Hardware-based InfiniBand transport and IP over InfiniBand (IPoIB) stateless offload engines
handle the segmentation, reassembly, and checksum calculations that otherwise burden the host
processor.
RDMA over the InfiniBand fabric further accelerates application run time while reducing CPU utilization.
RDMA allows very high-volume transaction-intensive applications typical of HPC and financial market
firms, as well as other industries where speed of data delivery is paramount. With the ConnectX-3-based
adapter, highly compute-intensive tasks running on hundreds or thousands of multiprocessor nodes, such
as climate research, molecular modeling, and physical simulations, can share data and synchronize
faster, resulting in shorter run times. High-frequency transaction applications are able to access trading
information more quickly, making sure that the trading servers are able to respond first to any new market
data and market inefficiencies, while the higher throughput enables higher volume trading, maximizing
liquidity and profitability.
In data mining or web crawl applications, RDMA provides the needed boost in performance to search
faster by solving the network latency bottleneck that is associated with I/O cards and the corresponding
transport technology in the cloud. Various other applications that benefit from RDMA with ConnectX-3
include Web 2.0 (Content Delivery Network), Business Intelligence, database transactions, and various
cloud-computing applications. The low-power consumption of Mellanox ConnectX-3 provides clients with
high bandwidth and low latency at the lowest cost of ownership.
I/O virtualization
Mellanox adapters that use Virtual Intelligent Queuing (Virtual-IQ) technology with SR-IOV provide
dedicated adapter resources and ensured isolation and protection for virtual machines (VM) within the
server. I/O virtualization on InfiniBand gives data center managers better server utilization and LAN and
SAN unification while reducing cost, power, and cable complexity.


2

Quality of service
Resource allocation per application or per VM is provided by the advanced quality of service (QoS) that is
supported by ConnectX-3. Service levels for multiple traffic types can be assigned on a per flow basis,
allowing system administrators to prioritize traffic by application, virtual machine, or protocol. This
powerful combination of QoS and prioritization provides the ultimate fine-grained control of traffic,
ensuring that applications run smoothly in today’s complex environments.

Specifications
The IBM Flex System IB6132D 2-port FDR InfiniBand Adapter has the following specifications:


Based on Mellanox Connect-X3 technology.



Two independent Mellanox ASICs, one port per ASIC.



Two-port card, with one port routed to each of the independent servers in the x222 Compute Node.



Each port operates at up to 56 Gbps.



InfiniBand Architecture Specification V1.2.1 compliant.



Supported InfiniBand speeds (auto-negotiated):






1X/2X/4X Single Data Rate (SDR) (2.5 Gbps per lane)
Double Data Rate (DDR) (5 Gbps per lane)
Quad Data Rate (QDR) (10 Gbps per lane)
FDR10 (40 Gbps, 10 Gbps per lane)
Fourteen Data Rate (FDR) (56 Gbps, 14 Gbps per lane)



PCI Express 3.0 x8 host-interface operates at up to 8 gigatransfers per second (GTps) bandwidth.



CPU offload of transport operations.



CORE-Direct® application offload.



GPUDirect application offload.



End-to-end QoS and congestion control.



Hardware-based I/O virtualization.



TCP/UDP/IP stateless offload.



Ethernet encapsulation (EoIB).



RoHS-6 compliant.

Note: To operate at InfiniBand FDR speeds, the IBM Flex System IB6131 InfiniBand Switch requires the
FDR Update license, 90Y3462.


3

Supported servers
The following table lists the IBM Flex System compute nodes that support the IB6132D 2-port FDR
InfiniBand Adapter.
Table 2. Supported servers

Description
IBM Flex System IB6132D 2-port FDR InfiniBand
Adapter

Part
number
90Y3486

N

Y

N

N

N

N

N

N

N

For the latest information about the expansion cards that are supported by each blade server type, see
IBM ServerProven® at the following web address:
http://ibm.com/servers/eserver/serverproven/compat/us/
Mid-mezzanine cards, such as the IB6132D, are installed in the x222, as shown in the following figure.
Only one adapter can be installed, but the adapter can connect to both servers in the x222 Compute
Node.

Figure 2. The adapter installed in the IBM Flex System x222 Compute Node


4

Supported I/O modules
The IB6132D 2-port FDR InfiniBand Adapter supports the I/O module that is listed in the following table.
To operate the IB6131 InfiniBand Switch at FDR speeds (56 Gbps), you must also install the FDR
Upgrade license, 90Y3462.
Table 3. I/O modules that are supported by the IB6132D 2-port FDR InfiniBand Adapter
Description

Part number

IBM Flex System IB6131 InfiniBand Switch

90Y3450

IBM Flex System IB6131 InfiniBand Switch (FDR Upgrade)*

90Y3462

* This license allows the switch to support FDR speeds.

A switch module must be installed in both bays 3 and 4 in the chassis. As shown in the following table, the
upper compute node in the x222 routes through the card to the switch in bay 4 and the lower compute
node in the x222 routes through the card to the switch in bay 3. The adapter has two ports, and each port
is driven by its own ASIC.
Table 4. Adapter to I/O bay correspondence
Compute node

IB6132D 2-port
FDR InfiniBand

Corresponding I/O module
bay in the chassis

Upper compute node

Upper Port 1

Module bay 4

Lower compute node

Lower Port 1

Module bay 3

The IB6132D 2-port FDR InfiniBand adapter is installed in the single I/O expansion slot in the x222. The
following figure shows how the IB6132D 2-port FDR InfiniBand adapter is connected to InfiniBand
switches that are installed in the chassis. The figure also shows the four ports of the two Embedded 10
GbE Virtual Fabric Adapters that are routed to the Ethernet switches in bays 1 and 2.

Figure 3. Logical layout of the x222 interconnects - Ethernet and InfiniBand


5

Supported operating systems
The IB6132D 2-port FDR InfiniBand Adapter supports the following 64-bit operating systems:












Microsoft Windows Server 2008 R2
Microsoft Windows Server 2008, Datacenter x64 Edition
Microsoft Windows Server 2008, Enterprise x64 Edition
Microsoft Windows Server 2008, Standard x64 Edition
Microsoft Windows Server 2008, Web x64 Edition
Microsoft Windows Server 2012
Red Hat Enterprise Linux 5 Server with Xen x64 Edition
Red Hat Enterprise Linux 5 Server x64 Edition
Red Hat Enterprise Linux 6 Server x64 Edition
SUSE Linux Enterprise Server 10 for AMD64/EM64T
SUSE Linux Enterprise Server 11 for AMD64/EM64T

Regulatory compliance
The adapter conforms to the following standards:

















United States FCC 47 CFR Part 15, Subpart B, ANSI C63.4 (2003), Class A
United States UL 60950-1, Second Edition
IEC/EN 60950-1, Second Edition
FCC - Verified to comply with Part 15 of the FCC Rules, Class A
Canada ICES-003, issue 4, Class A
UL/IEC 60950-1
CSA C22.2 No. 60950-1-03
Japan VCCI, Class A
Australia/New Zealand AS/NZS CISPR 22:2006, Class A
IEC 60950-1(CB Certificate and CB Test Report)
Taiwan BSMI CNS13438, Class A
Korea KN22, Class A; KN24
Russia/GOST ME01, IEC-60950-1, GOST R 51318.22-99, GOST R 51318.24-99, GOST R
51317.3.2-2006, and GOST R 51317.3.3-99
IEC 60950-1 (CB Certificate and CB Test Report)
CE Mark (EN55022 Class A, EN60950-1, EN55024, EN61000-3-2, and EN61000-3-3)
CISPR 22, Class A

Physical specifications
The dimensions and weight of the adapter are as follows:




Width: 158 mm (6.2 in.)
Depth: 108 mm (4.2 in.)
Weight: 230 g (0.5 lb)

Shipping dimensions and weight (approximate):





Height: 97 mm (3.8 in.)
Width: 165 mm (6.5 in.)
Depth: 215 mm (8.5 in.)
Weight: 430 g (0.95 lb)


6

Popular configurations
The IB6132D 2-port FDR InfiniBand Adapter is used with the IB6131 InfiniBand Switch. The following
figure shows one IB6132D adapter that is installed in an x222 Compute Node, which in turn is installed in
the chassis. Two IB6131 InfiniBand Switches are installed in I/O bays 3 and 4.

Figure 4. Example configuration
The following table lists the parts that are used in the configuration. This configuration includes the FDR
upgrade license for the IB6131 switch, as well as the FDR cables.
Table 5. Components that are used when connecting the IB6132D 2-port FDR InfiniBand Adapter to the
IB6131 InfiniBand Switches
Part number/machine type

Description

Quantity

7916

IBM Flex System x222 Compute Node

1 - 14

90Y3486

IB6132D 2-port FDR InfiniBand Adapter

1 per x222 Compute
Node

8721-A1x

IBM Flex System Enterprise Chassis

1

90Y3450

IBM Flex System IB6131 InfiniBand Switch (in bays 3
and 4)

2

90Y3462

IBM Flex System IB6131 InfiniBand Switch (FDR
Upgrade)

2

90Y3470

3m FDR InfiniBand Cable

Up to 36 (18 per switch)


7

Related publications
For more information, see the following resources:


IBM Flex System IB6131 InfiniBand Switch Product Guide

http://www.redbooks.ibm.com/abstracts/tips0871.html?Open


IBM Flex System x222 Compute Node Product Guide

http://www.redbooks.ibm.com/abstracts/tips1036.html?Open


IBM Redbooks® Product Guides for IBM Flex System servers and options
http://www.redbooks.ibm.com/Redbooks.nsf/portals/PureSystems?Open&page=pgbycat



IBM Flex System Information Center (including User's Guides)
http://publib.boulder.ibm.com/infocenter/flexsys/information



IBM Flex System Interoperability Guide
http://www.redbooks.ibm.com/fsig



IBM Redbooks publication IBM Flex System Products and Technology, SG24-7984
http://www.redbooks.ibm.com/abstracts/sg247984.html


8

Notices
This information was developed for products and services offered in the U.S.A.
IBM may not offer the products, services, or features discussed in this document in other countries. Consult your local
IBM representative for information on the products and services currently available in your area. Any reference to an
IBM product, program, or service is not intended to state or imply that only that IBM product, program, or service may
be used. Any functionally equivalent product, program, or service that does not infringe any IBM intellectual property
right may be used instead. However, it is the user's responsibility to evaluate and verify the operation of any non-IBM
product, program, or service. IBM may have patents or pending patent applications covering subject matter described
in this document. The furnishing of this document does not give you any license to these patents. You can send
license inquiries, in writing, to:

IBM Director of Licensing, IBM Corporation, North Castle Drive, Armonk, NY 10504-1785 U.S.A.
The following paragraph does not apply to the United Kingdom or any other country where such provisions are
inconsistent with local law : INTERNATIONAL BUSINESS MACHINES CORPORATION PROVIDES THIS
PUBLICATION "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING, BUT
NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS
FOR A PARTICULAR PURPOSE. Some states do not allow disclaimer of express or implied warranties in certain
transactions, therefore, this statement may not apply to you. This information could include technical inaccuracies or
typographical errors. Changes are periodically made to the information herein; these changes will be incorporated in
new editions of the publication. IBM may make improvements and/or changes in the product(s) and/or the program(s)
described in this publication at any time without notice.
Any references in this information to non-IBM Web sites are provided for convenience only and do not in any manner
serve as an endorsement of those Web sites. The materials at those Web sites are not part of the materials for this
IBM product and use of those Web sites is at your own risk.IBM may use or distribute any of the information you
supply in any way it believes appropriate without incurring any obligation to you. Information concerning non-IBM
products was obtained from the suppliers of those products, their published announcements or other publicly available
sources. IBM has not tested those products and cannot confirm the accuracy of performance, compatibility or any
other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to
the suppliers of those products. This information contains examples of data and reports used in daily business
operations. To illustrate them as completely as possible, the examples include the names of individuals, companies,
brands, and products. All of these names are fictitious and any similarity to the names and addresses used by an
actual business enterprise is entirely coincidental.
Any performance data contained herein was determined in a controlled environment. Therefore, the results obtained
in other operating environments may vary significantly. Some measurements may have been made on
development-level systems and there is no guarantee that these measurements will be the same on generally
available systems. Furthermore, some measurement may have been estimated through extrapolation. Actual results
may vary. Users of this document should verify the applicable data for their specific environment.
COPYRIGHT LICENSE:
This information contains sample application programs in source language, which illustrate programming techniques
on various operating platforms. You may copy, modify, and distribute these sample programs in any form without
payment to IBM, for the purposes of developing, using, marketing or distributing application programs conforming to
the application programming interface for the operating platform for which the sample programs are written. These
examples have not been thoroughly tested under all conditions. IBM, therefore, cannot guarantee or imply reliability,
serviceability, or function of these programs.
© Copyright International Business Machines Corporation 2013. All rights reserved .
Note to U.S. Government Users Restricted Rights -- Use, duplication or disclosure restricted by
GSA ADP Schedule Contract with IBM Corp.


9

This document was created or updated on July 31, 2013.
Send us your comments in one of the following ways:
Use the online Contact us review form found at:
ibm.com/redbooks

Send your comments in an e-mail to:
redbook@us.ibm.com

Mail your comments to:
IBM Corporation, International Technical Support Organization
Dept. HYTD Mail Station P099
2455 South Road
Poughkeepsie, NY 12601-5400 U.S.A.


This document is available online at http://www.ibm.com/redbooks/abstracts/tips1056.html .

Trademarks
IBM, the IBM logo, and ibm.com are trademarks or registered trademarks of International Business
Machines Corporation in the United States, other countries, or both. These and other IBM trademarked
terms are marked on their first occurrence in this information with the appropriate symbol (® or ™),
indicating US registered or common law trademarks owned by IBM at the time this information was
published. Such trademarks may also be registered or common law trademarks in other countries. A
current list of IBM trademarks is available on the Web at http://www.ibm.com/legal/copytrade.shtml.
The following terms are trademarks of the International Business Machines Corporation in the United
States, other countries, or both:
IBM®
IBM Flex System™
Redbooks®
Redbooks (logo)®
ServerProven®
The following terms are trademarks of other companies:
Linux is a trademark of Linus Torvalds in the United States, other countries, or both.
Microsoft, Windows, and the Windows logo are trademarks of Microsoft Corporation in the United States,
other countries, or both.
Other company, product, or service names may be trademarks or service marks of others.


10

IBM Flex System IB6132D 2-port FDR InfiniBand Adapter

More Related Content

What's hot (18)

Similar to IBM Flex System IB6132D 2-port FDR InfiniBand Adapter (20)

More from IBM India Smarter Computing (20)

Recently uploaded (20)

IBM Flex System IB6132D 2-port FDR InfiniBand Adapter