Mlag invisibile layer 2 redundancy

®
®
MLAG: Invisible Layer 2 Redundancy
Scott Emery
Cumulus Networks
May 20, 2015

®
Agenda
u  What is MLAG?
u  How does MLAG work?
u  How to set up an MLAG
u  Tools for MLAG analysis and debugging

®
MLAG Introduction
You need to set up a rack of servers for a new
application
u  Add some extra servers for redundancy
u  Uplink to redundant core switches
u  Redundant Internet connections
u  Backup power with batteries and generators
u  Over-provisioned cooling
You receive a midnight call that everything is
down

®
MLAG Introduction
MLAG – A LAG across more than one node
u  Multi-homing for redundancy
u  Active-active to utilize all links which otherwise
may get blocked by Spanning Tree
u  No modiﬁcation of LAG partner

®
MLAG Terminology
S1 S2
H1 H2 H3 H4 H5
Secondary Role
ISL – Inter-Switch Link
Dually Connected
Primary Role
Singly Connected

®
MLAG Partner View
S1 S2
H1 H2 H3 H4 H5
Switch

®
The Fundamental Job of MLAG
S1 S2
S1 S2
Make this:
Look like this:
Switch

®
MLAG and LACP
u  Both ends must run LACP
u  Normally, when connected
to two different systems,
only one link is used
•  Common system ID is used on
each switch
u  Identiﬁcation of which ports
on each system are dual-
connected pairs
S1 S2
H1 H2 H3 H4 H5

®
Eliminating Duplicate Packets
u  BUM1 packets are ﬂooded and result in:
§  Duplicate packets at dual-connected hosts
§  A dual-connected host receives packets which
it transmitted
1 BUM packets are: Broadcast, Unknown unicast, and Multicast

®
S1 S2
H1 H2 H3 H4 H5
H2 sends a BUM packet which goes up the link to S1

®
S1 sends the packet out all interfaces in the bridge, except the
interface on which the packet arrived
S1 S2
H1 H2 H3 H4 H5

®
S1 S2
H1 H2 H3 H4 H5

®
u  Dual-connected hosts receive duplicate
copies of the packet
u  Dual-connected hosts which send BUM
packets receive the packet they sent
u  To ﬁx this: Packets received on the ISL are
not forwarded to dual-connected ports

®
S2 only sends packet out singly-connected interfaces
S1 S2
H1 H2 H3 H4 H5

®
MAC Address Learning
u  To act as a single logical switch, both switches
must synchronize their MAC address tables
§  Addresses learned on dual-connected ports are
added to the corresponding port on the other
switch
§  Addresses learned on singly-connected ports are
added to the ISL on the other switch
§  Address learning is disabled on the ISL

®
H2 sends a BUM packet, S1 learns the port to H2
S1 S2
H1 H2 H3 H4 H5
H2

®
S1 S2
H1 H2 H3 H4 H5
H2

®
S2 would ordinarily learn H2 on the ISL and forward the
packet out all singly-connected ports
S1 S2
H1 H2 H3 H4 H5
H2
H2

®
But, learning is disabled on the ISL. Instead, S1 sends a MAC
sync message to S2 which adds H2 to the dual connected port
S1 S2
H1 H2 H3 H4 H5
H2 H2

®
For singly-connected hosts, the MAC sync message causes the
address to be added to the ISL
S1 S2
H1 H2 H3 H4 H5
H1
H1

®
Final MAC address tables may look like this. Red: Address
originally learned on switch. Blue: Address added by MAC sync
S1 S2
H1 H2 H3 H4 H5
H2 H5H1 H4H3
H5
H2
H1
H4H3

®
Switch-Switch MLAG
u  Just like a host can be connected to two
switches, a pair of MLAG'd switches can be
connected to another pair of MLAG'd
switches
§  Used to create larger redundant L2 networks
§  Each pair of MLAG'd switches views the other
switches as a single logical switch

®
Switch-Switch MLAG
S3 S4
S1 S2

®
Switch-Switch MLAG
S3 S4Switch
S1 S2Switch

®
Spanning Tree
u  One switch is set as the primary,
the other is secondary
u  Both switches use the same
bridge ID, dual connected ports
have the same port ID
u  Only primary sends BPDUs on
dual-connected ports
u  BPDUs received on dual
connected ports are sent to the
peer unmodiﬁed
u  BPDUs received on the root port
are sent to the peer unmodiﬁed
u  Source MACs of BPDUs received
on peer link are checked
u  Peer link never blocks
S1 S2
M1
R1

®
Split Brain
u  If one switch sees that
the ISL is down it
cannot distinguish
between the link going
down (split brain) and
the peer switch going
down (solo)
u  A backup link is used to
make this distinction
S1 S2
H1 H2 H3 H4 H5
S1 S2
H1 H2 H3 H4 H5
??? Which One ???

®
Split Brain
u  When the ISL goes
down, the backup
link can determine
if the peer switch is
still alive
S1 S2
H1 H2 H3 H4 H5

®
Conﬁguring MLAG
In /etc/network/interfaces put all dual-connected ports
in an 802.3ad bond and assign them a clag-id
auto bond1
iface bond1 inet static
bond-slaves swp48
bond-mode 802.3ad
bond-miimon 100
bond-use-carrier 1
bond-lacp-rate 1
bond-min-links 1
bond-xmit_hash_policy layer3+4
clag-id 1
auto bond11
iface bond11 inet static
bond-slaves swp4
bond-mode 802.3ad
bond-miimon 100
bond-use-carrier 1
bond-lacp-rate 1
bond-min-links 1
bond-xmit_hash_policy layer3+4
clag-id 1
Switch S1 Switch S2

®
Conﬁguring MLAG
In /etc/network/interfaces assign clagd
parameters on a VLAN sub-interface of the ISL link
auto peer6.4000
iface peer6.4000 inet static
address 169.254.0.1
netmask 255.255.255.0
clagd-peer-ip 169.254.0.2
clagd-sys-mac 44:38:39:ff:bb:01
clagd-backup-ip 192.168.1.101
auto peer16.4000
iface peer16.4000 inet static
address 169.254.0.2
netmask 255.255.255.0
clagd-peer-ip 169.254.0.1
clagd-sys-mac 44:38:39:ff:bb:01
clagd-backup-ip 192.168.1.100
Switch S1 Switch S2

®
MLAG Tools
clagctl can be used to get the current state of
the MLAG
# clagctl
The peer is alive
Peer Priority, ID, and Role: 32768 00:02:00:00:00:17 primary
Our Priority, ID, and Role: 32768 70:72:cf:e9:f0:76 secondary
Peer Interface and IP: peer6.4000 169.254.0.2
Backup IP: 192.168.1.101 (active)
System MAC: 44:38:39:ff:bb:01
Dual Attached Ports
Our Interface Peer Interface CLAG Id
---------------- ---------------- -------
bond4 bond14 4
bond5 bond15 5
bond1 bond11 1
bond2 bond12 2
bond3 bond13 3
$ clagctl
The peer is alive
Our Priority, ID, and Role: 32768 00:02:00:00:00:17 primary
Peer Priority, ID, and Role: 32768 70:72:cf:e9:f0:76 secondary
Peer Interface and IP: peer16.4000 169.254.0.1
Backup IP: 192.168.1.100 (active)
System MAC: 44:38:39:ff:bb:01
Dual Attached Ports
Our Interface Peer Interface CLAG Id
---------------- ---------------- -------
bond14 bond4 4
bond15 bond5 5
bond12 bond2 2
bond13 bond3 3
bond11 bond1 1
Switch S1 Switch S2

®
MLAG Tools
/var/log/syslog contains MLAG status changes
# grep clagd /var/log/syslog
May 19 16:25:31 act-5712-08 clagd[7253]: Beginning execution of clagd version 1.1.0
May 19 16:25:31 act-5712-08 clagd[7253]: Invoked with: /usr/sbin/clagd --daemon 169.254.0.2
peer6.4000 44:38:39:ff:bb:01
May 19 16:25:31 act-5712-08 clagd[7258]: Role is now secondary
May 19 16:25:32 act-5712-08 clagd[7258]: Initial config loaded
May 19 16:25:33 act-5712-08 clagd[7258]: The peer switch is active.
May 19 16:25:33 act-5712-08 clagd[7258]: Initial data sync from peer done.
May 19 16:25:33 act-5712-08 clagd[7258]: Initial handshake done.
May 19 16:25:33 act-5712-08 clagd[7258]: Initial data sync to peer done.
May 19 16:25:37 act-5712-08 clagd[7258]: bond2 is now dual connected.

®
© 2014 Cumulus Networks. Cumulus Networks, the Cumulus Networks Logo, and Cumulus Linux are trademarks or registered trademarks of Cumulus Networks, Inc.
or its affiliates in the U.S. and other countries. Other names may be trademarks of their respective owners. The registered trademark Linux® is used pursuant to a
sublicense from LMI, the exclusive licensee of Linus Torvalds, owner of the mark on a world-wide basis.
§ Thank You!
®
cumulusnetworks.com 32

Mlag invisibile layer 2 redundancy

More Related Content

What's hot (20)

Viewers also liked (20)

Similar to Mlag invisibile layer 2 redundancy (20)

More from Cumulus Networks (15)

Recently uploaded (20)

Mlag invisibile layer 2 redundancy