Mikotik_Load_Balancing_workshop_best_practice
- 1. © MikroTik 2012 MikroTik RouterOS Workshop Load Balancing Best Practice Warsaw MUM Europe 2012
- 2. © MikroTik 2012 2 About Me Jānis Meģis, MikroTik Jānis (Tehnical, Trainer, NOT Sales) Support & Training Engineer for almost 8 years Specialization: QoS, PPP, Firewall, Routing Teaching MikroTik RouterOS classes since 2005
- 3. © MikroTik 2012 3 Load Balancing Load Balancing is a technique to distribute the workload across two or more network links in order to maximize throughput, minimise response time, and avoid overload Using multiple network links with load balancing, instead of single network links, may increase reliability through redundancy
- 4. © MikroTik 2012 4 Types of Load Balancing Sub-Packet Load Balancing (MLPPP) Per Packet Load Balancing (Bonding) Per Connection Load Balancing (nth) Per address-pair Load Balancing (ECMP, PCC, Bonding) Custom Load Balancing (Policy Routing) Bandwidth based Load Balancing (MPLS RSVP-TE Tunnels)
- 5. © MikroTik 2012 5 Multi-Link PPP PPP Multi-link Protocol allows to divide packet equally and send each part into multiple channels MLPPP can be created: over single physical link – where multiple channels run on the same link (anti-fragmentation) over multiple physical links - where multiple channels run on the multiple link (load balancing) MLPPP must be supported by both ends (MLPPP is legacy stuff from modem era)
- 6. © MikroTik 2012 6 MLPPP configuration Server must have MLPPP support All lines must have same user name and password RouterOS has only the MLPPP client implementation
- 7. © MikroTik 2012 7 Bonding Bonding is a technology that allows you to aggregate multiple Ethernet-like interfaces into a single virtual link, thus getting higher data rates and providing fail-over Bonding (load balancing) modes: 802.3ad Balance-rr Balance-xor Balance-tlb Balance-alb
- 8. © MikroTik 2012 8 802.3ad ● 802.3ad mode is an IEEE standard also called LACP (Link Aggregation Control Protocol).
- 9. © MikroTik 2012 9 Balance-rr and balance-xor Balance-rr mode uses Round Robin algorithm - packets are transmitted in sequential order from the first available slave to the last. When utilizing multiple sending and multiple receiving links, packets often are received out of order (problem for TCP) Balance-xor balances outgoing traffic across the active ports based on a hash from specific protocol header fields and accepts incoming traffic from any active port
- 10. © MikroTik 2012 10 Balance-tlb The outgoing traffic is distributed according to the current load Incoming traffic is not balanced This mode is address- pair load balancing No additional configuration is required for the switch
- 11. © MikroTik 2012 11 Balance-alb In short alb = tlb + receive load balancing This mode requires a device driver capability to change the MAC address
- 12. © MikroTik 2012 12 ECMP Routes ECMP (Equal Cost Multi Path) routes have more than one gateway to the same remote network Gateways will be used in Round Robin per SRC/DST address combination Same gateway can be written several times!!
- 13. © MikroTik 2012 13 “Check-gateway” Option You can set the router to check gateway reachability using ICMP (ping) or ARP protocols If the gateway is unreachable in a simple route – the route will become inactive If one gateway is unreachable in an ECMP route, only the reachable gateways will be used in the Round Robin algorithm If Check-gateway option is enabled on one route it will affect all routes with that gateway.
- 14. © MikroTik 2012 14 Interface ECMP Routing In case you have more that one PPP connection from the same server, but MLPPP is impossible (different user names, server support missing) it is possible to use Interface routing Simple IP address routing is impossible for all PPP connections that have the same gateway IP address To enable interface routing just specify all PPP interfaces as route gateway-interfaces Works only on PPP interfaces.
- 15. © MikroTik 2012 15 ECMP and Masquerade As forwarding database is rebuilt every 10min in Linux Kernel, there is a chance that connection will jump to the other gateway In the case of masquerading this jump results in a change of source address and in eventual disconnect More info at: http://www.enyo.de/fw/security/notes/linux-dst-cache-dos.html http://marc.info/?m=105217616607144 http://lkml.indiana.edu/hypermail/linux/net/0305.2/index.html#19
- 16. © MikroTik 2012 16 Configuration Setup
- 17. © MikroTik 2012 17 Basic Configuration
- 18. © MikroTik 2012 18 Policy Routing Policy routing is a method that allows you to create separate routing polices for different traffic by creating custom routing tables In RouterOS these routing tables are created: For every table specified in /ip route rule For every routing-mark in mangle facility Marked traffic is automatically assigned to the proper routing table (no need for lookup rules)
- 19. © MikroTik 2012 19 Routing-mark RouterOS attribute assigned to each packet Routing-mark can be changed in firewall mangle facility just before any routing decision: chain Prerouting – for all incoming traffic chain Output – for outgoing traffic from router Every new routing mark has its own routing table with the same name By default all packets have the “main” routing mark
- 20. © MikroTik 2012 20 Traffic to Connected Networks As connected routes are available only in “main” routing table, it is necessary that traffic to connected networks stay in “main” routing table This will also allow proper communication between locally and remotely connected clients
- 21. © MikroTik 2012 21 Remote Connections In the case when a connection is initiated from a public interface it is necessary to ensure that these connections will be replied via the same interface (from the same public IP) First we need to capture these connections (you can ether use default connection mark “no- mark” or connection state “new” here)
- 22. © MikroTik 2012 22 Custom Policy Routing Let's create a jump rule to your custom policy routing here Now we need to create a default route for every routing table (or else it will be resolved by main routing table)
- 23. © MikroTik 2012 23 Mark Routing Mark routing rules in mangle chain “output” will ensure that router itself is reachable via both public IP addresses Mark routing rules in mangle chain “prerouting” will ensure your desired load balancing
- 24. © MikroTik 2012 24 Mangle configuration
- 25. © MikroTik 2012 25 Custom Policy Routing There is no best way that we can suggest for load balancing, you can either: Balance based on client IP address (address list) Balance based on traffic type (p2p, layer-7, protocol, port) Use automatic balancing (PCC) We do not suggest to use “nth” for policy routing of typical user traffic.
- 26. © MikroTik 2012 26 Per-address-pair Load Balancing In many situations communication between two hosts consist of more than one simultaneous connection. If those connections are taking different routing paths they might have different latency, drop rate, fragmentation or source address (NAT) – this way making multi-connection communications impossible. That is why instead of per-connection load balancing we should think about per-address- pair load balancing
- 27. © MikroTik 2012 27 Per Connection Classifier PCC is a firewall matcher that allows you to divide traffic into equal streams with ability to keep packets with specific set of options in one particular stream You can specify set of options from src-address, src-port, dst-address, dst-port More info at: http://wiki.mikrotik.com/wiki/PCC
- 28. © MikroTik 2012 28 PCC Configuration We just need to add 2 rules to our “policy_routing” chain to ensure automatic per- address-pair load balancing
- 29. © MikroTik 2012 29 Usual Problems Be careful about using “no-mark” connection mark if you have other mangle configuration in a different chain ISP specified DNS servers might block requests from non-ISP public IPs, so we suggest you use public (ISP independent) DNS servers. If you would like to ensure fail-over – enable “check-gateway” option in all default routes.
- 30. © MikroTik 2012 30 What about bandwidth based Load-Balancing?
- 31. © MikroTik 2012 31 Traffic Engineering TE is one of MPLS features that allow to establish unidirectional label switching paths TE is based on RSVP (Resource ReSerVation Protocol) + RFC 3209 that adds support for explicit route and label exchange TE tunnels are similar to LDP, but with additional features: Usage of either full or partial explicit routes Constraint (such as bandwidth and link properties) based LSP (Label Switched Path) establishment
- 32. © MikroTik 2012 32 How Does Constraints Work? Constraints are set by user and does not necessarily reflect actual bandwidth Constraints can be set for: bandwidth of link participating in a RSVP TE network bandwidth reserved for tunnel So, at any moment in time, the bandwidth available on TE link is bandwidth configured for link minus sum of all reservations made on the link (not physically available bandwidth)
- 33. © MikroTik 2012 33 TE Tunnel Establishment TE tunnels can be established: along the current routing path (no additional configuration required) along a statically configured explicit path (it is necessary to manually input path) CSPF (Constrained Shortest Path First) - This option needs assistance from IGP routing protocol (such as OSPF) to distribute bandwidth information throughout the network.
- 34. © MikroTik 2012 34 Network Layout Each router is connected to a neighbouring router using /30 network and each of them have unique Loopback address form 10.255.0.x network. Loopback addresses will be used as tunnel source and destination.
- 35. © MikroTik 2012 35 Network Layout
- 36. © MikroTik 2012 36 Loopback and CSPF Loopback addresses need to be reachable from whole network – we will use OSPF to distribute that information Also OSPF can help us to distribute TE reservations for CSPF
- 37. © MikroTik 2012 37 Resource Reservation Lets set up TE resource for every interface on which we might want to run TE tunnel. Configuration on all the routers are the same: Note that at this point this does not represent how much bandwidth will actually flow through the interface
- 38. © MikroTik 2012 38 First Task
- 39. © MikroTik 2012 39 TE tunnel setup We will use static path configuration as primary, and dynamic (CSPF) as secondary path if primary fails
- 40. © MikroTik 2012 40 TE Tunnel Monitoring
- 41. © MikroTik 2012 41 TE Tunnel Monitoring If multiple tunnels are created and all the bandwidth on that particular interface is used, then the tunnel will try to look for different path.
- 42. © MikroTik 2012 42 Route traffic over TE To route LAN traffic over a TE tunnel we will assign address 10.99.99.1/30 and 10.99.99.2/30 to each tunnel end.
- 43. © MikroTik 2012 43 Automatic Failover By default the tunnel will try to switch back to the primary path every minute. This setting can be changed with primary-retry-interval parameter.
- 44. © MikroTik 2012 44 Additional Tunnels
- 45. © MikroTik 2012 45 Additional Tunnels
- 46. © MikroTik 2012 46 Good luck! http://wiki.mikrotik.com/wiki/Manual:Simple_TE http://wiki.mikrotik.com/wiki/Manual:TE_Tunnels http://wiki.mikrotik.com/wiki/Manual:MPLS/Traffic-eng http://wiki.mikrotik.com/wiki/Manual:MPLS/Overview