Why You Should Choose a Software-Based Load Balancer

Editor's Notes

• #4: Thank you Owen. There is no doubt that in IT we are seeing a move away from using proprietary hardware solutions and toward the use of commodity hardware and virtualization. Today I will talk about this trend in relation to the Application Delivery Controller or Load Balancer.
• #5: In all parts of IT we are seeing this move away from hardware-based solutions. A major factor in this trend was the move to server consolidation brought about by the introduction of the hypervisor and of course cloud computing is taking the world by storm and wouldn’t be possible without virtualization. And we have seen the move away from proprietary hardware in other areas with the introduction of software-based switches, routers, firewalls, etc. And for the load balancer or Application Delivery Controller this is happening too, and for many of the same reasons, and that is what we will be focusing on today.
• #6: Before I start talking about some of the reasons why you should consider deploying a software load balancer or Application Delivery Controller (or ADC), let me first talk about the different types of load balancers. I won’t necessarily be covering every possible scenario, but I will cover the main ones.
• #7: Lets start with the hardware load balancer. In this case a proprietary hardware appliance runs the load balancing software on a proprietary operating system, usually based on Linux or FreeBSD. There is a single instance of the software and it can support multiple applications. By applications, I don’t mean that applications are running on the load balancer but that these are the applications that are being load balanced. The amount of isolation between applications will vary but is limited because they all share the same hardware resources.
• #8: A more recent development in the hardware appliance space has been support for virtual load balancers on a hardware load balancer appliance. So you have a proprietary hardware chassis and a basic hypervisor that can run multiple instances of load balancers. The degree of isolation between the virtual load balancers depends on whether the vendor has implemented their own hypervisor, implemented virtualization in software, or bundled an off-the-shelf hypervisor such as Xen. Here I show one application being supported by each instance of the load balancer, which is a common scenario, but it is certainly possible to support multiple applications as well. We will talk more about this later in the presentation.
• #9: Another recent development is the release, by hardware load balancing vendors, of virtual appliances. These are single, pre-build, load balancer instances that run on supported hypervisors. I’ve showed VMware here. A we will see, it’s often appropriate to consider them with hardware load balancers because they are priced and managed as if they were hardware.
• #10: And finally we get to the topic of today’s webinar, the software load balancer. Bare software, it can be installed in any supported environment, including bare metal servers, containers, hypervisors or the cloud. As flexible and as easy to manage as the software they are load-balancing e.g. webservers, app servers. Here I’ve shown just two possibilities, on bare metal running Linux and using Linux on VMware, but I could also have shown installing the load balancer in the cloud or in containers.
• #11: So to be clear, when I’m talking about a software-based load balancer or just software load balancer I’m talking about a load balancer that is delivered as pure software and can be run on the commodity hardware or virtualization technology of your choice, as opposed to a hardware load balancer where you purchase proprietary hardware that comes loaded with the load balancing software. In both cases the load balancers are software running on hardware but with the software load balancer the software is decoupled from the hardware. There can sometimes be confusion when it comes to virtual load balancers, where a hardware load balancing vendor has come out with a virtual appliance version of their product as we discussed earlier. Although this allows for the decoupling of the load balancer from the hardware it runs on, these products are not provided as a software install, only as a virtual appliance. This means that you are not able to run them on commodity bare metal hardware or in virtual or cloud environments that are not directly supported by the vendor. So virtual load balancers do not equal software load balancers. A case can be made for the use of both hardware and software load balancers, depending on the circumstances, but in short, software load balancers are appropriate for users who think in terms of applications and hardware load balancers are more appropriate for users who think in terms of the network. It is also important to note that a software load balancer can usually be used in place of a hardware load balancer but the opposite is often not true, as in the case of virtualization and cloud.
• #12: Now let’s start talking about some of the reasons why you should be looking at choosing a software load balancer.
• #13: If I had to sum up in one word the reason you should be looking at moving to a software load balancer, that word would be flexibility. During this webinar I will be talking about several more detailed reasons but in the end they are all dimensions of flexibility. Now let’s get into some of those detailed reasons.
• #14: One of the great advantages of using a software load balancer is that you can run it on the commodity hardware, hypervisor, container or cloud environment of your choice. For example, NGINX Plus is supported on a number of Linux distributions and FreeBSD on x86, so any x86 hardware capable of running these operating systems can be used and you can create your own virtual appliance in any virtualization environment that supports Linux or FreeBSD and if you can get a virtual Linux or FreeBSD instance in a cloud environment (which includes all the ones I know of) then you can run NGINX Plus there. When using a hardware ADC, you must purchase the hardware from the vendor so you are restricted to the models they have available. Even if you use a load balancer delivered as a virtual appliance, you can only use it in the virtualization environments supported by the vendor and the same is true with the cloud. The reason for this is that most hardware load balancers are built on a customized and proprietary version of Linux or FreeBSD and since these operating systems are proprietary, their virtual appliance can only run where the vendor has make the effort to get them to work. And they don’t make the OS or their load balancing software available outside their virtual appliance so you can’t build anything yourself. Any finally, when a load balancer is built from the ground up to be software-based, the same core software will be used no matter where it it is run. This is not necessarily the case with a load balancer that starts as being hardware-based because these products will usually rely on some hardware assist features, for example to spread request across CPU cores, so that when they move to develop a virtual appliance they can no longer rely on these hardware assist features, the vendor has to create more core software to handle these features. This can have a major impact on performance, so these new versions have had to mature in this area with regards to supported features and performance. So with a software load balancer, whether you are running it in production on a bare metal server, on a test server in the cloud or in a virtual machine on a developers desktop, you are running exactly the same software and can expect exactly the same behavior.
• #15: Another big benefit of using a software ADC over a hardware ADC is the ability to right size your hardware. I know that “right sizing” is a common industry buzzword, but in this case it really does apply. And there are a few different aspects to right sizing. The first one I will talk about is that when it comes to provisioning the hardware for a software load balancer, you can buy what you need for your current and near future load rather then having to forecast far out into the future. When purchasing a hardware ADC, you will usually have a refresh cycle of between 4 to 6 years. This means that when you purchase the devices (they usually come in pairs) you need to predict your load at the end of the refresh period and purchase the hardware to meet that need, regardless of what your current needs are.
• #16: Lets look at how you size hardware appliance. For this example we will assume a refresh cycle of 5 years. So the first thing we need to do is estimate what our load will be at the end of 5 years. Here we see the predicted growth rate for this example. Once we have this estimate we must purchase a pair of load balancers to meet that need [advance], but we must purchase them at the beginning [advance] with the expectation that they will remain in place for 5 years. Once we have made this purchase, there are three possibilities. 1) We estimated it just right which is great since we got the expected 5-years out of the appliances, but that means that until we hit the load target we were paying for more hardware then we needed. Or 2) we underestimated. In this example we exhausted the capacity of our hardware appliances in 3 years so we have to upgrade early to a bigger pair [Advance]. Since when you refresh hardware appliances you need to re-buy the hardware and software it is very expensive, despite some amount of trade-in, so it is extra costly to not get the number of years you were expecting out of the appliances.
• #17: And the 3rd possibility is that we overestimated, and we never hit the 5-year target. So in this example it turns out that we could have gotten along just fine with a much smaller, and cheaper set of appliances [advance], so we overpaid for the entire 5 years.
• #18: When using a software load balancer the story is completely different. You purchase the commodity hardware you need for your current load and for some future period of your choosing, perhaps a just a few months, or even less, especially if you are deploying in a cloud, where of course you don’t purchase any hardware and you may just worry about the load for today. So we no longer have to worry about estimating our load in 5 years time, which as we all know, is an awfully long time when it comes to technology. If you find that you have hit the limit of your hardware, you can simply replace one piece of commodity hardware with another. Most enterprises have a set of standard commodity hardware for various uses, such as web servers, application servers, data base servers, etc. This means that it is easy to upgrade the commodity hardware, and in the case of NGINX Plus, you don’t need to come back to us to do this upgrade, you are free to do it on you own. The server you were using for your ADC can now be repurposed for something else, such as a web server. So we can deploy the resources we need for this initial load [advance]. To keep the example simple, I’ve just included two upgrade points, but in reality, you could upgrade much more frequently. So lets say that we find that we have outgrown our hardware in 3 years, we do a simple upgrade to a new pair of commodity hardware boxes [advance], keeping the existing software and repurposing the hardware we were using for something else. Then 2-years later we upgrade again [advance]. Another option would to be to go with a scale-out model and we could add additional load balancers as our load increased. While this is possible with some hardware load balancers, in many cases there are caveats and issues that come with using more then a pair of hardware appliances, so in practice most companies stick with pairs.
• #19: When you purchase a hardware ADC you are buying a fixed amount of capacity. The Cost is based on licensed bandwidth, licensed SSL etc. And you are limited to the available models from that vendor. Each vendor has many different hardware appliances, usually all offering the same functionality, but differing in their performance (bandwidth, compression, SSL), the hardware (HDD or SSD), network ports etc. So it is hard to right-size a machine to exactly fit your needs. And to a certain extent, the same holds for virtual appliances that are sold by traditional hardware vendors - they are software-limited to match the pricing models of the equivalent hardware appliances. The vendor’s business model is based on upselling you to the most expensive device that meets all your requirements. With a software load balancer you get unlimited capacity. It will run as fast as the server you deploy it on. It is decoupled from the hardware, so can start with the box you need and easily add more capacity, whether it be adding memory, SSDs, more network ports, etc. And it is much more cost-effective to upgrade when you are using commodity hardware. And NGINX Plus does not come with any performance limits at all, so you are free to pass as much traffic through an NGINX Plus instance as you need.
• #20: Another aspect of having to choose a hardware appliance model, is that because you aren’t able to tailor the machine to meet your needs, unless your needs just happen to match exactly with one the available models, you will wind up with a model that is over provisioned in one or more areas. In general as you move up the product line of a hardware ADC vendor, the amount of CPU and bandwidth will increase, so if for example you need a large amount of CPU but not a lot of bandwidth you will have to buy a large model to get the CPU resources you need and that will likely come with more bandwidth then you need. And because different features of an ADC cause different loads on the machine, you may see far lower performance then the rated performance of the device. This graph shows the expected throughput of one particular hardware model, based on a vendor’s own data, as you enable various features. You see that for this model the maximum throughput is 10 Gig, but as you enable SSL and compression this drops to around 8 Gig and if you turn on the web application firewall it drops to around 4 Gig. So you are paying for more hardware then you need and you will also be paying more to power and cool it. With a software-based ADC, since it can run on a wide range of commodity hardware, you can install it on hardware tailor made to your needs, so if you need a machine with a large amount of CPU but a small amount of bandwidth, you can get a server to match those needs. In this case, if you need a box that can handle 4 Gig, you can buy a box with 4 Gig.
• #21: When discussing the previous two points, I was talking about fairly static environments, but an area where the software load balancer really shines is when it comes to being able to scale your environment. If you have very seasonal or event driven load patterns where you see large variations in your traffic, using a static deployment of your ADC’s means that you have to deploy enough resources all year to handle the spikes, so that for much of the year you are over provisioned. This is obviously not very cost effective. Using a software-based approach you can easily scale up or scale out the ADC layer so that you have the amount of resources you need when you need them and scale it back down when the extra capacity is no longer needed. This is an area where virtualization can be especially useful, so using a virtual ADC can help, but a software ADC is much more DevOps friendly and so allows for more deployment flexibility. And in addition NGINX Plus comes without any performance limitations.
• #22: Here we have an example of a site that has its busiest time around Christmas. The blue line is the amount of throughput during the year and the red line is the amount of throughput we want to support during the non-peak times of the year. In this example [advance] an active/passive pair of software load balancers can handle the non-peak load and we can scale-out [advance] to 5 active and 1 passive load balancers during the peak times and then scale back down [advance] after the peak season has passed. Again, since we are using commodity hardware, it is quite feasible to provision some servers as load balancers for just part of the year and use those servers for something else the rest of the year, or use the servers as load balancers for other applications. This is a case where virtual machines can also be used. This type of scalability is not feasible with hardware appliances because the scale out model is often not practical and even if it is, it is cost prohibitive to have a set of expensive hardware appliances sitting idle for much of the year since they can’t be repurposed for anything else. In this example, I showed a scale-out model, but you could scale-up the load balancers for the peak season.
• #23: Now lets talk about the deployment flexibility of software load balancers. If you need to deploy a new software load balancer, it is as simple as getting a bare metal server, a virtual machine or just an existing Linux or FreeBSD server with spare capacity and installing the software. Since it is running on commodity hardware, most enterprises have a set of servers in reserve, so getting a server can be quick, making this process very fast and NGINX Plus does not require a physical license file, removing another potential roadblock. Few enterprises can afford to purchase hardware-based ADCs to keep in reserve. Also, most companies have started using DevOps to one degree or another. Using DevOps tools, such as Puppet or Chef, makes the installation and configuration of NGINX Plus even easier. Of course this is not possible with a hardware appliance, but it is also much more difficult when dealing with pre-built virtual appliances.
• #24: So lets talk about the different process you go through when deploying a hardware load balancer versus a software load balancer. With a hardware load balancer you place your order and then you [advance] wait while the appliance is [advance] shipped to you and then you [advance] take delivery of the appliance and [advance] now you can install and configure it.
• #25: The process for ordering a software load balancer starts the same by placing your order, but after that everything happens very quickly. You get a [advance] commodity server out of your inventory, go [advance] to the Internet or a local repository to [advance] download the software and then [advance] install and configure it.
• #26: Now I want to move on to discussing using load balancers to handle multiple applications. Any load balancer, hardware or software can support many applications simultaneously, but when I talk about multi-tenancy I mean providing some extra functionality to allow you to isolate and control applications while sharing the same machine. It turns out that handling true multi-tenancy is difficult and you almost always wind up having to deal with the noisy neighbor problem. That is that the settings or resource utilization for one application impacts other applications so that you wind up with resource contention or conflicts. This leads to a couple of negative side effects. One is that the network team that owns the hardware appliances will often limit the features they allow the application teams to use. The other is that you need to size the appliance to that it can handle the maximum load of every application, which leads to an over-provisioned, under committed machine, which is very wasteful.
• #27: Here I show a configuration where a pair of hardware load balancers is handling 8 applications, or tenants. To be sure that we don’t overload the machine, we need to size it so that it can handle all 8 tenants when they are at full capacity, which can lead us to deploy a much larger appliance then we need. But since we can’t easily scale the appliances we have to do this to be safe.
• #28: With software you can move to a multi-instance approach to multi-tenancy where each application or tenant gets its own set of isolated load balancers. There are a number of options available to provide this isolation such as using separate servers, virtual machines, containers or jails. These are simple and non-proprietary approaches, and allow you to give each application the resources it needs and they can each be scaled independently. And since each tenant now has an isolated set of load balancers you don’t have to worry about the noisy neighbor problem so there is no longer any need to limit the features that the application teams can use, allowing you get the most out of your load balancers and your applications.
• #29: Here I show a configuration where each application or tenant gets its own set of NGINX Plus instances. These can be bare metal servers, virtual machines, containers, etc. and I’ve shown how they can have a different number or different sizes of machines. Now we can scale each application independently and get full use out of the load balancers.
• #30: Something else to consider is that hardware appliance vendors are limited to what they can offer. The engineering required to test each new chipset or new chip architecture and to support a wide array of platforms is substantial and forces each vendor to limit the number of choices available and also adds substantial lead time to bringing new technologies to market. When using a software ADC you can find the latest and greatest hardware to run it on, hardware that is almost always more current then anything available in a hardware appliance. This is especially true when dealing with new chip architectures like Atom or ARM. Porting software written for Linux onto a new chip architecture also running Linux is usually not a huge effort, but it is a much bigger effort for a hardware appliance vendor to develop and support a line of appliances using a new architecture and a proprietary operation system. And who knows what new technologies we will see in the future, but you can be sure that a software ADC like NGINX Plus will be available there before there is a hardware ADC available.
• #31: Now you may hear some objections to using a software load balancer. People may claim that you can’t get high performance out of a software-based solution. This objection usually revolves around two sets of specialized chips that many hardware appliances have. One is that many hardware appliances have custom chips for offloading SSL and the other is that many hardware appliances have custom ASIC’s – specialized chips for offloading some TCP handling. First, let’s be clear that these hardware appliances are just servers with a customized OS, usually based on Linux or FreeBSD, so that except for these special chips they are not too different from a commodity server, but they do cost more. Now with regards to SSL, it is certainly true that if you take a given server and add a SSL offload chip, there will be less load on the CPU for that server, however when you look at the cost of these chips you will find that they are far more expensive then a regular Intel or AMD CPU, so when looking at the cost per SSL request, it is cheaper to add Intel or AMD cores to a machine then it is to purchase an SSL offload card. In addition, more features are being added onto CPU’s, for example modern CPU’s have cryptographic functions included. With regards to the TCP chips, it is important to note that these chips are often not used for normal, real world web traffic processing, so if you are using the ADC to handle web traffic, you are often paying for these chips but aren’t getting any use out of them. In addition, modern network cards already have TCP optimizations that software can leverage. A simple example is the hardware calculation of checksums. A more modern example is virtualization optimized NICs (providing tight integration with the hypervisor), interrupt coalescing, pinning TCP streams to CPUs, etc. And finally, if you find that you really do need SSL or TCP offload cards, you can always add them to a commodity server and use them with NGINX Plus, so you can build your own equivalent of a hardware appliance, using commodity parts for a lower price. To look at performance from another angle, remember the right sizing I talked about earlier. The fact that you can create a machine to meet your needs means you can install the software on the biggest machine you can find, with as much CPU, memory, networking and storage as you need, or you can put it on a small server or virtual machine, but also with just the resources you need, and no more, and the cost will always be lower then with a proprietary hardware appliance. [advance] So you really can get the performance you need out of a software load balancer. And a final note: never assume that you will be able achieve the performance listed in spec sheets. All do testing in your environment to see what actual performance you can achieve in the real world.
• #32: This webinar is focused on the why you should consider choosing a software load balancer and although I have mentioned NGINX Plus, I haven’t said much about it. So let me take just a minute to give you a high level overview of NGINX Plus. NGINX Plus is built on top of the the well known NGINX Open Source project, currently in use by over 149 million web domains and 40% of the 10,000 busiest websites. It is unique in that it is a enterprise grade web server and load balancer and caching server, all in one lean, high performance package. And with the NGINX Plus commercial release it is also a feature rich Application Delivery Controller.
• #33: In summary, choosing a software load balancer lets you deploy what you need, where you need it, when you need it. By that I mean you can deploy just the right amount of resources you need for your application and you don’t have to worry about estimating your future load requirements and overprovisioning for the future. And you can run a software load balancer anywhere, on a bare metal server, in a virtual machine, in a container or in the cloud. And you can easily scale up or scale out the load balancer to handle variations in traffic. Of course having this flexibility is not going to do you any good if the load balancer doesn’t have the features you need and that is why we came out with NGINX Plus, adding ADC features to the already high performance NGINX Open Source tool.
• #34: Now we will open the floor for questions.
• #35: Thank you for taking the time to attend this session. I hope it was of use to you and I hope you attend some of our future webinars. You will find recordings of previous webinars at nginx.com and you will also blog posts there. I’ll take advantage of this time to plug the blog post I just published on different deployment scenarios for NGINX Plus. And it is easy to test out either the open source version of NGINX or NGINX Plus.