Note: The following content references Chapter 6 of “Building High-Availability Linux Servers” by Fuqin Zhujiu.
Setting up a load-balanced high-availability environment is relatively simple; the key is to understand the underlying principles. This article describes the advantages and disadvantages of three load balancers so you can make informed choices based on your needs in real-world production deployments.
Currently, the more commonly used load balancer hardware in online environments is the F5 BIG-IP. Software options include LVS, Nginx, and HAProxy, while high-availability software includes Heartbeat and Keepalived. Mature architectures include LVS+Keepalived, Nginx+Keepalived, HAProxy+keepalived, and DRBD+Heartbeat.
The advantages and disadvantages of the three load balancers are as follows:
LVS Advantages:
1. Strong load resistance, operates at Layer 4 only for distribution purposes, and generates no traffic itself. This characteristic makes it the strongest performer among load balancing software; having no traffic ensures the balancer’s IO performance is not affected by large traffic volumes.
2. Stable operation, with complete dual-machine hot-standby solutions, such as LVS+Keepalived and LVS+Heartbeat.
3. Wide application range; it can perform load balancing for all applications.
4. Low configurability, which is both a disadvantage and an advantage. Because there isn’t much to configure, it doesn’t require extensive interaction, greatly reducing the chance of human error.
LVS Disadvantages:
1. The software itself does not support regular expression processing and cannot separate dynamic and static content, which highlights the advantage of Nginx/HAProxy+Keepalived.
2. If the website application is relatively large, LVS/DR+Keepalived becomes more complex, especially when Windows Server machines are involved in the backend. The implementation, configuration, and maintenance processes become more cumbersome. Relatively speaking, Nginx/HAProxy+Keepalived is much simpler.
#############################################################
1. How Does LVS/DR Handle Request Packets, and Does It Modify IP Packet Contents?
1.1 vs/dr itself does not concern itself with information above the IP layer; even port numbers are verified by the TCP/IP protocol stack. vs/dr itself mainly does the following:
1) Receives client requests and selects the IP of a realserver based on your configured load balancing algorithm.
2) Uses the MAC address corresponding to the selected IP as the destination MAC, then re-encapsulates the IP packet into a frame and forwards it to this RS.
3) Records the connection information in the hash table.
vs/dr does very little and is very simple, so its efficiency is very high, not much inferior to hardware load balancing devices.
The approximate flow of data packets and frames is: client –> VS –> RS –> client
1.2 As answered previously, vs/dr does not modify the contents of IP packets.
2. Why Configure the VIP on the lo Interface on RealServer? Can It Be Configured on the Egress Network Card?
2.1 To enable RS to process IP packets destined for the VIP, RS must first be able to receive this packet.
Configuring the VIP on the lo interface accomplishes receiving the packet and returning the result to the client.
2.2 The answer is that you cannot configure the VIP on the egress network card; otherwise, it will respond to the client’s ARP requests, causing disorder in the client/gateway ARP table, and the entire load balance will fail to work properly.
3. Why Suppress ARP Frames on RealServer?
This question has already been explained in the previous point; here we elaborate further with implementation commands. During actual deployment, we make the following adjustments:
echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce
I believe many people don’t quite understand what these do; they just know they are required. I won’t dive into a detailed discussion here, but just add a few explanatory notes as a supplement.
3.1
echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignoreecho "2" >/proc/sys/net/ipv4/conf/lo/arp_announce
These two lines are optional, because ARP is meaningless for the loopback interface.
3.2 If your Real Server’s external network interface is eth0, then
echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignoreecho "2" >/proc/sys/net/ipv4/conf/all/arp_announce
What you actually need to execute is:
echo "1" >/proc/sys/net/ipv4/conf/eth0/arp_ignoreecho "2" >/proc/sys/net/ipv4/conf/eth0/arp_announce
So I personally recommend adding the above two lines to your script as well, because if the default system values for those settings are not 0, it could potentially cause problems.
4. Why Must the LVS/DR Load Balancer (Director) and RS Be in the Same Network Segment?
From the first question, you should understand how VS/DR forwards requests to the RS. It operates at the data link layer, so the director must be in the same network segment as the RS.
5. Why Does the Director Need an IP (DIP) on eth0 Besides the VIP on lo?
5.1 If you are using tools like keepalived for HA or Load Balancing, the DIP is required for health checks.
5.2 An HA or Load Balancing setup without a health check mechanism has no practical purpose.
6. Does ip_forward Need to Be Enabled for LVS/DR?
No, it doesn’t. Because the director and the real server are on the same network segment, forwarding does not need to be enabled.
7. Must the Director’s VIP Netmask Be 255.255.255.255?
In LVS/DR, the director’s VIP netmask does not need to be set to 255.255.255.255, nor is there a need to run
route add -host $VIP dev eth0:0
The director’s VIP is supposed to be advertised externally like any normal IP address, so don’t make it overly special.
8. How Does LVS/DR Handle the TCP Three-Way Handshake?
#####################################################################
Nginx Advantages:
1. Operates at OSI Layer 7, allowing traffic distribution strategies tailored to HTTP applications, such as routing by domain name or directory structure. Its regular expression capabilities are more powerful and flexible than HAProxy’s.
2. Nginx has very low network dependency; theoretically, if a server is pingable, it can handle load balancing, which is another of its strengths.
3. Nginx installation and configuration are relatively simple, making testing more convenient.
4. It can withstand high load pressure and remain stable, generally supporting over tens of thousands of concurrent connections.
5. Nginx can detect backend server failures via port checks, such as by analyzing the status codes returned from processing web pages, timeouts, etc., and will resubmit failed requests to another node.
6. Nginx is not just an excellent load balancer/reverse proxy; it is also a powerful Web application server. The LNMP stack is now a very popular web environment, rivaling the LAMP stack. Nginx holds an advantage over Apache in handling static pages, especially regarding high concurrency.
7. Nginx is now increasingly mature as a Web reverse acceleration cache, offering speeds faster than traditional Squid servers. Those needing such functionality may consider using it as a reverse proxy accelerator.
Nginx Disadvantages:
1. Nginx does not support health checks via URL detection.
2. Nginx only supports HTTP and Email protocols, which is a limitation.
3. Nginx’s session persistence and cookie-based routing capabilities are relatively lacking.
HAProxy Advantages:
1. HAProxy supports virtual hosts and can operate at both Layer 4 and Layer 7 (supports multiple network segments).
2. It can compensate for some of Nginx’s weaknesses, such as session persistence and cookie-based routing.
3. Supports backend server health checks via URL detection.
4. Like LVS, it is solely a load-balancing software. In terms of raw efficiency, HAProxy offers superior load-balancing speed compared to Nginx and also excels in concurrent connection handling.
5. HAProxy can load balance MySQL reads, performing health checks and distributing load across backend MySQL nodes. However, when the number of backend MySQL slaves exceeds 10, its performance falls behind LVS.
6. HAProxy offers a rich set of algorithms, totaling 8 different types.