<<
This article primarily introduces the architecture of LVS clusters. It first presents the general architecture of LVS clusters, discusses its design principles and corresponding characteristics, and finally applies LVS clusters to build scalable network services such as Web, Media, Cache, and Mail.
1. Introduction
Over the past decade, the Internet has evolved from a network connecting a few research institutions for information sharing into a global network hosting a vast array of applications and services, becoming an indispensable part of people’s lives. Although the Internet has grown rapidly, building and maintaining large-scale network services remains a challenging task because the system must be high-performance and highly reliable. Especially as access loads continue to grow, the system must be able to scale to meet increasing performance demands. The lack of frameworks and design methodologies for building scalable network services means that only organizations with exceptional engineering and management talent can build and maintain large-scale network services.
To address this situation, this article first presents the general architecture of LVS clusters, discusses its design principles and corresponding characteristics, and finally applies LVS clusters to build scalable network services such as Web, Media, Cache, and Mail.
2. General Architecture of LVS Clusters
LVS clusters utilize IP load balancing technology and content-based request distribution technology. The scheduler has excellent throughput, distributing requests evenly to different servers for execution, and automatically masks server failures, thus forming a group of servers into a high-performance, highly available virtual server. The entire server cluster structure is transparent to clients, and no modifications are required to client-side or server-side programs.
Figure 1: Architecture of LVS Cluster
Therefore, transparency, scalability, high availability, and ease of management need to be considered during design. Generally, an LVS cluster adopts a three-tier structure, as shown in Figure 1. The three main components are:
Load Balancer: This is the front-end machine of the entire cluster facing the outside world, responsible for forwarding client requests to a group of servers for execution, while clients perceive the service as originating from a single IP address (which we can call the Virtual IP Address).
Server Pool: A group of servers that actually execute client requests, providing services such as WEB, MAIL, FTP, and DNS.
Shared Storage: Provides a shared storage area for the server pool, making it easy for servers in the pool to have the same content and provide the same services.
The scheduler is the Single Entry Point for the server cluster system. It can use IP load balancing technology, content-based request distribution technology, or a combination of both. In IP load balancing technology, the server pool needs to have identical content to provide the same services. When a client request arrives, the scheduler selects a server from the pool based solely on server load conditions and a configured scheduling algorithm, forwards the request to the selected server, and records this scheduling decision; subsequent packets belonging to this request are also forwarded to the previously selected server. In content-based request distribution technology, servers can provide different services. When a client request arrives, the scheduler can choose a server to execute the request based on the request’s content. Because all operations are completed within the core space of the Linux operating system, its scheduling overhead is minimal, resulting in high throughput.
The number of nodes in the server pool is variable. When the total load received by the system exceeds the processing capacity of all current nodes, servers can be added to the server pool to meet the growing request load. For most network services, there is no strong correlation between requests, and requests can be executed in parallel on different nodes, so the overall system performance can essentially increase linearly with the number of nodes in the server pool.
Shared storage is typically a database, network file system, or distributed file system. Data that server nodes need to dynamically update is generally stored in database systems, which ensure data consistency during concurrent access. Static data can be stored in network file systems (such as NFS/CIFS), but the scalability of network file systems is limited; generally, an NFS/CIFS server can only support 3~6 busy server nodes. For larger-scale cluster systems, distributed file systems can be considered, such as AFS[1], GFS[2.3], Coda[4], and Intermezzo[5]. Distributed file systems can provide a shared storage area for each server, allowing them to access the distributed file system as if it were a local file system, while also offering good scalability and availability. Furthermore, when applications on different servers simultaneously read and write the same resource on the distributed file system, access conflicts need to be resolved to maintain a consistent state for the resource. This requires a Distributed Lock Manager, which may be provided internally by the distributed file system or externally. Developers can use the distributed lock manager when writing applications to ensure consistency during concurrent access across different nodes.
The load scheduler, server pool, and shared storage system are interconnected via high-speed networks, such as 100Mbps switched networks, Myrinet, and Gigabit networks. Using high-speed networks primarily avoids the interconnection network becoming a bottleneck as the system scale expands.
The Graphic Monitor is a monitor provided for system administrators to oversee the entire cluster system; it can monitor the system’s status. The Graphic Monitor is browser-based, so administrators can monitor system conditions whether locally or remotely. For security reasons, the browser must use the HTTPS (Secure HTTP) protocol and pass identity authentication before system monitoring, configuration, and management can be performed.
2.1. Why Use a Layered Architecture
A layered architecture allows layers to be independent of each other, with each layer providing different functions, enabling the reuse of different existing software within a layer. For example, the scheduler layer provides load balancing, scalability, and high availability, while the server layer can run various network services like Web, Cache, Mail, and Media to provide different scalable network services. Clear functional division and a distinct layered structure make the system easy to build, easy to maintain later, and facilitate easy system performance expansion.
2.2. Why Shared Storage
Shared storage, such as a distributed file system, is optional in this LVS cluster system. When network services require identical content, shared storage is an excellent choice; otherwise, each server would need to replicate the same content onto its local hard drive. The larger the amount of content stored in the system, the higher the cost of this Shared-nothing Structure, because each server requires equally large storage space, any update must involve every server, and the system maintenance cost becomes very high.
Shared storage provides a unified storage space for the server group, making content maintenance relatively easy. For instance, a webmaster only needs to update pages in shared storage for the changes to take effect across all servers. Distributed file systems offer good scalability and availability; when the storage space of the distributed file system increases, the storage space of all servers increases accordingly. For most Internet services, which are read-intensive applications, using local hard drives as Cache (e.g., 2GB of space) on each server can make access speeds from the distributed file system approach local hard disk access speeds.
Furthermore, advances in storage hardware technology are driving a migration from shared-nothing clusters to shared-storage clusters. Storage Area Network (SAN) technology enables each node in the cluster to directly connect/share a massive hard disk array. Hardware vendors also provide various hard disk sharing technologies, such as Fiber Channel and Shared SCSI. InfiniBand is a universal high-performance I/O specification that allows lower latency transmission of I/O messages and cluster communication messages within the Storage Area Network, providing excellent scalability. InfiniBand is supported by most major vendors, such as Compaq, Dell, Hewlett-Packard, IBM, Intel, Microsoft, and SUN Microsystems, and is becoming an industry standard. The development of these technologies makes shared storage easier, and mass production will gradually reduce costs.
2.3. High Availability
A characteristic of cluster systems is their redundancy in both software and hardware. High system availability can be achieved by detecting node or service process failures and correctly resetting the system, ensuring that incoming requests can be handled by surviving nodes.
Typically, a resource monitoring process runs on the scheduler to constantly monitor the health status of each server node. When a server is unreachable via ICMP ping or its network service does not respond within a specified time, the resource monitoring process notifies the operating system kernel to remove or disable that server from the scheduling list. Thus, new service requests will not be dispatched to the faulty node. The resource monitoring process can report failures to the administrator via email or pager. Once the monitoring process detects that the server has resumed operation, it notifies the scheduler to add it back to the scheduling list. Additionally, through the management program provided by the system, administrators can issue commands at any time to add new machines into service to enhance system processing performance, or to remove existing servers from service for system maintenance.
Currently, the front-end scheduler could potentially become a Single Point of Failure for the system. Generally, the reliability of the scheduler is high because it runs fewer programs and most of those programs have been thoroughly tested, but we cannot rule out major faults such as hardware aging, network line issues, or human operational errors. To prevent the entire system from failing due to scheduler failure, we need to set up a backup scheduler as a standby for the primary scheduler. Two Heartbeat processes [6] run on the primary and backup schedulers respectively, periodically reporting their health status to each other via heartbeat lines such as serial cables and UDP. When the backup scheduler cannot hear the primary scheduler’s heartbeat, it uses Gratuitous ARP to take over the cluster’s external Virtual IP Address, simultaneously taking over the primary scheduler’s load scheduling duties. When the primary scheduler recovers, there are two approaches: one is for the primary scheduler to automatically become the backup scheduler; the other is for the backup scheduler to release the Virtual IP Address, allowing the primary scheduler to reclaim it and resume providing load scheduling services. Here, multiple heartbeat lines can minimize the probability of misjudgment due to heartbeat line failure (i.e., the backup scheduler thinking the primary scheduler has failed when it is actually working normally).
Usually, when the primary scheduler fails, all established connection state information on the primary scheduler is lost, and existing connections will be interrupted. Clients need to reconnect, and only then will the backup scheduler dispatch the new connections to the servers, causing some inconvenience to clients. For this reason, the IPVS scheduler implements an efficient state synchronization mechanism in the Linux kernel, synchronizing the primary scheduler’s state information to the backup scheduler in a timely manner. When the backup scheduler takes over, the vast majority of established connections will persist.
3. Scalable Web Service
The architecture of an LVS-based Web cluster is shown in Figure 2: The first layer is the load scheduler, typically using IP load balancing technology to achieve high system throughput; the second layer is the Web server pool, where each node can run HTTP services or HTTPS services, or both; the third layer is shared storage, which can be a database, a network file system, a distributed file system, or a mixture of the three. Nodes in the cluster are connected via a high-speed network.
For static pages and files (such as HTML documents and images), they can be stored in a network file system or a distributed file system. The choice depends on the system’s scale and requirements. Through a shared network file system or distributed file system, the webmaster sees a unified document storage space, making page maintenance and updates more convenient. Modifications to pages in shared storage take effect for all servers.
Under this structure, when all server nodes become overloaded, administrators can quickly add new server nodes to handle requests without needing to copy Web documents to the nodes’ local hard drives.
Some Web services may use HTTP cookies, a mechanism that stores data in the client’s browser to track and identify the client. When HTTP cookies are used, different connections from the same client have dependencies; these connections must be sent to the same Web server. Some Web services use the secure HTTPS protocol, which is the HTTP protocol plus the SSL (Secure Socket Layer) protocol. When a client accesses an HTTPS service (the default port for HTTPS is 443), an SSL connection is first established to exchange certificates for symmetric public-key encryption and negotiate an SSL Key to encrypt subsequent sessions. Within the lifecycle of the SSL Key, all subsequent HTTPS connections use this SSL Key, so different HTTPS connections from the same client also have dependencies. To address these needs, the IPVS scheduler provides a persistent service function, which ensures that within a set period, different connections from the same IP address are sent to the same server node in the cluster, effectively solving the problem of client connection dependencies.
4. Scalable Media Services
The architecture of an LVS-based media cluster is shown in Figure 3: The first layer is the load scheduler, generally using IP load balancing technology, which can give the entire system high throughput; the second layer is the Web server pool, where corresponding media services can run on each node; the third layer is shared storage, storing media programs via a network file system/distributed file system. The nodes in the cluster are interconnected through a high-speed network.
Figure 3: LVS-based Media Cluster
The IPVS load scheduler generally uses the Direct Routing method (i.e., the VS/DR method, to be detailed in a future article) to construct the media cluster system. The scheduler distributes media service requests relatively evenly across servers, while the media servers return response data directly to the client, giving the entire media cluster system excellent scalability.
Media servers can run various media service software. Currently, LVS clusters offer excellent support for Real Media, Windows Media, and Apple Quicktime media services, with real systems running in production. Generally, streaming media services use a TCP connection (such as the RTSP protocol: Real-Time Streaming Protocol) for bandwidth negotiation and flow control, and return stream data to the client via UDP. Here, the IPVS scheduler provides functionality to consider TCP and UDP together, ensuring that media TCP and UDP connections from the same client are forwarded to the same media server in the cluster, allowing media services to proceed accurately.
Shared storage is the most critical issue in a media cluster system, because media files are often very large (a single film may require hundreds of megabytes to several gigabytes of storage space), demanding high storage capacity and read speed. For smaller-scale media cluster systems, for example, those with 3 to 6 media server nodes, the storage system can consider using a Linux server with a Gigabit network card, employing software RAID and a journaling file system, and then running the kernel’s NFS service, which yields good results. For larger-scale media cluster systems, it is best to choose a distributed file system that has good support for file striping storage and file caching; media files are stored in segments across multiple storage nodes in the distributed file system, improving file system performance and load balancing among storage nodes; media files are automatically cached on the media servers, increasing file access speed. Otherwise, one could consider developing corresponding tools on the media servers themselves, such as a caching tool that periodically calculates recent hot media files, copies hot files to the local hard drive, replaces non-hot files in the cache, and finally notifies other media server nodes about the media files it has cached and its load status; on the media server, there is an application-layer scheduling tool that receives a client’s media service request. If the requested media file is cached locally, it is directly passed to the local media service process; otherwise, it first considers whether the file is cached by other media servers. If the file is cached by another server and that server is not busy, the request is forwarded to the media service process on that server for handling; otherwise, it is directly passed to the local media service process, which reads the media file from the backend shared storage.
The benefit of shared storage is that media file administrators see a unified storage space, making media file maintenance work more convenient. When customer access continuously increases, causing the entire system to overload, administrators can quickly add new media server nodes to handle requests.
Real is famous for its high-compression-ratio audio/video formats, Real Media Server, and media player RealPlayer. Real is using the above structure to build an LVS scalable Web and media cluster composed of over 20 servers to provide Web and audio/video services to its global users. Real’s senior technical director claimed that LVS defeated all commercial load balancing products they had tried [7].
5. Scalable Cache Services
An effective network cache system can significantly reduce network traffic, lower response latency, and decrease server load. However, if a cache server is overloaded and cannot process requests in time, it will instead increase response latency. Therefore, the scalability of cache services is very important; when the system load continuously grows, the entire system can be expanded to improve the processing capacity of the cache service. Especially, cache services on backbone networks may require several Gbps of throughput, which a single server (for example, SUN’s current highest-end Enterprise 10000 server) is far from achieving. It’s clear that implementing scalable cache services through a PC server cluster is a very effective method, and also the method with the best price-performance ratio.
The architecture of an LVS-based Cache cluster is shown in Figure 4: The first layer is the load scheduler, generally using IP load balancing technology, which can give the entire system high throughput; the second layer is the Cache server pool, generally placed near the backbone Internet connection points, which can be distributed in different networks. There can be multiple schedulers, placed close to the clients.
Figure 4: LVS-based Cache Cluster
The IPVS load scheduler generally uses the IP Tunneling method (i.e., the VS/TUN method, to be detailed in a future article) to construct the Cache cluster system, because Cache servers may be placed in different locations (for example, near backbone Internet connection points), and the scheduler and the Cache server pool may not be in the same physical network. Using the VS/TUN method, the scheduler only schedules Web Cache requests, while Cache servers return response data directly to the client. In cases where the requested object cannot be found locally, the Cache server must send a request to the origin server, retrieve the result, and finally return the result to the client; if using a commercial scheduler based on NAT technology, four traversals in and out of the scheduler are required to complete this request. With the VS/TUN method (or VS/DR method), the scheduler only schedules one request, and the other three traversals are completed by the Cache server directly accessing the Internet. Therefore, this method is particularly effective for Cache cluster systems.
Cache servers use local hard drives to store cacheable objects, because storing cacheable objects involves write operations, which account for a certain proportion, and local hard drives can improve I/O access speed. Cache servers share a dedicated multicast channel and exchange information via the ICP protocol (Internet Cache Protocol). When a Cache server misses a current request in its local hard drive, it can query other Cache servers via ICP to see if they have a copy of the requested object. If a copy exists, it fetches the object copy from the neighboring Cache server, which can further improve the hit rate of the Cache service.
The UK national JANET Web Cache network, serving over 150 universities and regions, implemented a scalable Cache cluster using the above LVS structure in November 1999 [8], using only half of the original 50+ independent Cache servers. Users reported that the network speed felt as fast as in summer (when students are on summer vacation). This shows that load scheduling can smooth out the access bursts of a single server and improve the resource utilization of the entire system.
6. Scalable Mail Services
With the continuous growth of Internet users, many ISPs face the problem of their mail servers being overloaded. When a mail server can no longer accommodate more user accounts, some ISPs buy higher-end servers to replace the original ones. Migrating the original server’s information (such as user mail) to the new server is a tedious task that can cause service interruptions. Some ISPs set up new servers and new mail domains, placing new mail users on new servers. For example, Shanghai Telecom currently uses different mail servers like public1.sta.net.cn, public2.sta.net.cn through public9.sta.net.cn to host user mail accounts. This static partitioning of users onto different servers leads to unbalanced mail server loads and low system resource utilization, and for users, the email addresses are relatively hard to remember.
The LVS framework can be used to build a highly scalable and highly available mail service system. Its architecture is shown in Figure 5: the front end is a load balancer using IP load balancing technology; the second layer is the server pool, consisting of LDAP (Light-weight Directory Access Protocol) servers and a group of mail servers. The third layer is data storage, which stores user mail through a distributed file system. Nodes in the cluster are connected via high-speed networks.
User information such as usernames, passwords, home directories, and mail quota limits are stored in LDAP servers, and administrators can manage users via HTTPS. SMTP (Simple Mail Transfer Protocol), POP3 (Post Office Protocol version 3), IMAP4 (Internet Message Access Protocol version 4), and HTTP/HTTPS services run on each mail server. SMTP accepts and forwards user mail; the SMTP service process queries the LDAP server to obtain user information and then stores the mail. POP3 and IMAP4 obtain user information through the LDAP server, and after password verification, process user mail access requests. Here, a mechanism is needed to avoid read-write conflicts on user mail between SMTP, POP3, and IMAP4 service processes on different servers. The HTTP/HTTPS service allows users to access mail through a web browser.
The IPVS scheduler distributes SMTP, POP3, IMAP4, and HTTP/HTTPS request flows to the mail servers in a fairly balanced manner. Judging from the processing flows of the services above, the result is the same regardless of which mail server the request is sent to. Here, running SMTP, POP3, IMAP4, and HTTP/HTTPS on each mail server and performing centralized scheduling helps improve the resource utilization of the entire system.
The potential bottleneck in the system is the LDAP server. Optimizing the B+ tree parameters in the LDAP service, combined with high-end servers, can achieve higher performance. If the distributed file system lacks a load balancing mechanism among multiple storage nodes, a corresponding mail migration mechanism is needed to avoid skewed mail access patterns.
In this way, the cluster system appears to users as a single high-performance, high-reliability mail server (for example, Shanghai Telecom could use just one mail domain public.sta.net.cn). As the number of mail users continues to grow, one only needs to add server nodes and storage nodes to the cluster. Centralized storage of user information simplifies user management, and the cluster system helps improve resource utilization.
7. Summary
This article presented the general architecture of an LVS cluster, discussed its design principles and corresponding characteristics, and finally applied the LVS cluster to build scalable Web, Media, Cache, and Mail network services, pointing out key considerations during system setup. We will explain the technology, implementation, and applications of LVS clusters in detail in subsequent articles.
References
- J.H. Howard. An Overview of the Andrew File System. In Proceedings of the USENIX Winter Technical Conference, Dallas, TX, USA, February 1998.
- Kenneth W. Preslan, Andrew P. Barry, Jonathan E. Brassow, Grant M. Erickson, Erling Nygaard, Christopher J. Sabol, Steven R. Soltis, David C. Teigland, and Matthew T. O'Keefe. A 64-bit, Shared Disk File System for Linux. In Proceeding of 16th IEEE Mass Storage Systems Symposium, San Diego, CA, USA. March 15-18, 1999.
- Global File System Website. http://www.globalfilesystem.org.
- Coda File System Website. http://www.coda.cs.cmu.edu.
- InterMezzo File System Website. http://www.inter-mezzo.org.
- Alan Robertson, et al. Linux High Availability Project. http://www.linux-ha.org.
- Jerry Glomph Black. LVS testimonials from Real Networks. March 2000. http://marc.theaimsgroup.com/?l=linux-virtual-server&m=95385809030794&w=2
- Michael Sparks. Load Balancing the UK National JANET Web Cache Service Using Linux Virtual Servers. November 1999. http://wwwcache.ja.net/JanetService/PilotService.html.
About the Author
Wensong Zhang, an open-source and Linux kernel developer, is the founder and principal developer of the well-known Linux clustering project — LVS (Linux Virtual Server). He currently works at the National Key Laboratory of Parallel and Distributed Processing, mainly engaged in research on clustering technology, operating systems, object storage, and databases. He spends a great deal of time developing free software and takes great pleasure in it.