<<
1. What Is Clustered NAS?
A cluster is a loosely coupled collection of computing nodes that work together to provide external services. Clusters are mainly categorized into High-Performance Clusters (HPC), High-Availability Clusters (HAC), and Load-Balancing Clusters (LBC). Clustered NAS refers to the coordination of multiple nodes (commonly known as NAS heads) to deliver high-performance, high-availability, or load-balanced NAS (NFS/CIFS) services.
Unstructured data is currently experiencing a trend of rapid growth. IDC research reports indicate that unstructured data will account for over 80% of total data storage by 2012. Clustered NAS is a scale-out storage architecture that offers the advantage of linear scaling of capacity and performance, and it has gained global market recognition. From acquisitions such as EMC’s purchase of Isilon, HP’s acquisition of IBRIX, and Dell’s buyout of Exanet, to the launches of IBM SONAS and NetApp Data ONTAP 8, it is evident that clustered NAS has become a mainstream storage technology. Domestically, we also see clustered NAS solutions like UIT UFS, LoongStore, CZSS, and YFS. The potential future market for clustered NAS is enormous, and it will gradually find widespread application in sectors such as High-Performance Computing (HPC), broadcasting IPTV, video surveillance, and cloud storage.
2. Three Mainstream Technical Architectures for Clustered NAS
From an overall architecture perspective, clustered NAS consists of a storage subsystem, a NAS cluster (heads), clients, and a network. The storage subsystem can utilize Storage Area Network (SAN), Direct-Attached Storage (DAS), or Object-Based Storage Device (OSD) architectures. SAN and DAS approaches require a storage cluster to manage the backend storage media and provide a standard file access interface for the NAS cluster via a SAN file system or clustered file system. In an OSD-based architecture, the NAS cluster manages metadata while clients directly interact with OSD devices for data access; this is parallel NAS, specifically pNFS/NFSv4.1. The NAS cluster acts as an NFS/CIFS gateway, providing standard file-level NAS services to clients. For SAN and DAS architectures, the NAS cluster handles both metadata and I/O data access functions, whereas the OSD architecture only needs to handle metadata access. Based on the different backend storage subsystems used, clustered NAS can be divided into three technical architectures: the SAN shared storage architecture, the clustered file system architecture, and the pNFS/NFSv4.1 architecture.
(1) SAN Shared Storage Architecture
This architecture (shown in Figure 1) uses SAN for backend storage. All NAS cluster nodes are connected to the SAN via Fibre Channel and share all storage devices. It typically employs a SAN parallel file system to manage and export a POSIX interface to the NAS cluster. A SAN parallel file system usually requires a metadata control server, which can be a dedicated MDC or distributed across SAN clients in a fully decentralized manner. Installing the SAN file system client on the NAS cluster enables concurrent access to the SAN shared storage, and then NFS/CIFS services are run to serve clients. Here, the front-end network uses Ethernet, while the backend storage connection utilizes a SAN network.
Figure 1: SAN Shared Storage Clustered NAS Architecture
Due to the high-performance SAN storage network, this clustered NAS architecture can provide stable high bandwidth and IOPS performance, and it allows for the independent scaling of storage capacity and performance by adding storage disk arrays or NAS cluster nodes. Clients can connect directly to a specific NAS cluster node, using cluster management software to achieve high availability; alternatively, DNS or LVS can be used for load balancing and high availability, with clients connecting via a virtual IP. Both the SAN storage network and the parallel file system are relatively costly, so the main disadvantage of this clustered NAS architecture is its high cost. It also inherits the drawbacks of the SAN storage architecture, such as complex deployment and management and limited scalability. Typical examples of clustered NAS using this architecture are IBM SONAS (Figure 2) and Symantec FileStore.
Figure 2: SONAS
(2) Clustered File System Architecture
This architecture (shown in Figure 3) uses DAS for backend storage, where each storage server is directly connected to its own storage system, typically a set of SATA disks. A clustered file system then uniformly manages the physically distributed storage space to form a single namespace file system. In effect, the clustered file system combines the functions of RAID, Volume Manager, and File System into one. Mainstream clustered file systems today generally require a dedicated metadata service or a distributed metadata service cluster to provide metadata control and a unified namespace, though there are exceptions, such as the metadata-server-free architecture of GlusterFS. Installing the clustered file system client on the NAS cluster enables access to the global storage space, and NFS/CIFS services are run to provide NAS services externally. The NAS cluster often runs on the same physical nodes as the metadata service cluster or storage node cluster, which reduces the scale of physical node deployment but may impact performance. Unlike the SAN architecture, the clustered file system might share the TCP/IP network with the NAS service, causing mutual performance interference and leading to I/O performance jitter. Clustered file system storage nodes like those from Isilon use InfiniBand networks for interconnection, which can eliminate this impact and maintain performance stability.
Figure 3: Clustered File System Clustered NAS Architecture
In this architecture, the scaling of clustered NAS is achieved by adding storage nodes, often expanding storage space and performance simultaneously. Many systems can achieve near-linear scaling. The method for clients to access the clustered NAS is the same as in the first architecture, and load balancing and availability can be implemented similarly. Because both servers and storage media can use standard, low-cost commodity hardware, there is a significant cost advantage, and the scale can be very large. However, such devices are highly prone to failure. Server or disk failures can render some data unavailable, requiring HA mechanisms to ensure server availability and replication to ensure data availability, which often reduces system performance and storage utilization. Additionally, due to the relatively large number of server nodes, this architecture is less suited for productization and may be more appropriate for storage solutions. Typical examples of clustered NAS using this architecture include EMC Isilon, LoongStore, CZSS, YFS, and GlusterFS (Figure 4).
Figure 4: GlusterFS Architecture
(3) pNFS/NFSv4.1 Architecture
This architecture (shown in Figure 5) is actually parallel NAS, specifically pNFS/NFSv4.1. The RFC 5661 standard was approved in January 2010. Its backend storage uses Object-Based Storage Devices (OSD) and supports multiple data access protocols like FC, NFS, and OSD. When reading or writing data, clients interact directly with OSD devices, unlike the two architectures above where data must transit through the NAS cluster. Here, the NAS cluster serves only as a metadata service, while I/O data is handled by the OSD, achieving the separation of metadata and data. This architecture is more akin to a native parallel file system; it is not only simpler in system architecture but also achieves a great performance boost and excellent scalability.
Figure 5: pNFS/NFSv4.1 Clustered NAS Architecture
Clearly, this architecture is fundamentally different from the previous two. pNFS uses a metadata cluster to solve the single point of failure and performance bottleneck problems of traditional NAS, while the separation of metadata and data solves the performance and scalability issues. This is true parallel NAS, and pNFS is the genuine future of clustered NAS. However, since the pNFS standard was approved only a year ago, there are currently no mature product implementations. OSD storage devices, despite years of development, have not yet gained widespread market recognition or adoption. Panasas’s PanFS (Figure 6) is likely the closest to this clustered NAS architecture, and Panasas was, of course, one of the main contributors to the pNFS standard. Many storage companies are currently developing pNFS products, such as BlueArc. The author predicts that products will be launched successively by 2012.
Figure 6: PanFS Architecture
3. Open-Source Solutions
The clustered NAS storage products or solutions mentioned above are mostly commercial implementations and are relatively expensive. Some users might want to use open-source software to build a clustered NAS. Are there such open-source solutions available? The core of clustered NAS is the underlying parallel file system, clustered file system, or the pNFS protocol. Below is a brief introduction to open-source support and implementation in the realm of clustered NAS.
(1) SAN Shared Storage Architecture: Red Hat GFS is an open-source SAN shared file system; it also supports DAS connectivity. Integrating NFS/Samba services with it can create a clustered NAS.
(2) Clustered File System Architecture: Lustre, Gluster, PVFS2, and Ceph are all excellent clustered file systems. Gluster itself is a complete clustered NAS system. Similar to Gluster’s implementation, clustered