FastDFS is an open-source distributed file system designed for file management. Its features include file storage, file synchronization, and file access (upload and download), solving the problems of large-capacity storage and load balancing. It is particularly suitable for file-based online services such as photo album websites, video websites, and more.
The FastDFS server side has two roles: the tracker and the storage node. The tracker primarily handles scheduling and plays a role in load balancing for access.
Storage nodes store files and perform all file management functions: storage, synchronization, and providing access interfaces. FastDFS also manages file metadata. File metadata refers to the related attributes of a file, expressed as key-value pairs, for example: width=1024, where the key is width and the value is 1024. File metadata is a list of file attributes that can contain multiple key-value pairs.
The FastDFS system architecture is shown below:
Both trackers and storage nodes can consist of one or multiple servers. Servers in the tracker and storage node clusters can be added or taken offline at any time without affecting online services. All servers within a tracker are peers and can be added or removed based on server load pressure at any time.
To support large capacity, storage nodes (servers) adopt a volume (or group) organizational structure. The storage system consists of one or more volumes. Files between volumes are independent of each other, and the total file capacity across all volumes represents the entire file capacity of the storage system. A volume can consist of one or more storage servers. Files across storage servers within a volume are identical, providing redundancy backup and load balancing.
When adding a server to a volume, the system automatically synchronizes existing files. Once synchronization is complete, the system automatically switches the newly added server online to provide services.
When storage space is insufficient or nearly exhausted, volumes can be dynamically added. Simply adding one or more servers and configuring them as a new volume expands the storage system’s capacity.
File identification in FastDFS consists of two parts: the volume name and the file name; both are indispensable.
FastDFS file upload
File upload interaction process:
1. The client asks the tracker for a storage node to upload to, with no additional parameters needed;
2. The tracker returns an available storage node;
3. The client communicates directly with the storage node to complete the file upload.
FastDFS file download
File download interaction process:
1. The client asks the tracker for a storage node to download the file from, using the file identifier (volume name and file name) as the parameter;
2. The tracker returns an available storage node;
3. The client communicates directly with the storage node to complete the file download.
It should be noted that the client is the caller using the FastDFS service. The client should also be a server, and its calls to the tracker and storage are all server-to-server calls.