Category: Files

There are many ways to learn Linux. The fastest way is hands-on practice. Understand the boot process, file system, commands, and configurations. Mastering Linux takes patience…

Before diving into the details of RAID solutions, let’s first understand the advantages of RAID, followed by a comprehensive overview of RAID solutions that we hope you’ll find useful. RAID comes in various forms, all serving as the backbone of high-availability and high-performance storage. The earliest applications of RAID devices date back to the 198 …

Web 2.0 websites generate data at an exponential rate, especially small files ranging from a few kilobytes to a few hundred kilobytes in massive quantities. Traditional file systems struggle to handle them effectively. Many sites encounter these issues during scaling: high disk I/O; backup difficulties; single points of failure, with the inability to horizontally scale capacity and read/write operations, along with …

Data storage trends and big data challenges. Distributed storage and the CAP theorem. Distributed storage file system. Mogilefs principles and implementation. Nginx reverse proxy for Tracker nodes…

Storing massive numbers of small files is a classic application scenario. The following is a republished article analyzing the pros and cons of three solutions for massive small-file storage: Nginx+NFS, Redis, and the distributed file system MooseFS.
We now have 30 million records, each record is very small. If we use a key-value approach…

In the afternoon, at the Storage and System Architecture sub-forum of the IT168 System Architect Conference, Zhang Wensong, Chairman of the Taobao Technology Committee and Core Engineer at Taobao, gave us a detailed introduction to the architecture of Taobao’s image processing and storage system. Dr. Zhang Wensong’s presentation agenda covered Taobao’s overall system architecture, image processing…

There are many distributed file systems, including GFS, HDFS, Taobao’s open-source TFS, Tencent’s TFS for photo album storage (Tencent FS, hereafter referred to as QFS for differentiation), and Facebook Haystack. Among them, TFS, QFS, and Haystack address very similar problems and share a comparable architecture. These three file systems …

In today’s disk storage market, storage classification (see Table 1 below) is divided by server type into: closed-system storage and open-system storage. Closed systems primarily refer to mainframes, AS/400, and similar servers, while open systems refer to servers based on operating systems including Windows, UNIX, and Linux. Open-system storage is further divided into: internal storage and external storage; external storage for open systems is classified by connection method into: Direct-Attached Storage (DAS) and Fabric-Attached Storage (FAS); open-system networked storage is further divided by transport protocol into: Network-Attached Storage (NAS) and Storage Area Network (SAN). Since the vast majority of users today adopt open systems, with external storage accounting for over 70% of the current disk storage market, this article primarily discusses and explains external storage for open systems.

1.1 StorageStorage devices are the most fundamental layer in a website’s backend architecture. They are also the most critical component. If a storage device encounters an issue, it will directly cause services in the upper data and application layers to halt. Severe storage device damage and irrecoverable data loss can bring enormous …

How DRBD Works DRBD is a block device designed for High Availability (HA) scenarios. It functions like a network RAID-1. When you write data to the local file system, the data is also sent to another host on the network, where it is recorded in the same form within a file system. Local …