Main Configuration Files for MooseFS Components
master
Master Server
Metadata is stored in memory of the managing server and simultaneously is being saved on disk (as a periodically updated binary file and immediately updated incremental logs). The main binary file as well as the logs are replicated to metaloggers (if present).
Metadata is stored in the memory of the master server, and is also saved to disk (as a periodically updated binary file and real-time updated changelog). If metaloggers exist, the main binary file and logs are also replicated to them.
How much CPU/RAM resources are used?
How much CPU and RAM resources are consumed?
In our environment (ca. 500 TiB, 25 million files, 2 million folders distributed on 26 million chunks on 70 machines) the usage of chunkserver CPU (by constant file transfer) is about 15-20% and chunkserver RAM usually consumes about 100MiB (independent of amount of data).
In our test environment (approximately 500 TiB of data, 25 million files, 2 million folders, divided into 26 million chunks distributed across 70 machines), chunkserver CPU usage (for continuous file transfers) is about 15-20%, while chunkserver RAM usage is about 100MiB (independent of the amount of data).
The master server consumes about 30% of CPU (ca. 1500 operations per second) and 8GiB RAM. CPU load depends on amount of operations and RAM on number of files and folders.
The master server consumes about 30% of CPU (approximately 1500 operations per second) and 8GiB of RAM. CPU load depends on the number of operations, while RAM usage depends on the number of files and folders.
File data is divided to fragments (chunks) of maximum size 64MB each which are stored as files on selected disks on data servers (chunkservers). Each chunk is saved on different computers in a number of copies equal to a "goal" for the given file.
File data is divided into chunks (with a maximum chunk size of 64MB), which are stored as files on designated disks on data servers (chunkservers). Different chunks are stored on different machines, with the number of copies equal to the “goal” set for the given file.
What sort of sizing is required for the Master server?
What are the requirements for the Master server?
The most important factor is RAM of mfsmaster machine, as the full file system structure is cached in RAM for speed. Besides RAM mfsmaster machine needs some space on HDD for main metadata file together with incremental logs.
The most important factor is the RAM of the mfsmaster machine, as the entire file system structure is cached in RAM for faster access. In addition to RAM, the mfsmaster machine also requires some HDD space to store the metadata and incremental log files.
The size of the metadata file is dependent on the number of files (not on their sizes). The size of incremental logs depends on the number of operations per hour, but length (in hours) of this incremental log is configurable.
The size of the metadata file depends on the number of files (not their sizes). The size of the changelog depends on the number of operations per hour, but this time duration (in hours) for the incremental log is configurable.
1 million files takes approximately 300 MiB of RAM. Installation of 25 million files requires about 8GiB of RAM and 25GiB space on HDD.
1 million files require approximately 300 MiB of RAM. 25 million files require about 8GiB of RAM and 25GiB of HDD space.
Master Main Files
mfsmaster.cfg
mfsexports.cfg
MooseFS access control file
Several forms of address specifications: all IPs, single IP, IP network address/bitmask, IP network address/subnet mask, IP range.
Permission section:
ro Read-only sharing mode
rw Read-write sharing mode
alldirs Allows mounting any specified subdirectory
maproot Maps to root, or to a specified user
password Specifies the client password
.mfsmaster.lock
lock file of running MooseFS master process
The mfsmaster.lock file records the running MooseFS master process.
metadata.mfs, metadata.mfs.back
MooseFS filesystem metadata image
metadata.mfs and metadata.mfs.back are the metadata images of the MooseFS filesystem.
changelog.*.mfs
MooseFS filesystem metadata change logs (merged into metadata.mfs once per hour)
changelog.*.mfs are the metadata change logs of the MooseFS filesystem (merged into metadata.mfs every hour).
The size of the metadata file is dependent on the number of files (not on their sizes). The size of incremental logs depends on the number of operations per hour。
The size of the metadata file depends on the number of files (not their sizes), and the size of the changelog depends on the number of operations per hour.
metalogger
Main Files:
mfsmetalogger.cfg
changelog_ml.*.mfs
MooseFS filesystem metadata change logs (backup of master change log files)
changelog_ml.*.mfs are the metadata changelog logs of the MooseFS filesystem (backups of the Master’s changelog logs).
metadata.ml.mfs.back
Latest copy of complete metadata.mfs.back file from MooseFS master.
metadata.ml.mfs.back is the latest complete copy of the metadata.mfs.back file downloaded from the Master host.
sessions.ml.mfs
Latest copy of sessions.mfs file from MooseFS master.
sessions.ml.mfs is the latest copy of the sessions.mfs file downloaded from the master.