Varnish, similar to typical server software, is split into a master (management) process and a child (worker, mainly handling caching) process. The master process reads commands, performs some initialization, then forks and monitors the child process. The child process allocates multiple threads to work, primarily including some management threads and many worker threads. For the file caching part, the master reads the storage configuration (-s file[,path[,size[,granularity]]]), calls the appropriate storage type, and then creates/reads a large cache file of the corresponding size (according to its man page, to avoid storage fragmentation [2] that impacts read/write performance, the author recommends pre-creating the large file using the dd(1) command). Next, the master initializes the structures for managing this storage space. These variables are all global, and after forking, they are inherited by the child process (including file descriptors).
During the main thread initialization of the child process, the previously opened large storage file is mmaped entirely into memory (if it exceeds the system’s virtual memory, mmap fails, and the process reduces the originally configured mmap size and retries mmap). At this point, free storage structures are created and initialized, then attached to the storage management structure for future allocation.
Then, the real work begins. A thread in Varnish responsible for accepting new HTTP connections starts waiting for users (for the specific process, see [3]). If a new HTTP connection arrives, it is always responsible for accepting it, then wakes up a waiting thread and hands over the specific handling process. The Worker thread reads the URI from the HTTP request, looks up the existing object, and if there is a hit, returns it directly and responds to the user. If there is no hit, it needs to fetch the requested content from the backend server, store it in the cache, and then respond.
The cache allocation process works like this: it creates a cache file of the appropriate size based on the size of the object it reads. For read/write convenience, the program rounds the size of each object up to the nearest multiple of the memory page size. It then searches the existing free storage structures to find the most suitable free storage block and allocates it. If the free block is not fully used, the remaining memory is formed into another free storage block and attached to the management structure. If the cache is full, the oldest object is released based on the LRU [4] mechanism.
The cache release process works like this: a timeout thread checks the lifetime of all objects in the cache. If the set TTL (Time To Live) has been exceeded and