Optimizing Nginx for High Traffic Loads

That said, many parameters that influence Nginx behavior have defaults that are not perfectly suited for high-concurrency scenarios. We also need to consider the platform Nginx runs on and optimize our operating system when certain limitations exist.

Generally speaking, we cannot optimize the load time of a single connection, but we can ensure Nginx handles a high-concurrency environment. Of course, by high concurrency I mean hundreds of requests per second — most people do not need to know this. If you are too curious or want to know, then keep reading.

First, we need to recognize that Nginx is available on almost all platforms — MacOS, Linux, FreeBSD, Solaris, Windows, and even some more esoteric systems. Most of them implement high-performance event-based polling methods. Unfortunately, Nginx only supports four of these systems. Among the four, I lean toward FreeBSD, but you will not see a dramatic performance difference, so choosing an operating system you are comfortable with is more important than picking the most optimized one. (See 舟‘s first paragraph translated very well.)

I bet you have already guessed that Windows is not among them. There is really no compelling reason to use Nginx on Windows. Windows has its own way of handling event polling, so the Nginx author chose not to support it. Therefore, it defaults to using select(), which is less efficient and causes significant performance degradation.

The second biggest limitation, and one most people will encounter, is OS-related. Open a shell window, use su to switch to the user that Nginx runs as, and run the command `ulimit -a`. These values will also impose limits on Nginx while it is running. On many operating systems, the "open files" value is quite limited; on the OS I use, it is 1024. If Nginx hits this limit during operation, it will log an error log (24: Too many open files) and then return a response to the client.  Of course, Nginx can handle more files, and you can make some OS-level changes — you can confidently increase this value.

There are two ways to do this. You can set the OS "open files" limit via ulimit, and you can also declare your desired value through the Nginx configuration directive worker_rlimit_nofile.

Nginx Limits

Beyond paying attention to OS limits, let me now dive into Nginx itself and look at some directives and methods we can use to tune it.

Worker Processes

worker_processes is the backbone of Nginx. Once the master process binds to the specified IP and port, it spawns child processes using the designated Nginx user, and they then handle all the work. Workers are not multi-threaded, so you cannot scale them beyond the number of CPU cores. So we should understand the rationale for setting multiple (>1) workers: typically one worker per CPU core. More is not better — 2-4 workers can hurt the CPU, and Nginx will hit other bottlenecks before the CPU becomes an issue. Usually you will just see idle processes.

When you are dealing with a scenario where you have a lot of blocking disk I/O, you can appropriately increase the worker_processes value. You need to test against your configuration, check the waiting time for static files, and if the value is relatively large, you can increase worker_processes appropriately.

Worker Connections

worker_connections is a slightly odd concept. I do not fully understand the purpose of this directive, but it effectively limits the number of connections each worker can maintain at the same time. If I am not mistaken, this setting is meant to ensure that when keep-alive is misconfigured, you can increase connections before you run out of ports.

The default value is 1024. Let us assume a browser typically opens 2 connections to pipeline website resources, meaning you can handle at most 512 simultaneous user requests. That sounds awfully low, but consider that the default keepalive_timeout is 65 (the default config file provides the value 65; if not set, the default is 75, see wiki keepalive_timeout), which means we can actually handle only 8 connections per second. Obviously this value is higher than many people expect, especially considering we usually set 2-4 workers. But for higher-traffic websites, using keep-alive is worthwhile.

Additionally, we must also consider reverse proxies, which will open an extra connection to the backend. However, since Nginx does not support persistent connections to the backend, this is not a huge issue unless you have long-running backend processes.

Everything about configuring worker connections should be fairly clear: if your traffic increases, you should correspondingly increase the number of worker connections. 2048 should suffice for most people, but honestly, if your traffic grows, it should be quite obvious what the value for workers should be.

CPU Affinity

Setting CPU affinity basically means you tell each process which CPU core to use, and they will only use that core. I do not want to say much about this, but you need to know that if you plan to do this, you must be very careful. Understand that your OS’s CPU scheduler handles load balancing far better than you can. Of course, if you think your CPU load balancing has issues, optimize it at the scheduling level and, if possible, find an alternative scheduler. Do not touch this unless you know what you are doing.

Keep Alive

keep_alive is an HTTP feature that allows a client to maintain an already-established connection with the server to process a batch of requests until the specified timeout is reached. This does not actually change our Nginx server performance dramatically, because Nginx handles idle connections very well. The Nginx author claims 10,000 idle connections use only 2.5 megabytes of memory (unbelievable), and in my personal experience this value holds true.

The reason I mention this in a performance article is very simple. For the end user, keep alive has a massive impact on load time. This is one of the most important metrics and the very reason we keep optimizing. If your website feels fast to load for users, they will be happy. Amazon and some other large online retailers have conducted many studies showing a direct correlation between website load time and completed orders.

Why keep alive has such a huge impact should be obvious — you avoid creating individual connections for every HTTP request, which is extremely inefficient. You may not need to set keepalive_timeout to 65, but 10-20 should be a fairly universal choice, and as mentioned above, Nginx handles this very well.

tcp_nodelay and tcp_nopush

These two directives are perhaps the most difficult Nginx configurations to understand, as their impact operates at the lower layers of the network. You can simply think of these directives as determining how the operating system handles network buffers and when it flushes them to the end user (client). My only advice is that if you don’t understand these concepts beforehand, you’d better leave them alone. They won’t significantly improve or change performance, so it’s best to stick with their default values.

Hardware Limitations

Since we are dealing with all possible limitations brought by Nginx, we now need to figure out how to effectively utilize our server. To do this, we need to look at the hardware level, because this is where most server bottlenecks occur.

Generally, servers have three main bottleneck areas: CPU, memory, and IO. Nginx is very efficient in terms of CPU utilization, so I’ll tell you frankly that this won’t be the bottleneck. Likewise, Nginx is also very efficient in using memory, so that won’t be the bottleneck either. Now, that leaves only IO as the main culprit of server bottlenecks. (Feels like hunting down a criminal.)

If you work with servers frequently, you may have experienced this realization. Hard disk drives are really, really slow. Reading from a hard drive is probably the most expensive operation for a server. So the natural conclusion is that to avoid IO bottlenecks, we need to drastically reduce Nginx’s reads from and writes to the hard drive.

To achieve this, we can modify Nginx’s behavior to reduce disk write operations, as well as ensure that Nginx’s memory limits allow it to avoid disk access.

Access Logs

By default, Nginx logs every request to a log file on disk. You can use this method for statistics, security issue checks, etc., but this comes with a certain IO usage cost. If you don’t plan to use these access logs for any checks or other purposes, you can simply turn them off to avoid disk writes. However, if you need access logs, you can consider saving the logs to memory. This will be much faster than writing directly to disk and significantly reduce IO usage.

If you only plan to use access logs for statistics, you might consider using alternatives like Google Analytics instead (though GA and access logs are different and cannot simply replace each other), or you could log only partial request information rather than everything.

Error Logs

I struggled internally a little about whether I should elaborate on the error_log directive here, because you probably wouldn’t want to turn off error logs at all, especially considering that the volume of error logs in real applications tends to be very small. However, there is one small thing worth noting about this directive: you can specify the error log level parameter. If you set it too low, it will record 404 errors and even debug info. In practical applications, setting it to the ‘warn’ level will be more than sufficient and can reduce IO.

Open File Cache

 Reading files from the file system consists of two parts: opening and closing files. Considering this is a blocking operation, this part should not be ignored. Therefore, caching open file descriptors is very good for us, and this is the origin of the open_file_cache directive. The linked wiki page has an excellent explanation on its usage and configuration, so I suggest you go and read it.

Buffers

Configuring Nginx buffer sizes is a very important matter. If the buffer size is set too small, Nginx will have to write the upstream response results to temporary cache files, which will simultaneously increase both IO read and write operations, and the more traffic there is, the worse the problem becomes.

The client_body_buffer_size directive is used to specify the buffer size for handling client requests; this represents the body of the incoming request. It is used to process POST data, i.e., data from form submissions, file uploads, and similar requests. If you need to handle many large POST requests, you must ensure the buffer is set large enough.

The fastcgi_buffers and proxy_buffers directives are used to handle upstream response results, i.e., from PHP, Apache, etc. The concept is basically the same as mentioned above: if the buffer is not large enough, the data will be saved to disk before being returned to the user. Note that Nginx has a buffer limit before it synchronously transmits this buffered data to the client, and saving to disk is similarly limited. This limit is set via fastcgi_max_temp_file_size and proxy_max_temp_file_size. Additionally, for proxy connections, you can also completely turn off buffering by setting proxy_buffering to off. (This is usually not a good approach.) 

Eliminating Disk IO Entirely

The best way to reduce disk IO is undoubtedly to not use the disk at all. If your application has only a small amount of data transfer, you can place all the data into memory, thus completely avoiding the blockage of disk IO. Of course, by default, your operating system will also cache frequently accessed disk sectors, so the more memory you have, the less disk IO will be used. This means you can solve IO bottlenecks by increasing memory. The more data you have, the more memory you will need.

Network IO

For fun, let’s assume you have enough memory to cache all your data. This theoretically means your IO read speed reaches 3-6gbps. But you don’t have a network pipe that fast. Unfortunately, the network IO we can optimize is limited; we have to transmit data over the network, so we are still constrained by network IO. The only truly effective method is to minimize data volume or compress it.

Fortunately, Nginx provides the gzip module, which allows us to compress data before transmitting it to the client, greatly reducing the data size. Generally speaking, the gzip_comp_level value doesn’t make much difference in terms of performance; setting it to 4-5 is fine. Increasing it blindly is pointless and just wastes CPU cycles.
You can also reduce transfer file sizes through some JavaScript and CSS minification tools. But these are not very relevant to Nginx, so I trust you can get more related information through Google.

Phew~

This article is drawing to a close. If you still need further optimization, it might be time to consider adding more servers. There’s no need to continue wasting time on micro-optimizing Nginx, though that is a topic for another article at another time. If you were curious because this article happens to be 2400 words (the original draft), please take a short break before exploring my blog further.