[Author: Zhang Yan Version: v1.0
Original article link: http://blog.s135.com/file_get_contents/]
Sometimes, Linux servers running Nginx and PHP-CGI (php-fpm) web services suddenly experience increased system load. Using the top command, many php-cgi processes show CPU usage close to 100%. Later, through tracing, I discovered that this situation is closely related to PHP’s file_get_contents() function.
In large and medium-sized websites, API calls based on the HTTP protocol are commonplace. PHP programmers like to use the simple and convenient file_get_contents(“http://example.com/”) function to fetch the response content of a URL. However, if the website http://example.com/ responds slowly, file_get_contents() will hang there indefinitely without timing out.
We know that in php.ini, there is a parameter max_execution_time that can set the maximum execution time of a PHP script. However, in php-cgi (php-fpm), this parameter does not take effect. What actually controls the maximum execution time of PHP scripts is the following parameter in the php-fpm.conf configuration file:
- The timeout (in seconds) for serving a single request after which the worker process will be terminated
- Should be used when 'max_execution_time' ini option does not stop script execution for some reason
- '0s' means 'off'
- <value name="request_terminate_timeout">0s</value>
The default value is 0 seconds, meaning PHP scripts will run indefinitely. When all php-cgi processes are stuck at file_get_contents(), the Nginx+PHP WebServer can no longer handle new PHP requests, and Nginx will return “502 Bad Gateway” to users. Modifying this parameter to set a maximum execution time for PHP scripts is necessary, but it treats the symptoms rather than the root cause. For example, changing it to <value name="request_terminate_timeout">30s</value> means that if file_get_contents() is slow at fetching web content, 150 php-cgi processes can only handle 5 requests per second, and the WebServer will still struggle to avoid “502 Bad Gateway”.
To solve this thoroughly, PHP programmers must stop the habit of directly using file_get_contents(“http://example.com/”) and instead modify it slightly by adding a timeout, using the following method to make HTTP GET requests. If you find it troublesome, you can wrap the code below into a function yourself.
- <?php
- $ctx = stream_context_create(array(
- 'http' => array(
- 'timeout' => 1 //Set a timeout, in seconds
- )
- )
- );
- file_get_contents("http://example.com/", 0, $ctx);
- ?>
Of course, php-cgi process CPU 100% is not only caused by this one reason. So how do you determine that it is indeed caused by file_get_contents()?
First, use the top command to view php-cgi processes with high CPU usage.
Tasks: 561 total, 15 running, 546 sleeping, 0 stopped, 0 zombie
Cpu(s): 5.9%us, 4.2%sy, 0.0%ni, 89.4%id, 0.2%wa, 0.0%hi, 0.2%si, 0.0%st
Mem: 8100996k total, 4320108k used, 3780888k free, 772572k buffers
Swap: 8193108k total, 50776k used, 8142332k free, 412088k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
10747 www 18 0 360m 22m 12m R 100.6 0.3 0:02.60 php-cgi
10709 www 16 0 359m 28m 17m R 96.8 0.4 0:11.34 php-cgi
10745 www 18 0 360m 24m 14m R 94.8 0.3 0:39.51 php-cgi
10707 www 18 0 360m 25m 14m S 77.4 0.3 0:33.48 php-cgi
10782 www 20 0 360m 26m 15m R 75.5 0.3 0:10.93 php-cgi
10708 www 25 0 360m 22m 12m R 69.7 0.3 0:45.16 php-cgi
10683 www 25 0 362m 28m 15m R 54.2 0.4 0:32.65 php-cgi
10711 www 25 0 360m 25m 15m R 52.2 0.3 0:44.25 php-cgi
10688 www 25 0 359m 25m 15m R 38.7 0.3 0:10.44 php-cgi
10719 www 25 0 360m 26m 16m R 7.7 0.3 0:40.59 php-cgi
Pick the PID of one of the php-cgi processes with 100% CPU and trace it with the following command:
If the screen displays:
poll([{fd=6, events=POLLIN}], 1, 0) = 0 (Timeout)
select(7, [6], [6], [], {15, 0}) = 1 (out [6], left {15, 0})
poll([{fd=6, events=POLLIN}], 1, 0) = 0 (Timeout)
select(7, [6], [6], [], {15, 0}) = 1 (out [6], left {15, 0})
poll([{fd=6, events=POLLIN}], 1, 0) = 0 (Timeout)
select(7, [6], [6], [], {15, 0}) = 1 (out [6], left {15, 0})
poll([{fd=6, events=POLLIN}], 1, 0) = 0 (Timeout)
select(7, [6], [6], [], {15, 0}) = 1 (out [6], left {15, 0})
poll([{fd=6, events=POLLIN}], 1, 0) = 0 (Timeout)
select(7, [6], [6], [], {15, 0}) = 1 (out [6], left {15, 0})
poll([{fd=6, events=POLLIN}], 1, 0) = 0 (Timeout)
select(7, [6], [6], [], {15, 0}) = 1 (out [6], left {15, 0})
poll([{fd=6, events=POLLIN}], 1, 0) = 0 (Timeout)
select(7, [6], [6], [], {15, 0}) = 1 (out [6], left {15, 0})
poll([{fd=6, events=POLLIN}], 1, 0) = 0 (Timeout)
select(7, [6], [6], [], {15, 0}) = 1 (out [6], left {15, 0})
poll([{fd=6, events=POLLIN}], 1, 0) = 0 (Timeout)
select(7, [6], [6], [], {15, 0}) = 1 (out [6], left {15, 0})
poll([{fd=6, events=POLLIN}], 1, 0) = 0 (Timeout)
Then you can confirm that file_get_contents() is the cause of the problem.