Sysctl Command and Linux Kernel Parameter Tuning
I. The sysctl command is used to configure and display kernel parameters in the /proc/sys directory. To persist parameters permanently, you can edit the /etc/sysctl.conf file.
Command syntax:
sysctl [-n] [-e] -w variable=valuesysctl [-n] [-e] -p (default /etc/sysctl.conf)sysctl [-n] [-e] –a
Meaning of common options:
-w Temporarily change the value of a specified parameter, e.g.,
# sysctl -w net.ipv4.ip_forward=1
-a Display all system parameters
-p Load system parameters from a specified file; defaults to loading from /etc/sysctl.conf, e.g.:
# echo 1 > /proc/sys/net/ipv4/ip_forward# sysctl -w net.ipv4.ip_forward=1
Both methods above can immediately enable routing functionality. However, if the network is restarted, or if you run
# service network restart
the set value will be lost. To permanently retain the configuration, modify the /etc/sysctl.conf file, changing
net.ipv4.ip_forward=0
to
net.ipv4.ip_forward=1
II. Linux Kernel Parameter Tuning: There are two ways to adjust Linux kernel parameters
Method 1: Modify the kernel parameter files under /proc. You cannot use an editor to modify kernel parameter files because the kernel can change any of these files at any time. Additionally, these kernel parameter files are virtual files that do not actually exist, so they cannot be edited with an editor. Instead, use the echo command and redirect output from the command line to the selected file under /proc. For example, to set the timeout_timewait parameter to 30 seconds:
# echo 30 > /proc/sys/net/ipv4/tcp_fin_timeout
The parameter modification takes effect immediately, but after a system reboot, the parameter reverts to its default value. Therefore, to permanently change kernel parameters, you need to modify the /etc/sysctl.conf file.
Method 2. Modify the /etc/sysctl.conf file. Check the sysctl.conf file; if it already contains the parameter you need to modify, change its value. If the parameter is missing, add it to the sysctl.conf file. For example:
net.ipv4.tcp_fin_timeout=30
After saving and exiting, you can reboot the machine for the parameters to take effect. To make the parameters effective immediately, you can also run the following command:
# sysctl -p
III. Parameter Settings and Descriptions in sysctl.conf
proc/sys/net/core/wmem_max
Maximum socket write buffer; recommended optimization value: 873200
/proc/sys/net/core/rmem_max
Maximum socket read buffer; recommended optimization value: 873200
/proc/sys/net/ipv4/tcp_wmem
TCP write buffer; recommended optimization values: 8192 436600 873200
/proc/sys/net/ipv4/tcp_rmem
TCP read buffer; recommended optimization values: 32768 436600 873200
/proc/sys/net/ipv4/tcp_mem
Also has 3 values, meaning:
net.ipv4.tcp_mem[0]: Below this value, TCP has no memory pressure.
net.ipv4.tcp_mem[1]: At this value, enters memory pressure stage.
net.ipv4.tcp_mem[2]: Above this value, TCP refuses to allocate sockets.
The memory units above are in pages, not bytes. Recommended optimization values: 786432 1048576 1572864
/proc/sys/net/core/netdev_max_backlog
Maximum device queue for incoming packets. Default is 300; for heavily loaded servers, this value is too low and can be adjusted to 1000.
/proc/sys/net/core/somaxconn
Default parameter for listen(), the maximum number of pending requests. Default is 128. For busy servers, increasing this value helps network performance and can be adjusted to 256.
/proc/sys/net/core/optmem_max
Maximum initial value for socket buffer; default 10K.
/proc/sys/net/ipv4/tcp_max_syn_backlog
Maximum request queue for incoming SYN packets. Default 1024. For heavily loaded servers, can be adjusted to 2048.
/proc/sys/net/ipv4/tcp_retries2
Number of TCP failed retransmission attempts; default value 15, meaning it gives up completely after 15 retransmissions. Can be reduced to 5 to release kernel resources earlier.
/proc/sys/net/ipv4/tcp_keepalive_time
/proc/sys/net/ipv4/tcp_keepalive_intvl
/proc/sys/net/ipv4/tcp_keepalive_probes
These 3 parameters are related to TCP KeepAlive. Default values are:
tcp_keepalive_time = 7200 seconds (2 hours)tcp_keepalive_probes = 9tcp_keepalive_intvl = 75 seconds
This means if a TCP connection is idle for 2 hours, the kernel then initiates a probe. If 9 probes (75 seconds each) fail, the kernel completely gives up and considers the connection dead. For servers, these values are obviously too large. Can be adjusted to:
/proc/sys/net/ipv4/tcp_keepalive_time 1800
/proc/sys/net/ipv4/tcp_keepalive_intvl 30
/proc/sys/net/ipv4/tcp_keepalive_probes 3
/proc/sys/net/ipv4/ip_local_port_range
A configuration that specifies the port range, defaulting to 32768 61000, which is already large enough.
net.ipv4.tcp_syncookies = 1
Indicates SYN Cookies is enabled. When the SYN wait queue overflows, cookies are used to handle it, which can defend against small-scale SYN attacks. Defaults to 0, meaning disabled.
net.ipv4.tcp_tw_reuse = 1
Indicates reuse is enabled. Allows TIME-WAIT sockets to be reused for new TCP connections. Defaults to 0, meaning disabled.
net.ipv4.tcp_tw_recycle = 1
Indicates fast recycling of TIME-WAIT sockets in TCP connections is enabled. Defaults to 0, meaning disabled.
net.ipv4.tcp_fin_timeout = 30
If the socket is requested to close by the local end, this parameter determines the time it stays in the FIN-WAIT-2 state.
net.ipv4.tcp_keepalive_time = 1200
Indicates the frequency at which TCP sends keepalive messages when keepalive is enabled. The default is 2 hours; changed here to 20 minutes.
net.ipv4.ip_local_port_range = 1024 65000
Indicates the port range used for outgoing connections. The default is small: 32768 to 61000; changed here to 1024 to 65000.
net.ipv4.tcp_max_syn_backlog = 8192
Indicates the length of the SYN queue. Defaults to 1024; increasing the queue length to 8192 can accommodate more waiting network connections.
net.ipv4.tcp_max_tw_buckets = 5000
Indicates the maximum number of TIME_WAIT sockets the system holds simultaneously. If this number is exceeded, the TIME_WAIT socket is immediately cleared and a warning message is printed. Defaults to 180000, changed to 5000. For servers like Apache and Nginx, the parameters above can effectively reduce the number of TIME_WAIT sockets, but for Squid, the effect is minimal. This parameter can control the maximum number of TIME_WAIT sockets, preventing the Squid server from being bogged down by a large number of TIME_WAIT sockets.
NAT and iptables on Linux
When talking about NAT on Linux, most people will mention iptables. The reason is that iptables is currently a very good interface for implementing NAT on Linux. By directly operating network packets at the kernel level, its efficiency and stability are very high. Here are some simple examples of NAT-related iptables commands, which may be helpful for most implementations.
A note here: to save space, the preparatory commands are omitted, and only the core step commands are listed. So if you execute only these and fail to achieve the functionality, it’s likely because the preparation was not done properly. If you are interested in the full command details, you can directly visit my series of articles “How to Make Your Linux Gateway More Powerful,” where each script is explained and described in detail.
# Case 1: Implementing Gateway MASQUERADE
# Specific function: The internal network interface is eth1, the external is eth0, allowing the internal network to use this service as a gateway to access the external network.
EXTERNAL="eth0"INTERNAL="eth1"
# This step enables IP forwarding support, which is a prerequisite for NAT implementation
echo 1 > /proc/sys/net/ipv4/ip_forwardiptables -t nat -A POSTROUTING -o $EXTERNAL -j MASQUERADE
# Case 2: Implementing Simple Gateway Port Mapping
# Specific function: Allows external access to the gateway’s external IP:80 to directly reach a host on the private network at 192.168.1.10:80
LOCAL_EX_IP=11.22.33.44 #Set the gateway’s external network card IP; for multiple IPs, refer to the “How to Make Your Linux Gateway More Powerful” series
LOCAL_IN_IP=192.168.1.1 #Set the gateway’s internal network card IP
INTERNAL="eth1" #Set the internal network card
# This step enables IP forwarding support, which is a prerequisite for NAT implementation
echo 1 > /proc/sys/net/ipv4/ip_forward
# Load required IP modules; the following two are FTP-related modules. If you have other special requirements, they need to be added as well.
modprobe ip_conntrack_ftpmodprobe ip_nat_ftp
# This step changes the destination address for access targeting the gateway’s external IP:80 to 192.168.1.10:80
iptables -t nat -A PREROUTING -d $LOCAL_EX_IP -p tcp –dport 80 -j DNAT –to 192.168.1.10
# This step changes the source address of packets destined for 192.168.1.10:80 to the gateway’s own local IP, which is 192.168.1.1 here.
iptables -t nat -A POSTROUTING -d 192.168.1.10 -p tcp –dport 80 -j SNAT –to $LOCAL_IN_IP
# Add an allow rule on the FORWARD chain for 192.168.1.10:80, otherwise forwarding cannot occur
iptables -A FORWARD -o $INTERNAL -d 192.168.1.10 -p tcp –dport 80 -j ACCEPT
# With the three important lines above, the achieved effect is that all access through the gateway’s external IP:80 is forwarded to the internal network’s 192.168.1.10:80 port, implementing typical port mapping.
# Special note: All forwarded data packets have a source address that is the gateway’s internal network IP, so all accesses seen on 192.168.1.10 appear as if they came from the gateway itself, without seeing the external IP.
# An important concept: Data packets follow the strategy of “where it comes from, there it returns,” so there’s no need to worry about return data.
# There is still one issue: if the gateway itself accesses its own external IP:80, it will not be NATed to 192.168.1.10. This is not a critical problem, but it is rather annoying. The solution is as follows:
iptables -t nat -A OUTPUT -d $LOCAL_EX_IP -p tcp –dport 80 -j DNAT –to 192.168.1.10
Getting NAT Information in the System and Diagnosing Errors
Understanding the Meaning of the /proc Directory
In Linux systems, /proc is a special directory. The proc file system is a pseudo file system that exists only in memory and does not occupy external storage space. It contains some parameters of the current system
variables and status. It provides an interface to access kernel data through a file system structure.
Through /proc, you can obtain crucial real-time system information, including disk usage, memory usage, hardware details, network usage, and more. Many system monitoring tools (like HotSaNIC) retrieve system data via the /proc directory.
On the other hand, by directly manipulating parameters in /proc, you can tune system kernel parameters, such as whether IP forwarding is allowed, whether SYN cookies are enabled, and TCP timeout values.
How to read parameters:
Method 1: cat /proc/xxx/xxx, e.g., cat /proc/sys/net/ipv4/conf/all/rp_filter
Method 2: sysctl xxx.xxx.xxx, e.g., sysctl net.ipv4.conf.all.rp_filter
How to change parameters:
Method 1: echo value > /proc/xxx/xxx, e.g., echo 1 > /proc/sys/net/ipv4/conf/all/rp_filter
Method 2: sysctl [-w] variable=value, e.g., sysctl [-w] net.ipv4.conf.all.rp_filter=1
The above methods for setting system parameters only apply to the current session and are lost after a reboot. To persist them, you need to write them into the /etc/sysctl.conf file.
You can get an introduction to the /proc directory by running: man 5 proc
Checking NAT Status on the System
System variables related to NAT
/proc/slabinfo: Kernel slab allocator statistics
/proc/sys/net/ipv4/ip_conntrack_max: Maximum number of IPv4 connections the system supports, default 65536 (this is also the theoretical maximum)
/proc/sys/net/ipv4/netfilter/ip_conntrack_tcp_timeout_established: Timeout for established TCP connections, default 432000 seconds, which is 5 days
Status values related to NAT
/proc/net/ip_conntrack: Current tracked connection status; the NAT translation table is reflected here (for a Linux host primarily functioning as a gateway, most of the info here is the NAT translation table)
/proc/sys/net/ipv4/ip_local_port_range: Local open port range, which can indirectly limit the NAT table size
# 1. Check the current maximum connection limit
cat /proc/sys/net/ipv4/ip_conntrack_max
# Value: Default 65536. This value also correlates with memory size; if you have 128MB of RAM, the max is 8192; for RAM of 1GB and above, this value defaults to 65536
# Impact: This value determines your NAT gateway’s upper working capacity. Every connection from the LAN to the outside world through this gateway will occupy one entry. If this value is too low, it will affect throughput.
# 2. Check the TCP connection timeout
cat /proc/sys/net/ipv4/netfilter/ip_conntrack_tcp_timeout_established
# Value: Default 432000 (seconds), which is 5 days
# Impact: If this value is too large, it can cause some possibly unused connections to remain in memory, consuming significant connection resources and potentially leading to “NAT ip_conntrack: table full” issues.
# Recommendation: When the NAT load is heavy relative to the machine’s NAT table size, you might consider reducing this value to clear connections earlier and ensure available connection resources; if not under pressure, no change is needed.
# 3. Check NAT table usage (to determine if NAT table resources are under pressure)
# Run the following command to view the NAT table status on your gateway
cat /proc/net/ip_conntrack
# 4. Check the local open port range
cat /proc/sys/net/ipv4/ip_local_port_range
# Returns two values: the minimum and maximum
# The following command helps you determine the size of the NAT table
wc -l /proc/net/ip_conntrack#Orgrep ip_conntrack /proc/slabinfo | grep -v expect | awk '{print $1 ',' $2;}'
# The following command helps you see available NAT table entries. If this value is large, it indicates that NAT table resources are not under pressure.
grep ip_conntrack /proc/slabinfo | grep -v expect | awk '{print $1 ',' $3;}'
# The following command helps you count the top IPs consuming the most ports in the NAT table. These are likely the ones doing some “BT” stuff, yeah, BT stuff 🙂
cat /proc/net/ip_conntrack | cut -d ' ' -f 10 | cut -d '=' -f 2 | sort | uniq -c | sort -nr | head -n 10
# The above command has a slight flaw; cut -d’ ‘ -f10 can cause statistical deviations due to missing fields in some output lines. Here is a corrected approach:
cat /proc/net/ip_conntrack | perl -pe s/^/(.*?/)src/src/g | cut -d ' ' -f1 | cut -d '=' -f2 | sort | uniq -c | sort -nr | head -n 10