Version: 9.4.5.v20170502
Version: 9.4.5.v20170502 |
private support for your internal/customer projects ... custom extensions and distributions ... versioned snapshots for indefinite support ... scalability guidance for your apps and Ajax/Comet projects ... development services for sponsored feature development
Configuring Jetty for high load, whether for load testing or for production, requires that the operating system, the JVM, Jetty, the application, the network and the load generation all be tuned.
Machines handling load generation must have their OS, JVM, etc., tuned just as much as the server machines.
The load generation should not be over the local network on the server machine, as this has unrealistic performance and latency as well as different packet sizes and transport characteristics.
The load generator should generate a realistic load. Avoid the following pitfalls:
HttpClient is an ideal choice for building a load generator as it is asynchronous and can simulate many thousands of connections (see the CometD Load Tester for a good example of a realistic load generator).Both the server machine and any load generating machines need to be tuned to support many TCP/IP connections and high throughput.
Linux does a reasonable job of self-configuring TCP/IP, but there are a few limits and defaults that you should increase.
You can configure most of these in /etc/security/limits.conf or via sysctl.
You should increase TCP buffer sizes to at least 16MB for 10G paths and tune the auto-tuning (keep in mind that you need to consider buffer bloat).
$ sysctl -w net.core.rmem_max=16777216 $ sysctl -w net.core.wmem_max=16777216 $ sysctl -w net.ipv4.tcp_rmem="4096 87380 16777216" $ sysctl -w net.ipv4.tcp_wmem="4096 16384 16777216"
net.core.somaxconn controls the size of the connection listening queue.
The default value is 128.
If you are running a high-volume server and connections are getting refused at a TCP level, you need to increase this value.
This setting can take a bit of finesse to get correct: if you set it too high, resource problems occur as it tries to notify a server of a large number of connections, and many remain pending, but if you set it too low, refused connections occur.
$ sysctl -w net.core.somaxconn=4096
The net.core.netdev_max_backlog controls the size of the incoming packet queue for upper-layer (Java) processing.
The default (2048) may be increased and other related parameters adjusted with:
$ sysctl -w net.core.netdev_max_backlog=16384 $ sysctl -w net.ipv4.tcp_max_syn_backlog=8192 $ sysctl -w net.ipv4.tcp_syncookies=1
If many outgoing connections are made (for example, on load generators), the operating system might run low on ports.
Thus it is best to increase the port range, and allow reuse of sockets in TIME_WAIT:
$ sysctl -w net.ipv4.ip_local_port_range="1024 65535" $ sysctl -w net.ipv4.tcp_tw_recycle=1
Busy servers and load generators may run out of file descriptors as the system defaults are normally low.
These can be increased for a specific user in /etc/security/limits.conf:
theusername hard nofile 40000 theusername soft nofile 40000
Linux supports pluggable congestion control algorithms. To get a list of congestion control algorithms that are available in your kernel run:
$ sysctl net.ipv4.tcp_available_congestion_control
If cubic and/or htcp are not listed, you need to research the control algorithms for your kernel. You can try setting the control to cubic with:
$ sysctl -w net.ipv4.tcp_congestion_control=cubic
Intermediaries such as nginx can use a non-persistent HTTP/1.0 connection. Make sure to use persistent HTTP/1.1 connections.
The standard rule of thumb for the number of Accepters to configure is one per CPU on a given machine.