How to set Tomcat thread pool size?

Time:2020-11-4

background

In our daily development, we all use Tomcat as the server, but what size of thread pool should we set? And what are the principles for designing this thread pool? Next, I will introduce how I design and calculate.

Specific methods

As we all know, when Tomcat accepts a request, it will involve CPU and IO time. Among them, IO waiting time, CPU passively gives up execution, and other threads can use this time slice to operate. So we can adopt the general rules of server IO optimization.
Thread size = ((thread IO time + thread CPU) / thread CPU time) * CPU cores

give an example

The thread IO time is 100ms (IO operations such as database query, synchronous remote call, etc.), the thread CPU time is 10ms, and the server has 4 physical cores. From the above formula, we calculate the size is((100 + 10 )/10 ) *4 = 44。 In theory, we have a basis, but in the actual calculation process, how do we know the thread IO time and CPU time? This involves how to monitor the processing time in the actual coding process.

  • By implementing the built-in filter interface in Java, we can get the total time consumed by a request

public class MoniterFilter implements Filter {
  
    @Override
    public void doFilter(ServletRequest request, ServletResponse response,  
                            FilterChain chain) throws IOException,
                             ServletException {
        long start = System.currentTimeMillis();
        String params = getQueryString(httpRequest);
        try {
            chain.doFilter(httpRequest, httpResponse);
        } finally {
            logger.info("access url [{}{}], cost time [{}] ms )", uri, 
                        params, cost);
        }
  
    private String getQueryString(HttpServletRequest req) {
        StringBuilder buffer = new StringBuilder("?");
        Enumeration<String> emParams = req.getParameterNames();
        try {
            while (emParams.hasMoreElements()) {
                String sParam = emParams.nextElement();
                String sValues = req.getParameter(sParam);
                buffer.append(sParam).append("=").append(sValues).append("&");
            }
            return buffer.substring(0, buffer.length() - 1);
        } catch (Exception e) {
        }
        return "";
    }
}
  • Monitoring thread IO time consumption by adding facets (JDK, cglib)

public class DaoInterceptor implements MethodInterceptor {

    private static final Logger logger = LoggerFactory.getLogger(DaoInterceptor.class);

    @Override
    public Object invoke(MethodInvocation invocation) throws Throwable {
        StopWatch watch = new StopWatch();
        watch.start();
        Object result = null;
        Throwable t = null;
        try {
            result = invocation.proceed();
        } catch (Throwable e) {
            t = e == null ? null : e.getCause();
            throw e;
        } finally {
            watch.stop();
            logger.info("({}ms)", watch.getTotalTimeMillis());

        }

        return result;
    }
}

Through the above code can calculate the corresponding time, thus calculating the thread size. But shall we stop there?
In fact, it has not. The calculated value only exists in theory. We still need to use JMeter to test a offline server, and dynamically fine tune the thread pool size just calculated according to the QPS value.