Explain in detail how Tomcat implements asynchronous servlets

Time:2021-7-24

preface

Through my previous Tomcat series articles, I believe the students who read my blog should have a clear understanding of Tomcat. In the previous blogs, we discussed how Tomcat is started in the springboot framework, how the internal components of Tomcat are designed and how the request flows. Then, let’s talk about the asynchronous servlet of Tomcat in our blog, How Tomcat implements asynchronous servlets and the usage scenarios of asynchronous servlets.

Hand roll an asynchronous Servlet

We implement a servlet directly with the help of the springboot framework. Here we only show the servlet Code:

@WebServlet(urlPatterns = "/async",asyncSupported = true)
@Slf4j
public class AsyncServlet extends HttpServlet {

 ExecutorService executorService =Executors.newSingleThreadExecutor();

 @Override
  protected void doGet(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {
  //Enable asynchronous to get asynchronous context
  final AsyncContext ctx = req.startAsync();
  //Commit thread pool asynchronous execution
  executorService.execute(new Runnable() {


   @Override
   public void run() {
    try {
     Log.info ("async service is ready to execute");
     //Simulate time-consuming tasks
     Thread.sleep(10000L);
     ctx.getResponse().getWriter().print("async servlet");
     Log.info ("async service executed");
    } catch (IOException e) {
     e.printStackTrace();
    } catch (InterruptedException e) {
     e.printStackTrace();
    }
    //The callback is completed after the final execution.
    ctx.complete();
   }
  });
 }

The above code implements an asynchronous servletdoGetNote that you need to use the restart class in springboot@ServletComponentScanAnnotations to scan servlets. Now that the code is written, let’s see the actual operation effect.

After we send a request, we see that the page responds. At the same time, it takes 10.05s to see the request time, so our servlet can run normally. Some students will certainly ask, isn’t this an asynchronous servlet? Your response time has not accelerated. What’s the use? Yes, our response time can’t be accelerated. It still depends on our business logic. However, after our asynchronous servlet requests, it depends on the asynchronous execution of the business. We can return immediately, that is, Tomcat threads can be recycled immediately. By default, Tomcat’s core threads are 10 and the maximum number of threads is 200. We can recover threads in time, This means that we can handle more requests and increase our throughput, which is also the main role of asynchronous servlets.

Internal principle of asynchronous Servlet

After understanding the role of asynchronous servlets, let’s see how Tomcat first asynchronous servlets. In fact, the above code has two main core logic parts,final AsyncContext ctx = req.startAsync();andctx.complete();Let’s see what they did?


 public AsyncContext startAsync(ServletRequest request,
   ServletResponse response) {
  if (!isAsyncSupported()) {
   IllegalStateException ise =
     new IllegalStateException(sm.getString("request.asyncNotSupported"));
   log.warn(sm.getString("coyoteRequest.noAsync",
     StringUtils.join(getNonAsyncClassNames())), ise);
   throw ise;
  }

  if (asyncContext == null) {
   asyncContext = new AsyncContextImpl(this);
  }

  asyncContext.setStarted(getContext(), request, response,
    request==getRequest() && response==getResponse().getResponse());
  asyncContext.setTimeout(getConnector().getAsyncTimeout());

  return asyncContext;
 }

We foundreq.startAsync();It only saves an asynchronous context and sets some basic information, such asTimeoutBy the way, the default timeout set here is 30s, which means that your asynchronous processing logic will report an error after 30s. It will be executed at this timectx.complete();An IllegalStateException is thrown.

Let’s seectx.complete();Logic of

public void complete() {
  if (log.isDebugEnabled()) {
   logDebug("complete ");
  }
  check();
  request.getCoyoteRequest().action(ActionCode.ASYNC_COMPLETE, null);
 }
//Class: abstractprocessor 
 public final void action(ActionCode actionCode, Object param) {
 case ASYNC_COMPLETE: {
   clearDispatches();
   if (asyncStateMachine.asyncComplete()) {
    processSocketEvent(SocketEvent.OPEN_READ, true);
   }
   break;
  } 
 }
 //Class: abstractprocessor 
protected void processSocketEvent(SocketEvent event, boolean dispatch) {
  SocketWrapperBase<?> socketWrapper = getSocketWrapper();
  if (socketWrapper != null) {
   socketWrapper.processSocket(event, dispatch);
  }
 }
 //Class: abstractendpoint
public boolean processSocket(SocketWrapperBase<S> socketWrapper,
   SocketEvent event, boolean dispatch) {
  //Omit some codes
   SocketProcessorBase<S> sc = null;
   if (processorCache != null) {
    sc = processorCache.pop();
   }
   if (sc == null) {
    sc = createSocketProcessor(socketWrapper, event);
   } else {
    sc.reset(socketWrapper, event);
   }
   Executor executor = getExecutor();
   if (dispatch && executor != null) {
    executor.execute(sc);
   } else {
    sc.run();
   }
 
  return true;
 }

So, this will eventually callAbstractEndpointYesprocessSocketMethod, students who have read my blog before should be impressed,EndPointIt is used to accept and process requests, and then it will be handed over toProcessorTo deal with the agreement.

Class: abstractprocessorlight
public SocketState process(SocketWrapperBase<?> socketWrapper, SocketEvent status)
   throws IOException {
  //Omit part DIAM
  SocketState state = SocketState.CLOSED;
  Iterator<DispatchType> dispatches = null;
  do {
   if (dispatches != null) {
    DispatchType nextDispatch = dispatches.next();
    state = dispatch(nextDispatch.getSocketStatus());
   } else if (status == SocketEvent.DISCONNECT) {
   
   } else if (isAsync() || isUpgrade() || state == SocketState.ASYNC_END) {
    state = dispatch(status);
    if (state == SocketState.OPEN) {
     state = service(socketWrapper);
    }
   } else if (status == SocketEvent.OPEN_WRITE) {
    state = SocketState.LONG;
   } else if (status == SocketEvent.OPEN_READ){
    state = service(socketWrapper);
   } else {
    state = SocketState.CLOSED;
   }

  } while (state == SocketState.ASYNC_END ||
    dispatches != null && state != SocketState.CLOSED);

  return state;
 }

This part is the focus,AbstractProcessorLightWill be based onSocketEventTo determine whether to callservice(socketWrapper)This method will eventually be called to the container to complete the call of business logic. Our request is called after execution. It must not be entered into the container, otherwise it will be dead circulated.isAsync()If you judge, you will enterdispatch(status), which will eventually callCoyoteAdapterYesasyncDispatchmethod

public boolean asyncDispatch(org.apache.coyote.Request req, org.apache.coyote.Response res,
   SocketEvent status) throws Exception {
  //Omit some codes
  Request request = (Request) req.getNote(ADAPTER_NOTES);
  Response response = (Response) res.getNote(ADAPTER_NOTES);
  boolean success = true;
  AsyncContextImpl asyncConImpl = request.getAsyncContextInternal();
  try {
   if (!request.isAsync()) {
    response.setSuspended(false);
   }

   if (status==SocketEvent.TIMEOUT) {
    if (!asyncConImpl.timeout()) {
     asyncConImpl.setErrorState(null, false);
    }
   } else if (status==SocketEvent.ERROR) {
    
   }

   if (!request.isAsyncDispatching() && request.isAsync()) {
    WriteListener writeListener = res.getWriteListener();
    ReadListener readListener = req.getReadListener();
    if (writeListener != null && status == SocketEvent.OPEN_WRITE) {
     ClassLoader oldCL = null;
     try {
      oldCL = request.getContext().bind(false, null);
      res.onWritePossible();// Here, execute the browser response and write data
      if (request.isFinished() && req.sendAllDataReadEvent() &&
        readListener != null) {
       readListener.onAllDataRead();
      }
     } catch (Throwable t) {
      
     } finally {
      request.getContext().unbind(false, oldCL);
     }
    } 
    }
   }
   //It is judged that the asynchrony is in progress, indicating that it is not a callback to complete the method, but a normal asynchronous request. Continue to call the container.
   if (request.isAsyncDispatching()) {
    connector.getService().getContainer().getPipeline().getFirst().invoke(
      request, response);
    Throwable t = (Throwable) request.getAttribute(RequestDispatcher.ERROR_EXCEPTION);
    if (t != null) {
     asyncConImpl.setErrorState(t, true);
    }
   }
   //Note that if there is a timeout or error, request. Isasync () will return false. This is to output the error to the client as soon as possible.
   if (!request.isAsync()) {
    //This is also the output logic
    request.finishRequest();
    response.finishResponse();
   }
   //Destroy request and response
   if (!success || !request.isAsync()) {
    updateWrapperErrorCount(request, response);
    request.recycle();
    response.recycle();
   }
  }
  return success;
 }

The code above isctx.complete()The final method is executed (of course, many details are omitted), the data output is completed, and finally output to the browser.

Some students here may say, I know that after the asynchronous execution, the call is made.ctx.complete()It will be output to the browser, but how does Tomcat know not to return to the client after the first doget request is completed? Key code inCoyoteAdapterMediumserviceMethod, part of the code is as follows:

postParseSuccess = postParseRequest(req, request, res, response);
   //Omit some codes
   if (postParseSuccess) {
    request.setAsyncSupported(
      connector.getService().getContainer().getPipeline().isAsyncSupported());
    connector.getService().getContainer().getPipeline().getFirst().invoke(
      request, response);
   }
   if (request.isAsync()) {
    async = true;
    } else {
    //Output data to client
    request.finishRequest();
    response.finishResponse();
   if (!async) {
    updateWrapperErrorCount(request, response);
    //Destroy request and response
    request.recycle();
    response.recycle();
   }

This part of the code is calledServletAfter that, it will passrequest.isAsync()To determine whether it is an asynchronous request. If it is an asynchronous request, setasync = true。 If it is a non asynchronous request, execute the output data to the client logic and destroy it at the same timerequestandresponse。 This completes the operation of not responding to the client after the request is completed.

Why is the @ enableasync annotation of spring boot not an asynchronous servlet

Because I inquired a lot of information when I was preparing to write this article, and found that many information about springboot asynchronous programming depends on@EnableAsyncComments, and thenControllerUse multithreading to complete the business logic. Finally, summarize the results and return the output. Here’s an example of the article by the Nuggets: springboot asynchronous programming guide that novices can understand. This article is very easy to understand and very good. From the business level, it is indeed asynchronous programming, but there is a problem. Aside from the parallel processing of business, it is not asynchronous for the whole request, In other words, the Tomcat thread cannot be released immediately, so the effect of asynchronous servlet cannot be achieved. Here, I also wrote a demo with reference to the above. Let’s verify why it is not asynchronous.


@RestController
@Slf4j
public class TestController {
 @Autowired
 private TestService service;

 @GetMapping("/hello")
 public String test() {
  try {
   log.info("testAsynch Start");
   CompletableFuture<String> test1 = service.test1();
   CompletableFuture<String> test2 = service.test2();
   CompletableFuture<String> test3 = service.test3();
   CompletableFuture.allOf(test1, test2, test3);
   log.info("test1=====" + test1.get());
   log.info("test2=====" + test2.get());
   log.info("test3=====" + test3.get());
  } catch (InterruptedException e) {
   e.printStackTrace();
  } catch (ExecutionException e) {
   e.printStackTrace();
  }
  return "hello";
 }
@Service
public class TestService {
 @Async("asyncExecutor")
 public CompletableFuture<String> test1() throws InterruptedException {
  Thread.sleep(3000L);
  return CompletableFuture.completedFuture("test1");
 }

 @Async("asyncExecutor")
 public CompletableFuture<String> test2() throws InterruptedException {
  Thread.sleep(3000L);
  return CompletableFuture.completedFuture("test2");
 }

 @Async("asyncExecutor")
 public CompletableFuture<String> test3() throws InterruptedException {
  Thread.sleep(3000L);
  return CompletableFuture.completedFuture("test3");
 }
}
@SpringBootApplication
@EnableAsync
public class TomcatdebugApplication {

 public static void main(String[] args) {
  SpringApplication.run(TomcatdebugApplication.class, args);
 }

 @Bean(name = "asyncExecutor")
 public Executor asyncExecutor() {
  ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
  executor.setCorePoolSize(3);
  executor.setMaxPoolSize(3);
  executor.setQueueCapacity(100);
  executor.setThreadNamePrefix("AsynchThread-");
  executor.initialize();
  return executor;
 }

Here I run to see the effect

Here, after I request, I make a breakpoint before calling the container to execute the business logic, and then I make the same breakpoint after returningControllerAfter execution, the request returnedCoyoteAdapterAnd judgerequest.isAsync(), according to the figure, isfalse, then it will be executed nextrequest.finishRequest()andresponse.finishResponse()To execute the end of the response and destroy the request and response body. The interesting thing is that when I did the experiment, I found that I was performingrequest.isAsync()Previously, the response body has appeared on the browser page, which is that the springboot framework has passedStringHttpMessageConverterClasswriteInternalMethod has been output.

The core logic of the above analysis is the thread execution of TomcatCoyoteAdapterAfter calling the container, you must wait until the request returns, then judge whether it is an asynchronous request, process the request, and then recycle it after execution. In my initial asynchronous servlet example, after the doget method is executed, it will return immediately, that is, it will go directly to therequest.isAsync()Then the logic of the whole thread is executed, and the thread is recycled.

Talk about the usage scenarios of asynchronous servlets

After analyzing so many, what are the usage scenarios of asynchronous servlets? In fact, we only need to grasp one point to analyze, that is, asynchronous servlets improve the system throughput and can accept more requests. Suppose that Tomcat threads in the web system are not enough and a large number of requests are waiting. At this time, the optimization at the application level of the web system can no longer be optimized, that is, the response time of business logic can not be shortened. At this time, if you want to reduce the waiting time of users and improve throughput, you can try to use asynchronous servlet.

Take a practical example: for example, in a short message system, the short message system requires high real-time performance, so the waiting time is required to be as short as possible. In fact, we entrust the operator to send the sending function, that is, we need to call the interface. Assuming the concurrency is very high, then the business system calls our sending short message function at this time, It is possible to use up our Tomcat thread pool and the remaining requests will wait in the queue. At that time, the SMS delay will go up. In order to solve this problem, we can introduce asynchronous servlet to accept more SMS sending requests, so as to reduce the SMS delay.

summary

In this article, I started with writing an asynchronous servlet, analyzed the role of asynchronous servlet and how to implement asynchronous servlet in tomcat, and then I explained it according to the popular springboot asynchronous programming on the Internet. It is not an asynchronous servlet in Tomcat. Finally, I talked about the usage scenarios of asynchronous servlets, and analyzed the circumstances under which asynchronous servlets can be tried.

The above is the whole content of this article. I hope it will be helpful to your study, and I hope you can support developpaer.

Recommended Today

JS generate guid method

JS generate guid method https://blog.csdn.net/Alive_tree/article/details/87942348 Globally unique identification(GUID) is an algorithm generatedBinaryCount Reg128 bitsNumber ofidentifier , GUID is mainly used in networks or systems with multiple nodes and computers. Ideally, any computational geometry computer cluster will not generate two identical guids, and the total number of guids is2^128In theory, it is difficult to make two […]