Tomcat source code analysis series (12) nioendpoint

Time:2020-3-30

Preface
The previous article talked about the protocolhandler and its default implementation class http11nioprotocol, created a nioendpoint object in the construction method of http11nioprotocol, and the most important step in the init and start methods of http11nioprotocol is to call the init and start methods of this nioendpoint object. Nioendpoint inherits from abstractjsseendpoint, while abstractjsseendpoint inherits from abstractendpoint.


1. AbstractEndpoint#init
The init method of nioendpoint is in the parent class abstractendpoint of the parent class.

private boolean bindOnInit = true;

public final void init() throws Exception {
    if (bindOnInit) {
        bindWithCleanup();
        bindState = BindState.BOUND_ON_INIT;
    }
    if (this.domain != null) {
        // Register endpoint (as ThreadPool - historical name)
        oname = new ObjectName(domain + ":type=ThreadPool,name=\"" + getName() + "\"");
        Registry.getRegistry(null, null).registerComponent(this, oname, null);

        ObjectName socketPropertiesOname = new ObjectName(domain +
                ":type=ThreadPool,name=\"" + getName() + "\",subType=SocketProperties");
        socketProperties.setObjectName(socketPropertiesOname);
        Registry.getRegistry(null, null).registerComponent(socketProperties, socketPropertiesOname, null);

        for (SSLHostConfig sslHostConfig : findSslHostConfigs()) {
            registerJmx(sslHostConfig);
        }
    }
}
private void bindWithCleanup() throws Exception {
    try {
        bind();
    } catch (Throwable t) {
        // Ensure open sockets etc. are cleaned up if something goes
        // wrong during bind
        ExceptionUtils.handleThrowable(t);
        unbind();
        throw t;
    }
}

In the init method, the bindwithcleanup() method is called first, and then the registerjmx (sslhostconfig) method is called as required by sslhostconfig. The registerjmx method registers sslhostconfig to mbeanserver.
The bindwithcleanup() method just calls the bind() method. Bind() is an abstract method, but now nioendpoint class


2. NioEndpoint#bind

protected int acceptorThreadCount = 1;

private int pollerThreadCount = Math.min(2,Runtime.getRuntime().availableProcessors());

/**
 * Initialize the endpoint.
 */
@Override
public void bind() throws Exception {
    initServerSocket();

    // Initialize thread count defaults for acceptor, poller
    if (acceptorThreadCount == 0) {
        // FIXME: Doesn't seem to work that well with multiple accept threads
        acceptorThreadCount = 1;
    }
    if (pollerThreadCount <= 0) {
        //minimum one poller thread
        pollerThreadCount = 1;
    }
    setStopLatch(new CountDownLatch(pollerThreadCount));

    // Initialize SSL if needed
    initialiseSsl();

    selectorPool.open();
}

protected void setStopLatch(CountDownLatch stopLatch) {
    this.stopLatch = stopLatch;
}

In the bind method, first call the initserversocket method, and then initialize the acceptorthreadcount and pollerthreadcount. These two properties refer to the number of accepter threads and the number of poller threads. The acceptorthreadcount is 1 by default and the pollerthreadcount is 1 by default
Accepter and poller threads form the thread model of Tomcat,
Then create a countdownlatch object and assign it to the stoplatch property
Then call the initialisessl() method to initialize the SSL implementation class.
2.1. initialiseSsl

private String sslImplementationName = null;
private SSLImplementation sslImplementation = null;

public String getSslImplementationName() {
    return sslImplementationName;
}

protected void initialiseSsl() throws Exception {
    if (isSSLEnabled()) {
        sslImplementation = SSLImplementation.getInstance(getSslImplementationName());

        for (SSLHostConfig sslHostConfig : sslHostConfigs.values()) {
            sslHostConfig.setConfigType(getSslConfigType());
            createSSLContext(sslHostConfig);
        }

        // Validate default SSLHostConfigName
        if (sslHostConfigs.get(getDefaultSSLHostConfigName()) == null) {
            throw new IllegalArgumentException(sm.getString("endpoint.noSslHostConfig",
                    getDefaultSSLHostConfigName(), getName()));
        }

    }
}

The initialisessl () method is to create an implementation class of sslimplementation and assign it to the sslimplementation property.
Sslimplementation is an abstract class. Its implementation classes in Tomcat include jsseimplementation and opensslimplementation. Where jsseimplementation is the default implementation class.
See that the most important step of bind() is to call the initserversocket() method.

2.2 initServerSocket

/**
 * Server socket "pointer".
 */
private volatile ServerSocketChannel serverSock = null;

/**
 * Allows the server developer to specify the acceptCount (backlog) that
 * should be used for server sockets. By default, this value
 * is 100.
 */
private int acceptCount = 100;
public int getAcceptCount() { return acceptCount; }


/**
 * Use System.inheritableChannel to obtain channel from stdin/stdout.
 */
private boolean useInheritedChannel = false;
public boolean getUseInheritedChannel() { return useInheritedChannel; }


// Separated out to make it easier for folks that extend NioEndpoint to
// implement custom [server]sockets
protected void initServerSocket() throws Exception {
    if (!getUseInheritedChannel()) {
        serverSock = ServerSocketChannel.open();
        socketProperties.setProperties(serverSock.socket());
        InetSocketAddress addr = new InetSocketAddress(getAddress(), getPortWithOffset());
        serverSock.socket().bind(addr,getAcceptCount());
    } else {
        // Retrieve the channel provided by the OS
        Channel ic = System.inheritedChannel();
        if (ic instanceof ServerSocketChannel) {
            serverSock = (ServerSocketChannel) ic;
        }
        if (serverSock == null) {
            throw new IllegalArgumentException(sm.getString("endpoint.init.bind.inherited"));
        }
    }
    serverSock.configureBlocking(true); //mimic APR behavior
}

By default, useinheritedchannel is false, so if block will be used. In the if block, first initialize the serversock attribute of serversocketchannel type, and then set some attributes of serversocketchannel.

public void setProperties(ServerSocket socket) throws SocketException{
    if (rxBufSize != null)
        socket.setReceiveBufferSize(rxBufSize.intValue());
    if (performanceConnectionTime != null && performanceLatency != null &&
            performanceBandwidth != null)
        socket.setPerformancePreferences(
                performanceConnectionTime.intValue(),
                performanceLatency.intValue(),
                performanceBandwidth.intValue());
    if (soReuseAddress != null)
        socket.setReuseAddress(soReuseAddress.booleanValue());
    if (soTimeout != null && soTimeout.intValue() >= 0)
        socket.setSoTimeout(soTimeout.intValue());
}

Socketproperties are in abstractendpoint.

2.3 selectorPool.open()
The bind () method finally calls the selectorPool.open () method.

private NioSelectorPool selectorPool = new NioSelectorPool();

Selectorpool is an attribute in nioendpoint.

protected NioBlockingSelector blockingSelector;

protected volatile Selector SHARED_SELECTOR;


public void open() throws IOException {
    enabled = true;
    getSharedSelector();
    if (SHARED) {
        blockingSelector = new NioBlockingSelector();
        blockingSelector.open(getSharedSelector());
    }

}

protected Selector getSharedSelector() throws IOException {
    if (SHARED && SHARED_SELECTOR == null) {
        synchronized ( NioSelectorPool.class ) {
            if ( SHARED_SELECTOR == null )  {
                SHARED_SELECTOR = Selector.open();
            }
        }
    }
    return  SHARED_SELECTOR;
}

In the NioSelectorPool#open method, the SHARED_SELECTOR is initialized first, then a NioBlockingSelector object is created and assigned to the blockingSelector attribute, and then the open method of the object is invoked. Nioblockingselector is a class defined in Tomcat.

protected Selector sharedSelector;

protected BlockPoller poller;

public void open(Selector selector) {
    sharedSelector = selector;
    poller = new BlockPoller();
    poller.selector = sharedSelector;
    poller.setDaemon(true);
    poller.setName("NioBlockingSelector.BlockPoller-" + threadCounter.incrementAndGet());
    poller.start();
}

The input parameter of nioblockingselectorාopen method is the shared selector object in nioselectorpoolාopen. In the open method, the shared selector object is assigned to the sharedselector property, and then a blockpoller object is created and its start method is called. The parent class of blockpoller is thread, and the start of blockpoller is called Method is actually to start a thread. Blockpoller overloads the run method of thread.





3. NioEndpoint#start
The start method of nioendpoint is in the parent class abstractendpoint,

public final void start() throws Exception {
    if (bindState == BindState.UNBOUND) {
        bindWithCleanup();
        bindState = BindState.BOUND_ON_START;
    }
    startInternal();
}

Abstractendpoint ා in start, the startinternal() method is simply called, while nioendpoint overloads the startinternal method.

/**
 * Cache for SocketProcessor objects
 */
protected SynchronizedStack<SocketProcessorBase<S>> processorCache;

/**
 * Cache for poller events
 */
private SynchronizedStack<PollerEvent> eventCache;

/**
 * Bytebuffer cache, each channel holds a set of buffers (two, except for SSL holds four)
 */
private SynchronizedStack<NioChannel> nioChannels;


/**
 * Start the NIO endpoint, creating acceptor, poller threads.
 */
@Override
public void startInternal() throws Exception {

    if (!running) {
        running = true;
        paused = false;

        processorCache = new SynchronizedStack<>(SynchronizedStack.DEFAULT_SIZE,
                socketProperties.getProcessorCache());
        eventCache = new SynchronizedStack<>(SynchronizedStack.DEFAULT_SIZE,
                        socketProperties.getEventCache());
        nioChannels = new SynchronizedStack<>(SynchronizedStack.DEFAULT_SIZE,
                socketProperties.getBufferPool());

        // Create worker collection
        if ( getExecutor() == null ) {
            createExecutor();
        }

        initializeConnectionLatch();

        // Start poller threads
        pollers = new Poller[getPollerThreadCount()];
        for (int i=0; i<pollers.length; i++) {
            pollers[i] = new Poller();
            Thread pollerThread = new Thread(pollers[i], getName() + "-ClientPoller-"+i);
            pollerThread.setPriority(threadPriority);
            pollerThread.setDaemon(true);
            pollerThread.start();
        }

        startAcceptorThreads();
    }
}

In the startinternal method, three synchronized stack objects are created and assigned to processorcache, eventcache and niochannels respectively. These three attributes are used for reuse, reusing socketprocessorbase object, pollerevent object and niochannel object respectively. Processorcache is declared in abstractendpoint, and the other two are declared in nioendpoint.

Then, call createexecutor(). Createexecutor declared in abstractendpoint

private Executor executor = null;
public Executor getExecutor() { return executor; }

public void createExecutor() {
    internalExecutor = true;
    TaskQueue taskqueue = new TaskQueue();
    TaskThreadFactory tf = new TaskThreadFactory(getName() + "-exec-", daemon, getThreadPriority());
    executor = new ThreadPoolExecutor(getMinSpareThreads(), getMaxThreads(), 60, TimeUnit.SECONDS,taskqueue, tf);
    taskqueue.setParent( (ThreadPoolExecutor) executor);
}

The createexecutor method creates a thread pool and assigns it to the executor property.

Next, the startinternal method calls the initializeconnectlatch method,

protected LimitLatch initializeConnectionLatch() {
    if (maxConnections==-1) return null;
    if (connectionLimitLatch==null) {
        connectionLimitLatch = new LimitLatch(getMaxConnections());
    }
    return connectionLimitLatch;
}

The initializeconnection latch method initially changes the connectionlimitlatch property, which is used to limit the maximum number of connections of Tomcat.

Then, startinternal creates pollerthreadcount poller objects and threads, and starts these threads, which become poller threads. The poller class implements the runnable interface.

Finally, startinternal calls the startacceptorthreads () method.

protected void startAcceptorThreads() {
    int count = getAcceptorThreadCount();
    acceptors = new ArrayList<>(count);

    for (int i = 0; i < count; i++) {
        Acceptor<U> acceptor = new Acceptor<>(this);
        String threadName = getName() + "-Acceptor-" + i;
        acceptor.setThreadName(threadName);
        acceptors.add(acceptor);
        Thread t = new Thread(acceptor, threadName);
        t.setPriority(getAcceptorThreadPriority());
        t.setDaemon(getDaemon());
        t.start();
    }
}

In the startacceptorthreads method, acceptorthreadcount acceptor objects and threads are created and started. These threads are called acceptor threads. Acceptor, like poller, also implements the runnable interface.

The acceptor thread handles client connections, while poller handles read and write events on these connection channels. Acceptor and poller constitute the thread model of tomcat, which is a very important component. The following article will be explained separately, and will not be discussed here.


Summary
This paper introduces the init and start methods of nioendpoint. In the init method, the serversocketchannel object (in the nioendpoint ා initserversocket method) and a selector object (in the nioselectorpool ා open method) are created. In the start method, the acceptor thread and the poller thread are started.

Recommended Today

Configure Apache to support PHP in the Apache main configuration file httpd.conf Include custom profile in

In Apache’s main configuration file / conf/ http.conf Add at the bottom Include “D:workspace_phpapache-php.conf” The file path can be any In D: workspace_ Create under PHP file apache- php.conf file Its specific content is [html] view plain copy PHP-Module setup LoadFile “D:/xampp/php/php5ts.dll” LoadModule php5_module “D:/xampp/php/php5apache2_2.dll” <FilesMatch “.php$”> SetHandler application/x-httpd-php </FilesMatch> <FilesMatch “.phps$”> SetHandler application/x-httpd-php-source </FilesMatch> […]