Interpretation of asp.net core 5-kestrel source code

Time:2021-12-2

Interpretation of asp.net core 5-kestrel source code

The last section talked about the configuration and use of kestrel server. I believe many students have a preliminary understanding of kestrel server. Then some students may want to have a deeper understanding of how kestrel server monitors and receives HTTP requests. Today, let’s take a look at the source code of kestrel server. I believe after reading these, You will certainly have a deeper understanding of the operating mechanism of kestrel server.

First, let’s start with the program startup class program. CS.

public class Program
{
    public static void Main(string[] args)
    {
        CreateHostBuilder(args).Build().Run();
    }
 
    public static IHostBuilder CreateHostBuilder(string[] args) =>
        Host.CreateDefaultBuilder(args)
            .ConfigureWebHostDefaults(webBuilder =>
             {
                 webBuilder.UseStartup();
             });
}

 

The host class chain calls two methods:

  • CreateDefaultBuilder
  • ConfigureWebHostDefaults

First, let’s take a lookCreateDefaultBuidlermethod:

public static IHostBuilder CreateDefaultBuilder(string[] args)
{
      HostBuilder hostBuilder = new HostBuilder();
      hostBuilder.UseContentRoot(Directory.GetCurrentDirectory());
      hostBuilder.ConfigureHostConfiguration((Action) (config =>
      {
        ...
      }));
      hostBuilder.ConfigureAppConfiguration((Action) ((hostingContext, config) =>
      {
        ...
      })).ConfigureLogging((Action) ((hostingContext, logging) =>
      {
        ...
      })).UseDefaultServiceProvider((Action) ((context, options) =>
      {
        ...
      }));
      return (IHostBuilder) hostBuilder;
    }
 }

 

As can be seen from the above code,CreateDefaultBuilderIt does not involve the relevant codes of kestrel server, but only the initialization configuration of some applications, such as setting the application directory, setting the configuration file, etc.

Let’s take another lookConfigureWebHostDefaultsmethod:

public static IHostBuilder ConfigureWebHostDefaults(
      this IHostBuilder builder,
      Action configure)
{
      if (configure == null)
        throw new ArgumentNullException(nameof (configure));
      return builder.ConfigureWebHost((Action) (webHostBuilder =>
      {
        Microsoft.AspNetCore.WebHost.ConfigureWebDefaults(webHostBuilder);
        configure(webHostBuilder);
      }));
}

By reading the source code, you can find:ConfigureWebHostDefaultsIn methodMicrosoft.AspNetCore.WebHost.ConfigureWebDefaults(IWebHostBuilder)Code that initializes the kestrel server for actual execution.

internal static void ConfigureWebDefaults(IWebHostBuilder builder)
{
     ...
     builder.UseKestrel((Action) ((builderContext, options) => options.Configure((IConfiguration) builderContext.Configuration.GetSection("Kestrel"), true))).ConfigureServices((Action) ((hostingContext, services) =>
     {
       services.PostConfigure((Action) (options =>
       {
        ...
       }
     })).UseIIS().UseIISIntegration();
}

See here, some students may be eager to see the code related to kestrel initialization process. Don’t worry. Let’s take it step by step.

First, let’s take a look at the aboveUseKestrelExtension method:

public static IWebHostBuilder UseKestrel(
      this IWebHostBuilder hostBuilder,
      Action configureOptions)
    {
      return hostBuilder.UseKestrel().ConfigureKestrel(configureOptions);
    }

It is found that this method only encapsulates the incoming configuration item kestrelserverpoptions, and finally calls the extension method of iwebhostbuilderUsekestrel and configurekestrel (action) configureOptions)Extend the method to initialize the kestrel server configuration, which is also a chain call.

Now let’s take a lookUseKestrel()This extension method:

public static IWebHostBuilder UseKestrel(this IWebHostBuilder hostBuilder)
{
    return hostBuilder.ConfigureServices((Action) (services =>
    {
      services.TryAddSingleton();
      services.AddTransient, KestrelServerOptionsSetup>();
      services.AddSingleton();
    }));
}

Careful students may find that only three lines of code are required to configure a kestrel server? Does it feel weird?Kestrel server is so simple? Yes, the kestrel server is that simple.So, how does the kestrel server listen and receive requests?

First, look at the iconnectionlistenerfactory interface class:

public interface IConnectionListenerFactory
{
    ValueTask BindAsync(
      EndPoint endpoint,
      CancellationToken cancellationToken = default (CancellationToken));
}

This interface has only one responsibility, which is to perform sokcert’s binding endpoint operation, and then return an iconnectionlistener object. Endpoint can be implemented in three ways:

  • FileHandleEndPoint
  • UnixDomainSocketEndPoint
  • IPEndPoint

Let’s take another look at the implementation class sockettransportfactory:

public sealed class SocketTransportFactory : IConnectionListenerFactory
{
    private readonly SocketTransportOptions _options;
    private readonly SocketsTrace _trace;
    public SocketTransportFactory(
      IOptions options,
      ILoggerFactory loggerFactory)
    {
      if (options == null)
        throw new ArgumentNullException(nameof (options));
      if (loggerFactory == null)
        throw new ArgumentNullException(nameof (loggerFactory));
      this._options = options.Value;
      this._trace = new SocketsTrace(loggerFactory.CreateLogger("Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets"));
    }
    public ValueTask BindAsync(
      EndPoint endpoint,
      CancellationToken cancellationToken = default (CancellationToken))
    {
      SocketConnectionListener connectionListener = new SocketConnectionListener(endpoint, this._options, (ISocketsTrace) this._trace);
      connectionListener.Bind();
      return new ValueTask((IConnectionListener) connectionListener);
    }
}

The code is very simple. First instantiate the SocketConnectionListener object, then invoke the Bind method of SocketConnectionListener and create the Socket object according to the incoming EndPoint type, so as to realize the monitoring and binding operation of EndPoint.

internal void Bind()
{
    if (this._listenSocket != null)
      throw new InvalidOperationException(SocketsStrings.TransportAlreadyBound);
    Socket listenSocket;
    switch (this.EndPoint)
    {
      case FileHandleEndPoint fileHandleEndPoint:
        this._socketHandle = new SafeSocketHandle((IntPtr) (long) fileHandleEndPoint.FileHandle, true);
        listenSocket = new Socket(this._socketHandle);
        break;
      case UnixDomainSocketEndPoint domainSocketEndPoint:
        listenSocket = new Socket(domainSocketEndPoint.AddressFamily, SocketType.Stream, ProtocolType.IP);
        BindSocket();
        break;
      case IPEndPoint ipEndPoint:
        listenSocket = new Socket(ipEndPoint.AddressFamily, SocketType.Stream, ProtocolType.Tcp);
        if (ipEndPoint.Address == IPAddress.IPv6Any)
          listenSocket.DualMode = true;
        BindSocket();
        break;
      default:
        listenSocket = new Socket(this.EndPoint.AddressFamily, SocketType.Stream, ProtocolType.Tcp);
        BindSocket();
        break;
    }
    this.EndPoint = listenSocket.LocalEndPoint;
    listenSocket.Listen(this._options.Backlog);
    this._listenSocket = listenSocket;
    void BindSocket()
    {
      try
      {
        listenSocket.Bind(this.EndPoint);
      }
      catch (SocketException ex) when (ex.SocketErrorCode == SocketError.AddressAlreadyInUse)
      {
        throw new AddressInUseException(ex.Message, (Exception) ex);
      }
    }
}

Now we know how to bind and listen inside the kestrel server. So how does the kestrel server receive and process HTTP requests?

Next, let’s look at the iserver interface:

public interface IServer : IDisposable
{
    IFeatureCollection Features { get; }
    Task StartAsync(IHttpApplication application, CancellationToken cancellationToken) where TContext : notnull;
    Task StopAsync(CancellationToken cancellationToken);
}

Iserver interface is also very simple. It defines a server with two basic functions: start and stop. So how does kestrel server implement this interface?

Let’s take a look at the implementation class kestrelserverimpl officially injected by Microsoft for iserver:

internal class KestrelServerImpl : IServer
{
    ...
    public IFeatureCollection Features { get; }
    public KestrelServerOptions Options => ServiceContext.ServerOptions;
    private ServiceContext ServiceContext { get; }
    private IKestrelTrace Trace => ServiceContext.Log;
    private AddressBindContext AddressBindContext { get; set; }
    public async Task StartAsync(IHttpApplication application, CancellationToken cancellationToken)
    {
        ...
        async Task OnBind(ListenOptions options)
        {
            if (!BitConverter.IsLittleEndian)
            {
                throw new PlatformNotSupportedException(CoreStrings.BigEndianNotSupported);
            }
            ValidateOptions();
            if (_hasStarted)
            {
                    // The server has already started and/or has not been cleaned up yet
                throw new InvalidOperationException(CoreStrings.ServerAlreadyStarted);
            }
            _hasStarted = true;

            ServiceContext.Heartbeat?.Start();
            if ((options.Protocols & HttpProtocols.Http3) == HttpProtocols.Http3)
            {
                if (_multiplexedTransportFactory is null)
                {
                    throw new InvalidOperationException($"Cannot start HTTP/3 server if no {nameof(IMultiplexedConnectionListenerFactory)} is registered.");
                }
 
                options.UseHttp3Server(ServiceContext, application, options.Protocols);
                var multiplexedConnectionDelegate = ((IMultiplexedConnectionBuilder)options).Build();
 
                multiplexedConnectionDelegate = EnforceConnectionLimit(multiplexedConnectionDelegate, Options.Limits.MaxConcurrentConnections, Trace);
                options.EndPoint = await _transportManager.BindAsync(options.EndPoint, multiplexedConnectionDelegate, options.EndpointConfig).ConfigureAwait(false);
            }
 
            if ((options.Protocols & HttpProtocols.Http1) == HttpProtocols.Http1
                || (options.Protocols & HttpProtocols.Http2) == HttpProtocols.Http2
                || options.Protocols == HttpProtocols.None) // TODO a test fails because it doesn't throw an exception in the right place
                                                                // when there is no HttpProtocols in KestrelServer, can we remove/change the test?
            {
               if (_transportFactory is null)
                {
                    throw new InvalidOperationException($"Cannot start HTTP/1.x or HTTP/2 server if no {nameof(IConnectionListenerFactory)} is registered.");
                }
                options.UseHttpServer(ServiceContext, application, options.Protocols);
                var connectionDelegate = options.Build();
                connectionDelegate = EnforceConnectionLimit(connectionDelegate, Options.Limits.MaxConcurrentConnections, Trace);
                options.EndPoint = await _transportManager.BindAsync(options.EndPoint, connectionDelegate, options.EndpointConfig).ConfigureAwait(false);
            }
         }

         AddressBindContext = new AddressBindContext
         {
             ServerAddressesFeature = _serverAddresses,
             ServerOptions = Options,
             Logger = Trace,
             CreateBinding = OnBind,
         };
         await BindAsync(cancellationToken).ConfigureAwait(false);
         ...
    }
 
    public async Task StopAsync(CancellationToken cancellationToken)
    {
            ...
    }
    ...
    private async Task BindAsync(CancellationToken cancellationToken)
    {
             ...
         await AddressBinder.BindAsync(Options.ListenOptions, AddressBindContext).ConfigureAwait(false);
             ...
    }
    ...
}

Let’s sort out the process of startasync method:

  1. Byte order check: bigendian is not supported
  2. Request parameter length verification, max. 8KB
  3. Determine whether the server has been started
  4. Start heartbeat detection
  5. Instantiate addressbindcontext for bindasync method use
  6. Execute the bindasync method to bind the address

Bindasync called the onbind method of addressbindcontext. The onbind method creates different httpconnectionmiddleware middleware according to the HTTP protocol type used and adds it to the connection pipeline to process HTTP requests.

The specific rules are as follows:

  • When the protocol is httpprotocols.http1/2, create httpconnectionmiddleware
  • When the protocol is httpprotocols.http3, create http3connectionmiddleware

At present, httpconnectionmiddleware is commonly used:

IConnectionBuilder UseHttpServer(
      this IConnectionBuilder builder,
      ServiceContext serviceContext,
      IHttpApplication application,
      HttpProtocols protocols)
    {
      HttpConnectionMiddleware middleware = new HttpConnectionMiddleware(serviceContext, application, protocols);
      return builder.Use((Func) (next => new ConnectionDelegate(middleware.OnConnectionAsync)));
    }

 

The usehttpserver method adds an httpconnectionmiddleware middleware to the connection pipeline (note that it is not the request pipeline in iaapplicationbuilder). When the request arrives, it will execute onconnectionasync method to create an HTTPCONNECTION object, and then process the HTTP request through this object:

public Task OnConnectionAsync(ConnectionContext connectionContext)
{
     IMemoryPoolFeature memoryPoolFeature = connectionContext.Features.Get();
     HttpConnectionContext context = new HttpConnectionContext();
     context.ConnectionId = connectionContext.ConnectionId;
     context.ConnectionContext = connectionContext;
     HttpProtocolsFeature protocolsFeature = connectionContext.Features.Get();
     context.Protocols = protocolsFeature != null ? protocolsFeature.HttpProtocols : this._endpointDefaultProtocols;
     context.ServiceContext = this._serviceContext;
     context.ConnectionFeatures = connectionContext.Features;
     context.MemoryPool = memoryPoolFeature?.MemoryPool ?? MemoryPool.Shared;
     context.Transport = connectionContext.Transport;
     context.LocalEndPoint = connectionContext.LocalEndPoint as IPEndPoint;
     context.RemoteEndPoint = connectionContext.RemoteEndPoint as IPEndPoint;
     return new HttpConnection(context).ProcessRequestsAsync(this._application);
}

Processrequestsasync is a specific method for processing requests. This method will create http1connection or http2connection according to the HTTP protocol version used, and then use this HTTPCONNECTION to create a context object (note that it is not an httpcontext object).

The kestrel server receives requests through transportmanager.bindasync in onbind.

public async Task BindAsync(
      EndPoint endPoint,
      ConnectionDelegate connectionDelegate,
      EndpointConfig? endpointConfig)
{
     if (this._transportFactory == null)
       throw new InvalidOperationException("Cannot bind with ConnectionDelegate no IConnectionListenerFactory is registered.");
     IConnectionListener connectionListener = await this._transportFactory.BindAsync(endPoint).ConfigureAwait(false);
     this.StartAcceptLoop((IConnectionListener) new TransportManager.GenericConnectionListener(connectionListener), (Func) (c => connectionDelegate(c)), endpointConfig);
     return connectionListener.EndPoint;}

 

The startacceptloop method is the method that actually receives data. Through the method name “start circular reception”, we guess whether the kestrel server receives data by circular listening to the socket accept method? So is it? Let’s continue to track the connectiondispatcher. Startacceptingconnections method:

public Task StartAcceptingConnections(IConnectionListener listener)
{
     ThreadPool.UnsafeQueueUserWorkItem>(new Action>(this.StartAcceptingConnectionsCore), listener, false);
     return this._acceptLoopTcs.Task;
}
private void StartAcceptingConnectionsCore(IConnectionListener listener)
{
     AcceptConnectionsAsync();
 
     async Task AcceptConnectionsAsync()
     {
       try
       {
         while (true)
         {
           T connectionContext = await listener.AcceptAsync(new CancellationToken());
           if ((object) connectionContext != null)
           {
             long id = Interlocked.Increment(ref ConnectionDispatcher._lastConnectionId);
             KestrelConnection kestrelConnection = new KestrelConnection(id, this._serviceContext, this._transportConnectionManager, this._connectionDelegate, connectionContext, this.Log);
             this._transportConnectionManager.AddConnection(id, (KestrelConnection) kestrelConnection);
             this.Log.ConnectionAccepted(connectionContext.ConnectionId);
             KestrelEventSource.Log.ConnectionQueuedStart((BaseConnectionContext) connectionContext);
             ThreadPool.UnsafeQueueUserWorkItem((IThreadPoolWorkItem) kestrelConnection, false);
           }
           else
             break;
         }
       }
       catch (Exception ex)
       {
         this.Log.LogCritical((EventId) 0, ex, "The connection listener failed to accept any new connections.");
       }
       finally
       {
         this._acceptLoopTcs.TrySetResult();
       }
  }
}

I believe you already know what’s going on? Originally, the kestrel server receives the user request data through the while (true) loop, and then distributes the request to the CLR thread pool for processing through the threadpool.unsafequeueuserworkitem method of the thread pool. In other words, when the request arrives, the transportmanager adds the onconnectionasync method to the thread pool and waits for the CLR thread pool to schedule.

So back to the beginning, how did the kestrel server start?

Let’s review the methods in program. CS again

public static void Main(string[] args)
{
   CreateHostBuilder(args).Build().Run();
}

I believe smart students have guessed that it is executed through the run () method. What does the run () method do?

The run method actually executes the startasync method in the host class. This method finally calls the startasync method of the iserver implementation class by obtaining the iserver class injected in the pre injected genericewebhostservice class.

internal class GenericWebHostService : IHostedService
{
  ...
  public IServer Server { get; }
  ...
    public async Task StartAsync(CancellationToken cancellationToken)
    {
     ...
     var httpApplication = new HostingApplication(application, Logger, DiagnosticListener, HttpContextFactory);
     await Server.StartAsync(httpApplication, cancellationToken);
     ...
    }
}

So far, kestrel has successfully started and started listening to user requests.

One sentence summary: in fact, the kestrel server in asp.net core 5 is just a simple encapsulation of the socket. It is as simple as directly using the socket to receive socket requests in a while (true) way, and directly putting it into the CLR thread pool to wait for the thread pool scheduling processing.

Originally, kestrel server is so simple~

I believe that through the introduction of this article, you have understood the kestrel server in asp.net core 5?