Usage of. Net performance testing framework crank

Time:2022-5-16

CrankIt is a new performance testing framework of Microsoft, which integrates a variety of benchmark tools, such as Bombardier, wrk and so on.

Through unified configuration, crank can be converted into different benchmark tool commands for testing. Can refer toBombardier Jobrealization.

Install crank

Run the following two commands to installCrankofcli(Controller)andAgent


dotnet tool update Microsoft.Crank.Controller --version "0.2.0-*" --global
dotnet tool update Microsoft.Crank.Agent--version "0.2.0-*" --global

need. Net SDK 5.0 environment

Execute the command after the installation is completedcrank, the following configurable parameters and descriptions will be printed out, orgithubView the introduction of relevant parameters.


PS C:\Users\Stack\Desktop> crank
Crank Benchmarks Controller

The Crank controller orchestrates benchmark jobs on Crank agents.

Usage: Crank [command] [options]

Options:
  -?|-h|--help         Show help information
  -c|--config          Configuration file or url
  -s|--scenario        Scenario to execute
  -j|--job             Name of job to define
  --profile            Profile name
  --script             Execute a named script available in the configuration files. Can be used multiple times.
  -j|--json            Saves the results as json in the specified file.
  --csv                Saves the results as csv in the specified file.
  --compare            An optional filename to compare the results to. Can be used multiple times.
  --variable           Variable
  --sql                Connection string of the SQL Server Database to store results in
  --table              Table name of the SQL Database to store results in
  --session            A logical identifier to group related jobs.
  --description        A string describing the job.
  -p|--property        Some custom key/value that will be added to the results, .e.g. --property arch=arm --property
                       os=linux

implementcrank-agent, start the agent service required for the benchmark.githubView the introduction of relevant parameters.


PS C:\Users\Stack\Desktop> crank-agent
Hosting environment: Production
Content root path: C:\Users\Stack\.dotnet\tools\.store\microsoft.crank.agent

PS C:\Users\Stack\Desktop> crank-agent
Hosting environment: Production
Content root path: C:\Users\Stack\.dotnet\tools\.store\microsoft.crank.agent\0.2.0-alpha.21567.1\microsoft.crank.agent\0.2.0-alpha.21567.1\tools\net5.0\any\
Now listening on: http://[::]:5010
.2.0-alpha.21567.1\microsoft.crank.agent

PS C:\Users\Stack\Desktop> crank-agent
Hosting environment: Production
Content root path: C:\Users\Stack\.dotnet\tools\.store\microsoft.crank.agent\0.2.0-alpha.21567.1\microsoft.crank.agent\0.2.0-alpha.21567.1\tools\net5.0\any\
Now listening on: http://[::]:5010
.2.0-alpha.21567.1\tools\net5.0\any\ Now listening on: http://[::]:5010

Create a crank profile

Configuration file reference officialhello.benchmarks.ymlExamples

Introduced in the sample filebombardier.yml, due to the influence of the environment on githubusercontent COM domain name is not very friendly, you can considerbombardier.ymlDownload to local,importsImport the local path or directly add the file content to the new configuration file.


imports:
  -https://raw.githubusercontent.com/dotnet/crank/main/src/Microsoft.Crank.Jobs.Bombardier/bombardier.yml

The recommended imports file is used in production. A small amount of configuration in the crank configuration file can complete the benchmark test, and different Microsoft. Com can be introduced Crank. Jobs. XXX/XXX. YML, based on different benchmarking tools.

Other test tool configuration files are available https://github.com/dotnet/crank/blob/main/src/Microsoft.Crank.XXX Down.

variables:
  headers:
    none: ''
    plaintext: '--header "Accept: text/plain,text/html;q=0.9,application/xhtml+xml;q=0.9,application/xml;q=0.8,*/*;q=0.7" --header "Connection: keep-alive"'
    html: '--header "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8" --header "Connection: keep-alive"'
    json: '--header "Accept: application/json,text/html;q=0.9,application/xhtml+xml;q=0.9,application/xml;q=0.8,*/*;q=0.7" --header "Connection: keep-alive"'
    connectionclose: '--header "Connection: close"'
  presetHeaders: none

jobs:
  bombardier:
    source:
      repository: https://github.com/dotnet/crank.git
      branchOrCommit: main
      project: src/Microsoft.Crank.Jobs.Bombardier/Microsoft.Crank.Jobs.Bombardier.csproj
      sourceKey: bombardier
      noBuild: true
    readyStateText: Bombardier Client
    waitForExit: true
    variables:
      Connections: 256 # sets the number of connections
      Warmup: 15 # set the test warm-up times
      Duration: 15 # set the test time
      Requests: 0 # sets the number of test request instances
      Rate: 0 # sets the request frequency per second
      Transport: fasthttp # | http1 | http2 set to send HTTP requests using the fasthttp Library of golang
      serverScheme: http
      serverAddress: localhost
      serverPort: 5000
      path: 
      bodyFile: # path or url for a file to use as the body content
      verb: # GET when nothing is specified
      customHeaders: [ ] # list of headers with the format: '<name1>: <value1>', e.g. [ 'content-type: application/json' ]
    arguments: "-c {{connections}} -w {{warmup}} -d {{duration}} -n {{requests}} --insecure -l {% if rate != 0 %} --rate {{ rate }} {% endif %} {% if transport %} --{{ transport}} {% endif %} {{headers[presetHeaders]}} {% for h in customHeaders %}{% assign s = h | split : ':' %}--header \"{{ s[0] }}: {{ s[1] | strip }}\" {% endfor %} {% if serverUri == blank or serverUri == empty %} {{serverScheme}}://{{serverAddress}}:{{serverPort}}{{path}} {% else %} {{serverUri}}:{{serverPort}}{{path}} {% endif %} {% if bodyFile != blank and bodyFile != empty %} -f {{bodyFile}} {% endif %}  {% if verb != blank and verb != empty %} -m {{verb}} {% endif %}"
    onConfigure: 
      # - job.timeout = Number(job.variables.duration) + Number(job.variables.warmup) + 10;

  server:
    Source: # specify the project to be tested. This article uses the local path directly
      localFolder: .
      project: crank_demo.csproj
    readyStateText: Application started.
    #Source: specify the remote warehouse address of the test project and specify the branch through branchorcommit
    #   repository: https://github.com/dotnet/crank
    #   branchOrCommit: main
    #   project: samples/hello/hello.csproj
    # readyStateText: Application started.

Scenarios: # configure benchmark scenarios
  crank_ Demo: # define the scheme name, which is specified when executing the crank command
    application: 
      Job: server # specifies that the test item is the server defined above
    load:
      Job: Bombardier # specifies the test tool Bombardier
      variables:
        Serverport: 5000 # configure HTTP service port
        Path: / # configure HTTP service address

profiles:
  local:
    variables:
      serverAddress: localhost
    jobs: 
      application:
        endpoints: 
          - http://localhost:5010
      load:
        endpoints: 
          - http://localhost:5010

Start crank agent

start-upagentPost executioncrank, there will be an action of installing SDK as follows


[09:29:05.261] Runtime: 6.0.0 (Current)
[09:29:05.262] SDK: 6.0.100 (Current)
[09:29:05.263] ASP.NET: 6.0.0 (Current)
[09:29:05.265] Creating custom global.json
[09:29:05.266] Desktop: 6.0.0 (Current)
[09:29:05.266] Installing SDK '6.0.100' ...

So startagentSpecify one more whendotnethomeParameter to avoid repeated installation of SDK.


crank-agent --dotnethome 'C:\Program Files\dotnet'

Start crank


crank --config .\demo.benchmarks.yml --scenario crank_demo --profile local 

--scenario crank_demo: specify the defined test scenario

--profile local: set the output result to local, i.e. console output

You can specify the output of the results to the local JSON file (- – output results. JSON) or database (- – SQL [connection string] – table [table name]) through parameters

Result output

Here, the log output during pressure measurement is omitted and the results are listed directly.


| application           |               |
| --------------------- | ------------- |
| CPU Usage (%)         | 56            |
| Cores usage (%)       | 447           |
| Working Set (MB)      | 140           |
| Private Memory (MB)   | 157           |
| Build Time (ms)       | 7,232         |
| Start Time (ms)       | 501           |
| Published Size (KB)   | 91,292        |
| .NET Core SDK Version | 6.0.100       |
| ASP.NET Core Version  | 6.0.0+ae1a6cb |
| .NET Runtime Version  | 6.0.0+4822e3c |


| load                  |                | 
| --------------------- | -------------- |
| CPU Usage (%)         | 46             | 
| Cores usage (%)       | 370            | 
| Working Set (MB)      | 29             |
| Private Memory (MB)   | 30             |
| Build Time (ms)       | 11,891         |
| Start Time (ms)       | 226            |
| Published Size (KB)   | 68,228         |
| .NET Core SDK Version | 3.1.415        |
| ASP.NET Core Version  | 3.1.21+458d974 |
| .NET Runtime Version  | 3.1.21+df8abc0 |
| First Request (ms)    | 185            |
| Requests              | 912,005        |
| Bad responses         | 0              |
| Mean latency (us)     | 4,207          |
| Max latency (us)      | 138,999        |
| Requests/sec          | 60,305         |
| Requests/sec (max)    | 128,523        |

More references

crank readme

BenchmarksCrank is the refactored version of benchmarks

Sample code for this article

The above is the whole content of this article. I hope it will be helpful to your study, and I hope you can support developpaer.