Ten thousand words: thoroughly understand the jest unit test framework

• What is jest
• What does testing mean
• How do I know what to test

• Test blocks, assertions, and matchers

  • How to implement test blocks
  • How to implement assertion and matcher
  • CLI and configuration

• simulation

  • How to simulate a function

• execution environment

  • Scope isolation
  • V8 virtual machine
  • Run single test callback
  • Hook function
• Generate report
• jest-cli
• jest-config
• jest-haste-map
• jest-runner
• jest-environment-node
• jest-circus
• jest-runtime
• Last & source code

Thoroughly understand the jest unit test framework

This article mainly gives you an in-depth understanding of the operation principle behind jest, and simply implements a jest unit test framework from scratch to facilitate understanding how the unit test engine works. I believe we are familiar with jest writing single test, but we may be unfamiliar with how jest works. Let’s go into jest’s heart together, Explore how the unit test engine works together.

First, attach the code implementation of the jest core engine to the students in need. Welcome to pay attention and communicate:https://github.com/Wscats/jest-tutorial

What is jest

Jest is a JavaScript testing framework developed by Facebook, which is used to create, run and write JavaScript libraries for testing.

Jest is published as an NPM package and can be installed and run in any JavaScript project. Jest is one of the most popular front-end test libraries.

What does testing mean

In technical terms, testing means checking that our code meets certain expectations. For example, one is called summation（sum）The function should return the expected output of a given number of operation results.

There are many types of tests, and soon you will be inundated with terms, but to make a long story short, the tests are divided into three categories:

• unit testing
• integration testing
• E2E test

How do I know what to test

In terms of testing, even the simplest code block may confuse beginners. The most common question is “how do I know what to test?”.

If you are writing a web page, a good starting point is to test each page of the application and each user interaction. However, web pages also need to be composed of code units such as tested functions and modules.

Most of the time, there are two situations:

• You inherit legacy code that comes with no tests
• You have to implement a new function out of thin air

What should I do? In both cases, you can see the test as: checking whether the function produces the expected results. The most typical test process is as follows:

• Import the function to test
• Give the function an input
• Define the desired output
• Check whether the function produces the expected output

Generally, it’s that simple. Mastering the following core ideas, writing tests will no longer be terrible:

Input – > expected output – > assertion result.

Test blocks, assertions, and matchers

We will create a simple JavaScript function code for the addition of two numbers, and write the corresponding jest based test for it

const sum = (a, b) => a + b;

Now, to test, create a test file in the same folder, namedtest.spec.js, this special suffix is jest’s convention, which is used to find all test files. We will also import the function under test to execute the code under test. The jest test follows the BDD style test, and each test should have a main testtestTest blocks, and there can be multiple test blocks, which can now besumMethod to write a test block. Here we write a test to add 2 numbers and verify the expected results. We will provide numbers 1 and 2 and expect output 3.

testIt requires two parameters: a string describing the test block and a callback function to wrap the actual test.expectWrap the objective function and combine it with the matchertoBeUsed to check whether the function calculation results meet expectations.

This is a complete test:

test("sum test", () => {
  expect(sum(1, 2)).toBe(3);
});

We observed the above code and found two points:

• testBlock is a separate test block, which has the function of describing and dividing the scope, that is, it represents the function we want to calculate for thissumA generic container for written tests.
• expectIs an assertion that calls the in the function under test using inputs 1 and 2sumMethod and expected output 3.
• toBeIs a matcher used to check the expected value. If the expected result is not met, an exception should be thrown.

How to implement test blocks

In fact, the test block is not complex. The simplest implementation is as follows. We need to store the callback function of the actual test in the test package, so we encapsulate onedispatchMethod to receive command types and callback functions:

const test = (name, fn) => {
  dispatch({ type: "ADD_TEST", fn, name });
};

We need to create one globallystateSave the callback function of the test. The callback function of the test is saved with an array.

global["STATE_SYMBOL"] = {
  testBlock: [],
};

dispatchAt this time, the method only needs to screen the corresponding command and save the tested callback function into the globalstateJust.

const dispatch = (event) => {
  const { fn, type, name } = event;
  switch (type) {
    case "ADD_TEST":
      const { testBlock } = global["STATE_SYMBOL"];
      testBlock.push({ fn, name });
      break;
  }
};

How to implement assertion and matcher

The implementation of the assertion library is also very simple. You only need to encapsulate a function and expose the matcher method to meet the following formula:

expect(A).toBe(B)

Here we realizetoBeThis common method throws an error when the result is not equal to the expectation:

const expect = (actual) => ({
    toBe(expected) {
        if (actual !== expected) {
            throw new Error(`${actual} is not equal to ${expected}`);
        }
    }
};

It will actually be used in the test blocktry/catchCatch errors and print stack information to locate problems.

In simple cases, we can also use the built-in function of nodeassertModule assertion, of course, there are many more complex assertion methods, which are essentially the same principle.

CLI and configuration

After writing the test, we need to enter the command in the command line to run the single test. Under normal circumstances, the command is similar to the following:

node jest xxx.spec.js

The essence here is to parse the parameters of the command line.

const testPath = process.argv.slice(2)[0];
const code = fs.readFileSync(path.join(process.cwd(), testPath)).toString();

In complex situations, you may also need to read the parameters of the local jest configuration file to change the execution environment. Jest uses a third-party library hereyargs execaandchalkWait to parse, execute and print the command.

simulation

In complex test scenarios, we can’t avoid a jest term: simulation（mock)

In the jest document, we can find jest’s description of simulation as follows: “the simulation function makes it easy to link the test code by erasing the actual implementation of the function, capturing the calls to the function and the parameters passed in these calls“

In short, you can create simulations by assigning the following code snippets to functions or dependencies:

jest.mock("fs", {
  readFile: jest.fn(() => "wscats"),
});

This is a simple simulation example, which simulates the return value of the FS module readfile function when testing specific business logic.

How to simulate a function

Next, we will study how to implement it. The first isjest.mock, the first parameter accepts the module name or module path, and the second parameter is the specific implementation of the external exposure method of the module

const jest = {
  mock(mockPath, mockExports = {}) {
    const path = require.resolve(mockPath, { paths: ["."] });
    require.cache[path] = {
      id: path,
      filename: path,
      loaded: true,
      exports: mockExports,
    };
  },
};

Our plan is actually the same as abovetestThe implementation of the test block is consistent. We only need to find a place to save the specific implementation method and replace it when the modified module is used in the future, so we save it torequire.cacheOf course, we can also save it to the overall situationstateYes.

andjest.fnThe implementation of is not difficult. Here we use a closuremockFnStore the replaced functions and parameters to facilitate subsequent test inspection and statistics of call data.

const jest = {
  fn(impl = () => {}) {
    const mockFn = (...args) => {
      mockFn.mock.calls.push(args);
      return impl(...args);
    };
    mockFn.originImpl = impl;
    mockFn.mock = { calls: [] };
    return mockFn;
  },
};

execution environment

Some students may have noticed that in the test framework, we do not need to introduce it manuallytest、expectandjestThese functions can be used directly by each test file, so we need to create a running environment for injecting these methods.

Scope isolation

Because the scope isolation is required when the single test file is running. Therefore, in design, the test engine runs under the global scope of node, while the code of test files runs under the local scope of VM virtual machine in node environment.

• global scopeglobal
• Local scopecontext

Two scopes passdispatchMethod to realize communication.

dispatchCollect test block, life cycle and test report information under the VM local scope to the node global scopeSTATE_SYMBOLMedium, sodispatchIt mainly involves the following communication types:

• Test block

  • ADD_TEST

• life cycle

  • BEFORE_EACH
  • BEFORE_ALL
  • AFTER_EACH
  • AFTER_ALL

• Test report

  • COLLECT_REPORT

V8 virtual machine

Since everything is ready, we only need to inject the method required for V8 virtual machine testing, that is, inject the local scope of testing.

const context = {
  console: console.Console({ stdout: process.stdout, stderr: process.stderr }),
  jest,
  expect,
  require,
  test: (name, fn) => dispatch({ type: "ADD_TEST", fn, name }),
};

After the scope is injected, we can make the code of the test file run in the V8 virtual machine. The code I pass in here is the code that has been processed into a string. Jest will do some code processing, security processing and sourcemap sewing here. Our example doesn’t need to be so complicated.

vm.runInContext(code, context);

Before and after code execution, you can use the time difference to calculate the running time of single test. Here, jest will pre evaluate the size and quantity of single test files to decide whether to enable worker to optimize the execution speed

const start = new Date();
const end = new Date();
log("\x1b[32m%s\x1b[0m", `Time: ${end - start} ms`);

Run single test callback

After the V8 virtual machine is executed, the globalstateAll wrapped test callback functions in the test block will be collected. Finally, we only need to traverse and take out all these callback functions and execute them.

testBlock.forEach(async (item) => {
  const { fn, name } = item;
  await fn.apply(this);
});

Hook function

We can also add a life cycle in the single test execution process, such asbeforeEach，afterEach，afterAllandbeforeAllWait for hook function.

Adding a hook function to the above infrastructure actually means injecting the corresponding callback function in each process of executing test, such asbeforeEachJust put it intestBlockBefore traversing and executing the test function,afterEachJust put it intestBlockAfter traversing and executing the test function, it is very simple. You can expose the hook function at any time only by placing it in the right position.

testBlock.forEach(async (item) => {
  const { fn, name } = item;
  beforeEachBlock.forEach(async (beforeEach) => await beforeEach());
  await fn.apply(this);
  afterEachBlock.forEach(async (afterEach) => await afterEach());
});

andbeforeAllandafterAllYou can put it in,testBlockBefore and after all tests are run.

beforeAllBlock.forEach(async (beforeAll) => await beforeAll());
testBlock.forEach(async (item) => {})
afterAllBlock.forEach(async (afterAll) => await afterAll());

Generate report

After the single test is executed, the information set of success and error can be collected,

try {
    dispatch({ type: "COLLECT_REPORT", name, pass: 1 });
    log("\x1b[32m%s\x1b[0m", `√ ${name} passed`);
} catch (error) {
    dispatch({ type: "COLLECT_REPORT", name, pass: 0 });
    log("\x1b[32m%s\x1b[0m", `× ${name} error`);
}

Then hijacklogThe detailed results can be printed on the terminal, or the report can be generated locally with the IO module.

const { reports } = global["STATE_SYMBOL"];
const pass = reports.reduce((pre, next) => pre.pass + next.pass);
log("\x1b[32m%s\x1b[0m", `All Tests: ${pass}/${reports.length} passed`);

So far, we have implemented the core of a simple jest test framework. The above parts basically implement test blocks, assertions, matchers, CLI configuration, function simulation, using virtual machines, scope and life cycle hook functions, etc. on this basis, we can enrich assertion methods, matchers and support parameter configuration. Of course, the actual implementation of jest will be more complex, I have only refined the key parts, so I attach my personal notes on reading the jest source code for your reference.

jest-cli

Download the jest source code and execute it in the root directory

yarn
npm run build

It essentially runs two files in the script folder, build JS and buildts js:

"scripts": {
    "build": "yarn build:js && yarn build:ts",
    "build:js": "node ./scripts/build.js",
    "build:ts": "node ./scripts/buildTs.js",
}

build. JS essentially uses the Babel library, creates a new build folder in package / xxx package, and then uses transformfilesync to generate files into the build folder:

const transformed = babel.transformFileSync(file, options).code;

And buildts JS essentially uses the TSC command, compiles the TS file into the build folder, and uses the execa library to execute the command:

const args = ["tsc", "-b", ...packagesWithTs, ...process.argv.slice(2)];
await execa("yarn", args, { stdio: "inherit" });

The successful execution will be shown as follows. It will help you compile all the JS and TS files in the packages folder into the build folder of the directory:

Next, we can start jest’s command:

npm run jest
#Equivalent to
# node ./packages/jest-cli/bin/jest.js

Here, parsing can be performed according to different parameters passed in, for example:

npm run jest -h
node ./packages/jest-cli/bin/jest.js /path/test.spec.js

Will executejest.jsFile, and then go tobuild/cliThe run method in the file will analyze various parameters in the command. The specific principle is that the yargs library cooperates with process Argv implementation

const importLocal = require("import-local");

if (!importLocal(__filename)) {
  if (process.env.NODE_ENV == null) {
    process.env.NODE_ENV = "test";
  }

  require("../build/cli").run();
}

jest-config

After obtaining various command parameters, it will be executedrunCLIThe core method is@jest/core -> packages/jest-core/src/cli/index.tsThe core method of library.

import { runCLI } from "@jest/core";
const outputStream = argv.json || argv.useStderr ? process.stderr : process.stdout;
const { results, globalConfig } = await runCLI(argv, projects);

runCLIMethod will use the passed in parameter argv parsed in the command just now to matchreadConfigsMethod to read the information of the configuration file,readConfigsFrompackages/jest-config/src/index.ts, there will be normalize to fill in and initialize some default configured parameters. Its default parameters arepackages/jest-config/src/Defaults.tsIt is recorded in the file. For example, if only JS single test is run, the default setting will be setrequire.resolve('jest-runner')In order to run the runner of single test, it will also generate OutputStream output content to the console in cooperation with the chalk library.

Here, by the way, the principle and idea of introducing the jest module will be introduced firstrequire.resolve(moduleName)Find the path of the module, save the path to the configuration, and then use the tool librarypackages/jest-util/src/requireOrImportModule.tsofrequireOrImportModuleMethod call encapsulated nativeimport/reqiureMethod to take out the module with the path in the configuration file.

• Globalconfig configuration from argv
• Configs comes from jest config. JS configuration

const { globalConfig, configs, hasDeprecationWarnings } = await readConfigs(
  argv,
  projects
);

if (argv.debug) {
  /*code*/
}
if (argv.showConfig) {
  /*code*/
}
if (argv.clearCache) {
  /*code*/
}
if (argv.selectProjects) {
  /*code*/
}

jest-haste-map

Jest haste map is used to get all the files in the project and the dependencies between themimport/requireCall to achieve this, extract them from each file and build a map, including each file and its dependencies. Here, haste is a module system used by Facebook, and it also has something called hastecontext, Because it has hastfs (haste file system), hastfs is just a list of files in the system and all dependencies associated with it. It is a map data structure, in which the key is the path and the value is metadata. It is generated herecontextsWill be used untilonRunCompletePhase.

const { contexts, hasteMapInstances } = await buildContextsAndHasteMaps(
  configs,
  globalConfig,
  outputStream
);

jest-runner

_run10000Method according to the configuration informationglobalConfigandconfigsobtaincontexts，contextsIt will store the configuration information and path of each local file, and then bring the callback functiononComplete, global configurationglobalConfigAnd scopecontextsget intorunWithoutWatchmethod.

The next step is to enterpackages/jest-core/src/runJest.tsDocumentrunJestIn the method, the passed method will be used herecontextsTraverse all the unit tests and save them in an array.

let allTests: Array<Test> = [];
contexts.map(async (context, index) => {
  const searchSource = searchSources[index];
  const matches = await getTestPaths(
    globalConfig,
    searchSource,
    outputStream,
    changedFilesPromise && (await changedFilesPromise),
    jestHooks,
    filter
  );
  allTests = allTests.concat(matches.tests);
  return { context, matches };
});

And useSequencerMethods to sort the single test

const Sequencer: typeof TestSequencer = await requireOrImportModule(
  globalConfig.testSequencer
);
const sequencer = new Sequencer();
allTests = await sequencer.sort(allTests);

runJestMethod calls a key methodpackages/jest-core/src/TestScheduler.tsofscheduleTestsmethod.

const results = await new TestScheduler(
  globalConfig,
  { startRun },
  testSchedulerContext
).scheduleTests(allTests, testWatcher);

scheduleTestsMethod will do a lot of things, willallTestsMediumcontextsCollectedcontextsMiddle, putdurationCollectedtimingsArray and subscribe to four life cycles before executing all single tests:

• test-file-start
• test-file-success
• test-file-failure
• test-case-result

Then putcontextsTraverse and use a new empty objecttestRunnersDo some processing and save it, which will be called@jest/transformProvidedcreateScriptTransformerMethod to handle the introduced module.

import { createScriptTransformer } from "@jest/transform";

const transformer = await createScriptTransformer(config);
const Runner: typeof TestRunner = interopRequireDefault(
  transformer.requireAndTranspileModule(config.runner)
).default;
const runner = new Runner(this._globalConfig, {
  changedFiles: this._context?.changedFiles,
  sourcesRelatedToTestsInChangedFiles: this._context?.sourcesRelatedToTestsInChangedFiles,
});
testRunners[config.runner] = runner;

andscheduleTestsMethod will callpackages/jest-runner/src/index.tsofrunTestsmethod.

async runTests(tests, watcher, onStart, onResult, onFailure, options) {
  return await (options.serial
    ? this._createInBandTestRun(tests, watcher, onStart, onResult, onFailure)
    : this._createParallelTestRun(
        tests,
        watcher,
        onStart,
        onResult,
        onFailure
      ));
}

final_createParallelTestRunperhaps_createInBandTestRunIn the method:

• _createParallelTestRun

There will be one in itrunTestInWorkerMethod, as its name implies, is to perform a single test in the worker.

• _createInBandTestRunIt will be executedpackages/jest-runner/src/runTest.tsA core methodrunTest, andrunJestThere is a method to executerunTestInternal, many things will be prepared before the execution of single test, including global method rewriting and hijacking of import and export methods.

await this.eventEmitter.emit("test-file-start", [test]);
return runTest(
  test.path,
  this._globalConfig,
  test.context.config,
  test.context.resolver,
  this._context,
  sendMessageToJest
);

stayrunTestInternalMethodfsThe module reads the contents of the file and puts it into thecacheFSFor example, if the content of the later file is JSON, it can be directly cached in thecacheFSRead, also useDate.nowThe time difference calculation takes time.

const testSource = fs().readFileSync(path, "utf8");
const cacheFS = new Map([[path, testSource]]);

stayrunTestInternalMethodpackages/jest-runtime/src/index.ts, it will help you cache and read modules and trigger execution.

const runtime = new Runtime(
  config,
  environment,
  resolver,
  transformer,
  cacheFS,
  {
    changedFiles: context?.changedFiles,
    collectCoverage: globalConfig.collectCoverage,
    collectCoverageFrom: globalConfig.collectCoverageFrom,
    collectCoverageOnlyFrom: globalConfig.collectCoverageOnlyFrom,
    coverageProvider: globalConfig.coverageProvider,
    sourcesRelatedToTestsInChangedFiles: context?.sourcesRelatedToTestsInChangedFiles,
  },
  path
);

jest-environment-node

Use here@jest/consoleThe package rewrites the global console so that the console of the single test file code block can print the results on the node terminal smoothlyjest-environment-nodePackage, put the globalenvironment.globalAll rewriting is to facilitate the subsequent method of obtaining these scopes in the VM. In essence, it is the scope provided for the VM’s running environment for subsequent injectionglobalProvide convenience, involving rewritingglobalThe methods are as follows:

• global.global
• global.clearInterval
• global.clearTimeout
• global.setInterval
• global.setTimeout
• global.Buffer
• global.setImmediate
• global.clearImmediate
• global.Uint8Array
• global.TextEncoder
• global.TextDecoder
• global.queueMicrotask
• global.AbortController

testConsoleIn essence, it is rewritten using the node’s console to facilitate subsequent coverage of the console method in the VM scope

testConsole = new BufferedConsole();
const environment = new TestEnvironment(config, {
  console: testConsole,
  docblockPragmas,
  testPath: path,
});
//How to really rewrite the console
setGlobal(environment.global, "console", testConsole);

runtimeThese two methods are mainly used to load modules. First, judge whether to use ESM module. If yes, useruntime.unstable_importModuleLoad the module and run it, if not, useruntime.requireModuleLoad the module and run it.

const esm = runtime.unstable_shouldLoadAsEsm(path);

if (esm) {
  await runtime.unstable_importModule(path);
} else {
  runtime.requireModule(path);
}

jest-circus

ThenrunTestInternalMediumtestFrameworkIt will accept the incoming runtime call to run the single test file,testFrameworkThe method comes from a library with an interesting namepackages/jest-circus/src/legacy-code-todo-rewrite/jestAdapter.ts, wherelegacy-code-todo-rewriteMeansLegacy code to-do rewriting，jest-circusIt will mainly focus on the overall situationglobalSome methods are rewritten, involving these:

• afterAll
• afterEach
• beforeAll
• beforeEach
• describe
• it
• test

Here, before calling the single test, you willjestAdapterFunction, which is mentioned aboveruntime.requireModuleloadxxx.spec.jsFile, which has been used before execution hereinitializeThe execution environment is presetglobalsandsnapshotState, and rewritebeforeEach, if configuredresetModules，clearMocks，resetMocks，restoreMocksandsetupFilesAfterEnvThe following methods will be executed:

• runtime.resetModules
• runtime.clearAllMocks
• runtime.resetAllMocks
• runtime.restoreAllMocks
• runtime. Requiremedule or runtime unstable_ importModule

When running is finishedinitializeAfter method initialization, becauseinitializeOverriding the globaldescribeandtestAnd other methods, these methods are/packages/jest-circus/src/index.tsRewrite here, pay attention heretestThere is one in the methoddispatchSyncMethod, which is a key method, will be maintained globallystate，dispatchSyncJust puttestThe functions and other information in the code block are stored in thestateInside,dispatchSyncInside usenamecoordinationeventHandlerMethod to modifystate, this idea is very similar to the data flow in redux.

const test: Global.It = () => {
  return (test = (testName, fn, timeout) => (testName, mode, fn, testFn, timeout) => {
    return dispatchSync({
      asyncError,
      fn,
      mode,
      name: "add_test",
      testName,
      timeout,
    });
  });
};

Single testxxx.spec.jsThat is, the testpath file will be displayed in theinitializeAfter that, it will be imported and executed. Note that the single test will be executed when it is imported herexxx.spec.jsThe document is written according to the specification, there will betestanddescribeWait for code blocks, so all at this timetestanddescribeAll accepted callback functions will be saved to the globalstateInside.

const esm = runtime.unstable_shouldLoadAsEsm(testPath);
if (esm) {
  await runtime.unstable_importModule(testPath);
} else {
  runtime.requireModule(testPath);
}

jest-runtime

Here, you will first determine whether to use the ESM module. If so, use theunstable_importModuleOtherwise, userequireModuleWill you enter the following function.

this._loadModule(localModule, from, moduleName, modulePath, options, moduleRegistry);

\_ The logic of loadmodule has only three main parts

• Judge whether to use JSON suffix file, execute readfile to read text, and use transformjson and JSON Parse the output of the grid.
• Judge whether the node suffix file is, and execute the require native method to introduce the module.
• If the above two conditions are not met, execute\_ Execmodule execution module.

\_ In execmodule, Babel will be used to convert the source code read by FStransformFilenamelypackages/jest-runtime/src/index.tsoftransformmethod.

const transformedCode = this.transformFile(filename, options);

\_ Used in execmodulecreateScriptFromCodeMethod calls the node’s native VM module to execute JS. The VM module accepts the secure source code and uses the V8 virtual machine with the incoming context to execute the code immediately or delay the execution of the code. Here, it can accept different scopes to execute the same code to calculate different results, which is very suitable for the use of the test framework, The vmcontext of the injection here is the above global rewrite scope, including afterall, aftereach, beforeall, beforeeach, describe, it, test, so our single test code will get these methods with the injection scope when running.

const vm = require("vm");
const script = new vm().Script(scriptSourceCode, option);
const filename = module.filename;
const vmContext = this._environment.getVmContext();
script.runInContext(vmContext, {
  filename,
});

When the global method is copied and saved abovestateAfter that, it will enter the real executiondescribeIn the logic of the callback function ofpackages/jest-circus/src/run.tsofrunMethod, which is used heregetStateMethoddescribeTake out the code block and use it_runTestsForDescribeBlockExecute this function, and then go to_runTestMethod, and then use_callCircusHookHook functions before and after execution, using_callCircusTestExecution.

const run = async (): Promise<Circus.RunResult> => {
  const { rootDescribeBlock } = getState();
  await dispatch({ name: "run_start" });
  await _runTestsForDescribeBlock(rootDescribeBlock);
  await dispatch({ name: "run_finish" });
  return makeRunResult(getState().rootDescribeBlock, getState().unhandledErrors);
};

const _runTest = async (test, parentSkipped) => {
  // beforeEach
  //Test function block, testcontext scope
  await _callCircusTest(test, testContext);
  // afterEach
};

This is the core position of hook function implementation and the core element of jest function.

last

I hope this article can help you understand the core implementation and principle of the jest test framework. Thank you for your patient reading. If the articles and notes can give you a little help or inspiration, please don’t be stingy with your star and fork. The articles are updated synchronously and continuously. Your comments must be the biggest driving force for me to move forward

• https://github.com/Wscats/jest-tutorial