How to improve the performance of JSON. stringify ()?

Time:2019-7-29

1. FamiliarJSON.stringify()

On the browser or server side,JSON.stringify()They are the most commonly used methods:

  • Store JSON object in local Storage;
  • JSON body in POST request;
  • Processing data in JSON form in response body;
  • Even under certain conditions, we will use it to achieve a simple deep copy.
  • ……

In some performance-sensitive situations (such as server handling large amounts of concurrency) or in the face of a large number of stringify operations, we hope that it will perform better and faster. This also leads to some optimized stringify schemes/libraries, which are compared with the performance of the original method in the following figure:

How to improve the performance of JSON. stringify ()?

The green part is nativeJSON.stringify()Visible performance is much lower than those of these libraries. So what are the technical principles behind the dramatic performance improvements?

2. ratiostringifyFasterstringify

Because JavaScript is a dynamic language, for a variable of Object type, its key name, key value and key type can only be determined at runtime. Therefore, implementationJSON.stringify()There will be a lot of work to do. In the absence of any knowledge, there is obviously nothing we can do to optimize substantially.

So what if we knew the key name and the key value information in this Object — that is, the structure information of it, would it help?

Let’s take an example:

The following Object,

const obj = {
    name: 'alienzhou',
    status: 6,
    working: true
};

We apply itJSON.stringify()The result is

JSON.stringify(obj);
// {"name":"alienzhou","status":6,"working":true}

Now if we know thisobjThe structure is fixed:

  • Keyname unchanged
  • The type of key must be certain

In fact, I can create a “customized” stringify method.

function myStringify(o) {
    return (
        '{"name":"'
        + o.name
        + '","status":'
        + o.status
        + ',"isWorking":'
        + o.working
        + '}'
    );
}

Look at ourmyStringifyMethod output:

myStringify({
    name: 'alienzhou',
    status: 6,
    working: true
});
// {"name":"alienzhou","status":6,"isWorking":true}

myStringify({
    name: 'mengshou',
    status: 3,
    working: false
});
// {"name":"mengshou","status":3,"isWorking":false}

You can get the right results, but only type conversion and string splicing are used, so “customization” can make “stringify” faster.

To sum up, how to get a comparisonstringifyFasterstringifyHow?

  1. The structure information of the object needs to be determined first.
  2. According to its structure information, create “customized” objects for this structurestringifyMethod: The inner part of the method is actually generated by string splicing.
  3. Finally, use the “customized” approach to stringify objects.

This is also the routine of most stringify acceleration libraries, which translates into code like this:

import faster from 'some_library_faster_stringify';

// 1. Define your object structure by the corresponding rules
const theObjectScheme = {
    // ……
};

// 2. According to the structure, a customized method is obtained.
const stringify = faster(theObjectScheme);

// 3. Call method, fast stringify
const target = {
    // ……
};
stringify(target);

3. How to generate customized methods

According to the above analysis, the core function is:According to its structure information, a “customized” stringify method is created for this kind of object. Its internal reality is simple attribute access and string splicing.

In order to understand the specific implementation, I will briefly introduce two open source libraries with slightly different implementations.

3.1. fast-json-stringify

How to improve the performance of JSON. stringify ()?

The following figure is a performance comparison based on the benchmark results provided by fast-json-stringify.

How to improve the performance of JSON. stringify ()?

As you can see, in most scenarios there is a 2-5 times performance improvement.

3.1.1. The Definition of Scheme

Fast-json-stringify uses JSON Schema Validation to define the data format of (JSON) objects. The structure of its scheme definition itself is also in JSON format, such as objects.

{
    name: 'alienzhou',
    status: 6,
    working: true
}

The corresponding scheme is:

{
    title: 'Example Schema',
    type: 'object',
    properties: {
        name: {
            type: 'string'
        },
        status: {
            type: 'integer'
        },
        working: {
            type: 'boolean'
        }
    }
}

Its scheme definition rules are rich, and the specific use can refer to Ajv, the JSON checking library.

3.1.2. Generation of stringify method

Fast-json-stringify will concatenate the actual function code string according to the scheme just defined, and then use the Function constructor to dynamically generate the corresponding stringify function at run time.

In code generation, it first injects pre-defined tools and methods, which are the same for different schemes:

var code = `
    'use strict'
  `

  code += `
    ${$asString.toString()}
    ${$asStringNullable.toString()}
    ${$asStringSmall.toString()}
    ${$asNumber.toString()}
    ${$asNumberNullable.toString()}
    ${$asIntegerNullable.toString()}
    ${$asNull.toString()}
    ${$asBoolean.toString()}
    ${$asBooleanNullable.toString()}
  `

Secondly, the concrete code of stringify function will be generated according to the specific content defined by scheme. And the way to generate it is relatively simple: by traversing the scheme.

When traversing the scheme, insert the corresponding tool function at the corresponding code for key value conversion according to the type defined. For example, in the example abovenameThis property:

var accessor = key.indexOf('[') === 0 ? sanitizeKey(key) : `['${sanitizeKey(key)}']`
switch (type) {
    case 'null':
        code += `
            json += $asNull()
        `
        break
    case 'string':
        code += nullable ? `json += obj${accessor} === null ? null : $asString(obj${accessor})` : `json += $asString(obj${accessor})`
        break
    case 'integer':
        code += nullable ? `json += obj${accessor} === null ? null : $asInteger(obj${accessor})` : `json += $asInteger(obj${accessor})`
        break
    ……

In the code abovecodeVariables are saved as code strings of the last generated function body. Because in scheme definition,namebystringType, and not empty, so it willcodeAdd the following code string:

"json += $asString(obj['name'])"

Because of the complexity of arrays and associated objects, the actual code is omitted a lot.

Then, the complete generatedcodeThe strings are roughly as follows:

function $asString(str) {
    // ……
}
function $asStringNullable(str) {
    // ……
}
function $asStringSmall(str) {
    // ……
}
function $asNumber(i) {
    // ……
}
function $asNumberNullable(i) {
    // ……
}
/* These are a series of general key value conversion methods.*/

/* $main is the main function of stringify*/
function $main(input) {
    var obj = typeof input.toJSON === 'function'
        ? input.toJSON()
        : input

    var json = '{'
    var addComma = false
    if (obj['name'] !== undefined) {
        if (addComma) {
            json += ','
        }
        addComma = true
        json += '"name":'
        json += $asString(obj['name'])
    }

    //... Stitching of other attributes (status, work)

    json += '}'
    return json
}

return $main

Finally, thecodeThe string is passed into the Function constructor to create the corresponding stringify function.

// Dependencies are mainly used to deal with situations involving anyOf and if grammars
dependenciesName.push(code)
return (Function.apply(null, dependenciesName).apply(null, dependencies))

3.2. slow-json-stringify

How to improve the performance of JSON. stringify ()?

Slow-json-stringify is actually a “fast” stringify library (naughty name).

The slowest stringifier in the known universe. Just kidding, it’s the fastest (:

Its implementation is lighter and clever than the fast-json-stringify mentioned earlier. At the same time, it is more efficient in many scenarios than fast-json-stringify.

How to improve the performance of JSON. stringify ()?

How to improve the performance of JSON. stringify ()?

3.2.1. Definition of scheme

The scheme definition of slow-json-stringify is more natural and simple, which mainly replaces the key value with the type description. As an example of the above object, scheme becomes

{
    name: 'string',
    status: 'number',
    working: 'boolean'
}

It’s really very intuitive.

3.2.2. Generation of stringify method

I wonder if you noticed it.

// scheme
{
    name: 'string',
    status: 'number',
    working: 'boolean'
}

// Target object
{
    name: 'alienzhou',
    status: 6,
    working: true
}

Is the structure of scheme similar to that of the original object?

The ingenuity of this scheme is that after defining it, we can start with scheme.JSON.stringifyFirst, we “deduct” all the type values, and finally we will fill the actual values directly into the type declaration corresponding to scheme.

How to operate it?

First, you can call scheme directlyJSON.stringify()To generate basic templates and borrow them at the same timeJSON.stringify()The second parameter is the access path to collect attributes as a traversal method:

let map = {};
const str = JSON.stringify(schema, (prop, value) => {
    const isArray = Array.isArray(value);
    if (typeof value !== 'object' || isArray) {
        if (isArray) {
            const current = value[0];
            arrais.set(prop, current);
        }

        _validator(value);

        map[prop] = _deepPath(schema, prop);
        props += `"${prop}"|`;
    }
    return value;
});

At this time,mapIt collects access paths for all attributes. Simultaneous generationpropsRegular expressions that can be spliced together to match the corresponding type of characters, such as the regular expression in our example/name|status|working"(string|number|boolean|undef)"|\\[(.*?)\\]/

Then, these attributes are matched sequentially according to regular expressions, replacing the string of attribute types with a unified placeholder string."__par__"And based on"__par__"Split string:

const queue = [];
const chunks = str
    .replace(regex, (type) => {
      switch (type) {
        case '"string"':
        case '"undefined"':
          return '"__par__"';
        case '"number"':
        case '"boolean"':
        case '["array-simple"]':
        case '[null]':
          return '__par__';
        default:
          const prop = type.match(/(?<=\").+?(?=\")/)[0];
          queue.push(prop);
          return type;
      }
    })
    .split('__par__');

So you’ll get it.chunksandpropsTwo arrays.chunksIt contains the split JSON string. For example, the two arrays are as follows

// chunks
[
    '{"name":"',
    '","status":"',
    '","working":"',
    '"}'
]

// props
[
    'name',
    'status',
    'working'
]

Finally, because the mapping between attribute names and access paths is preserved in map, the value of an attribute in the object can be accessed according to prop, and the array can be iterated through, then it can be spliced with the corresponding chunks.

From the point of view of code quantity and implementation method, this scheme will be more convenient and ingenious, and it does not need to dynamically generate or execute functions through Function, eval, etc.

4. Summary

Although different libraries have different implementations, the way to achieve high-performance stringify is the same from the overall point of view:

  1. The developer defines the JSON scheme of Object.
  2. Stringify library generates the corresponding template method according to scheme. In the template method, attributes and values are stitched together by strings (obviously, the efficiency of attribute access and string stitching is much higher).
  3. Finally, the developer calls the returned method stringify Object.

In the final analysis, it essentially preposes optimization and analysis through static structural information.

Tips

Finally, I would like to mention that.

  • All benchmarks can only be used as a reference, whether there is performance improvement, how much improvement or suggest that you test in the actual business;
  • Function constructors are used in fast-json-stringify, so it is recommended not to use user input directly as a scheme in case of security problems.