Fast generation of parent in front end_ Tree structure data associated with ID

Time:2021-1-15

If you want to see the conclusion directly, you can jump toFast generation method

In tob projects, it is inevitable to process tree related data, such as multi-level Department list, multi-level distribution list, and so on. For all cascading data with upper and lower levels, it is inevitable to generate tree structure data.

There are many storage methods for tree structured data in the back end, and the most common one is parent_ ID is the upper and lower level association table. Of course, there are other storage structures, such as left and right value tree table.

For the time being, this paper only discusses parent_ In this case, the data in the back end range is the list data of all tree nodes.

Preparation – test data generation code

In order to facilitate testing, first write a random generation of parent_ The code of list data associated with ID

code

// genTestData.js
function getRandomByRange(start, end) {
  let length = end - start
  return start + (Math.random() * length) >>> 0
}

function getRandomCode() {
  return Math.random().toString(36).slice(2)
}

function getRandomCodeByData(data) {
  return data[getRandomByRange(0, data.length)].code
}

export default function() {
  let data = []
  //Randomly generate 20-30 primary nodes
  for (let i = 0, l = getRandomByRange(20, 30); i < l; i++) {
    data.push({
      facode: '0',
      code: getRandomCode()
    })
  }
  //Randomly generate 100-120 random association nodes
  for (let i = 0, l = getRandomByRange(100, 120); i < l; i++) {
    data.push({
      facode: getRandomCodeByData(data),
      code: getRandomCode()
    })
  }
  return data.sort(() => (Math.random() - 0.5))
}

ModifiablegetRandomByRangeTo generate different amount of test data

test

The test generation effect is as follows

// demo.js
import genTestData from './genTestData.js'
const oriData = genTestData()
console.table(oriData)

Now that we have the test data, let’s get to the point

General generation method

thinking

Starting from the root node, traverse the data source, find the child node of the current node each time, and hang it to the node of the current nodechildrenAnd then call recursively

code

// tree.js
export class TreeCreate {
  constructor(treeNode, treeConfig) {
    this.treeNode = treeNode
    this.treeConfig = Object.assign({
      FID: 'PID', // associated parent_ ID field name
      ID: 'ID', // associated parent_ ID field name of ID
      Rootid: '0' // the parent associated with the starting root node_ Field name corresponding to ID
    }, treeConfig)
    this.treeData  =[] // tree data
  }
  //Tree generation, recursive call
  createTree(data, parent = null) {
    let config = this.treeConfig
    let TreeNode = this.treeNode
    let treeData, pid, i, l, val, children
    treeData = []
    for (i = 0, l = data.length; i < l; i++) {
      val = data[i]
      if (parent === null) {
        pid = config.rootId
      } else {
        pid = parent[config.id]
      }
      if (val[config.fId] === pid) {
        let t = {}
        TreeNode.call(t, val)
        children = this.createTree(data, Object.assign(t, val))
        treeData.push(t)
        treeData[treeData.length - 1].children = children
      }
    }
    return treeData
  }
  //Spanning tree
  create(data) {
    this.treeData = this.createTree(data)
  }
  //Get tree data of tree (deep copy)
  getTreeData() {
    //If the tree node needs to retain the parent reference, it needs to change to a deep copy method that can handle the circular reference
    return JSON.parse(JSON.stringify(this.treeData))
  }
}

Usage

treeNodePass in the tree node data generation function. You cannot use the arrow function. This is the current node and the default is empty. Accept the parameter item as the original data object, and assign fields to this object as needed. If you need a full field, you can use theObject.keysTraverse all the fields of the parameter, and then assign all the values to the corresponding fields of this.
treeConfigConfigure parameters for

  • FIdAssociated parent_ ID field name
  • IdAssociated parent_ ID field name of ID
  • rootIdThe parent associated with the starting root node_ Field name corresponding to ID

Test performance

import genTestData from './genTestData.js'
import { TreeCreate } from './tree.js'
//Test data
const oriData = genTestData()
console.log (number of tree nodes)${ oriData.length >)

//Trees
let demoTree = new TreeCreate(function(item) {
  for (let key of Object.keys(item)) {
    this[key] = item[key]
  }
}, { fId: 'facode', id: 'code', rootId: '0' })

console.time ('spanning tree time ')
demoTree.create(oriData, true)
console.timeEnd ('spanning tree time ')

Let’s test the performance

Hundred level data

Thousand level data

You can see that the performance has decreased significantly

Ten thousand data

It takes nearly 3 seconds, which will obviously cause page jam

At this time, let’s find out why the generator function decreases so seriously with the increase of n.
If you think about the algorithm just now, it is not difficult to find that its time complexity is basically o (N 2), which will increase rapidly with the increase of n.

If the amount of data is changed to 100000 level, the page is basically stuck. Of course, under normal circumstances, the front end will not need to process 100000 level data, and the back end must be lazy, but the normal 10000 level data may still need to be processed.

How to optimize

After careful thinking about the repeated running of the code, we find that every time we find the children of a certain node, we will traverse all the data sources to determine whether it is the children of the current node.

At this time, the optimization point of the first reaction is to delete node a from the data source when node a is already the children of node B. the next time you go to find the children of node C, you will not traverse to node a.

You can try this on your own. It’s effective, but not obvious, because it doesn’t change the nature of a node being traversed many times. For example, if node a is the child of the last node, then the previous generation of children will still traverse node a every time, and node a will still be traversed many times.
And removing a node from the data source is actually a time-consuming operation.

To sum up, the time-consuming is caused by invalid multiple traversal of nodes, so can we only traverse once, and then use the conventional operation of space for time to optimize the traversal.
At this time, we will think of the JS map object, hash storage, through the key can directly get value.
If you use FID as the key and store all the children’s information as the value, then every time you look up the children of the current node, it will change from traversal to value from the map, because the hash storage is basically ignored.

Fast generation method

thinking

As mentioned in the optimization above, use the key value feature of JS map to search all parent-child relationships at one time, and then recursively generate them

code

// tree.js
export class TreeCreateFast {
  constructor(treeNode, treeConfig) {
    this.treeNode = treeNode
    this.treeConfig = Object.assign({
      FID: 'PID', // associated parent_ ID field name
      ID: 'ID', // associated parent_ ID field name of ID
      Rootid: '0' // the parent associated with the starting root node_ Field name corresponding to ID
    }, treeConfig)
    this.treeData  =[] // tree data of tree
  }
  //Group by FID
  groupBy(data) {
    let config = this.treeConfig
    let group = []
    data.forEach(v => {
      let key = v[config.fId]
      if (!group.hasOwnProperty(key)) {
        group[key] = [v]
      } else {
        group[key].push(v)
      }
    })
    return group
  }
  //Tree generation, recursive call
  createTree(data, parent = null) {
    let config = this.treeConfig
    let TreeNode = this.treeNode
    let treeData, pid, children
    treeData = []
    if (parent === null) {
      pid = config.rootId
    } else {
      pid = parent[config.id]
    }
    if (data.hasOwnProperty(pid)) {
      data[pid].forEach(val => {
        let t = {}
        TreeNode.call(t, val)
        children = this.createTree(data, Object.assign(t, val))
        treeData.push(t)
        treeData[treeData.length - 1].children = children
      })
    }
    return treeData
  }
  //Spanning tree
  create(data) {
    this.treeData = this.createTree(this.groupBy(data))
  }
  //Get tree data of tree (deep copy)
  getTreeData() {
    //If the tree node needs to retain the parent reference, it needs to change to a deep copy method that can handle the circular reference
    return JSON.parse(JSON.stringify(this.treeData))
  }
}

Usage

It is used in the same way as the normal tree generating function

Test performance

import genTestData from './genTestData.js'
import { TreeCreateFast } from './tree.js'
//Test data
const oriData = genTestData()
console.log (number of tree nodes)${ oriData.length >)

//Trees
let demoTree = new TreeCreateFast(function(item) {
  for (let key of Object.keys(item)) {
    this[key] = item[key]
  }
}, { fId: 'facode', id: 'code', rootId: '0' })

console.time ('spanning tree time ')
demoTree.create(oriData, true)
console.timeEnd ('spanning tree time ')

Test the performance directly from the 10000 level

Ten thousand data

In terms of the amount of data that can be touched by a conventional project, it is already in the state of no sense, and it will basically block a frame

100000 level data

In the acceptable range, after all, there can be no 100000 level data to be processed

Challenge the next million

Just play it. It doesn’t make any sense

summary

After optimizing the traversal through map and solving the repeated useless traversal, the performance improvement is very obvious, and it will not cause any browser jam in the basic normal project scenarios.
Of course, if web worker optimization is used again, users can quickly see the required data on the smooth access experience.

have little talent and less learning. If there are any omissions, please correct them. If there are any suggestions, please put forward. Welcome to discuss with you.

Recommended Today

JS function

1. Ordinary function Grammar: Function function name (){ Statement block } 2. Functions with parameters Grammar: Function function name (parameter list){ Statement block } 3. Function with return value Grammar: Function function name (parameter list){ Statement block; Return value; } Allow a variable to accept the return value after calling the function Var variable name […]