XML online editor made of pure JS (supports modifying local files)

Time:2021-12-30

preface

I haven’t updated my blog for more than a year because “horse riding and slashing 2” was on sale during the epidemic, and then I went to write the game mod.

I used c# to write the game mod for about 7 months. I stayed late every night. Then I learned PR because of introducing the game mod, and then became the up Master of station B.

Later, I had some other ideas and business adjustment of the company. I didn’t bother to write a blog. More than a year passed unconsciously.

There are still gains:

  • For example, when the mod is disconnected, whether it is in the Chinese station or the 3DM mod station, the download volume of the mod is the first, and the second place is far away. If you have a friend who plays the mod of riding and chopping 2, you should guess who I am.
  • For another example, I gained more than 5000 fans at station B. from the beginning, I stuttered, and finally I stuttered. However, because of their own editing, the viewing effect is also good.
  • For example, I deeply realize how troublesome it is to be an up and anchor. In fact, my hip pulling data has been ahead of many up hosts in station B. The up master is not the head up, but the up master for video 0 playback. You can take a look at the latest video of station B. It has turned over dozens of pages and is 0 played. It’s very spectacular.
  • Interesting life experiences have increased

All right, back to business.

Now the basic mod is broken, and the up master is too lazy to continue to do it seriously.

Here we mainly talk about technology related, that is, a pure front-end implementation, an XML online editor for writing mod.

It is an imitationVSCodeStyle editor, which can automatically learn the constraint rules of game mod file generation, and help us realize code prompt and code verification.

More importantly, it can directly modify the files on your computer.

This is the code warehouse for the final product:https://gitee.com/vvjiang/mod-xml-editor

And a finished product display diagram:

Technologies covered in this blog:

  • CodeMirror
  • react-codemirror2
  • xmldom
  • FileReader
  • IndexDB
  • Web Worker
  • File System Access

Let’s start from the beginning.

Requirements for online XML editors

XML files need to be written frequently when doing the mod of Qiqiao 2.

Because the data configuration of qiqi2 is saved in the form of XML, and then after mod is loaded, the XML of mod is used to overwrite the official XML.

Usually when we do mod data, we refer to the official XML to write XML files ourselves.

But this will encounter a problem. XML has no code prompt and code verification, and it is difficult to find a wrong character.

Or sometimes when the game is updated, its XML rules may change.

The official will not issue a notice to tell you these changes, so if you still use the previous elements and attributes, you will be wrong.

The result of writing wrong is that the game crashes directly when loading mod, and it won’t give you any tips. You can only look for bugs slowly.

As a large game, Qijian 2 takes a long time to start each time, resulting in a very long test process to test whether a mod data is configured correctly.

Mom, how many nights, the moment the game collapsed, I collapsed.

So later I wanted to be an XML online editor to solve this problem.

Technical pre research

Visual programming

In fact, I didn’t think of doing this XML editor at first, because it’s difficult to do at first sight. Instead, I wanted to implement it through visual programming by dragging and dropping elements and attributes.

Don’t tell me, I really made a preliminary scheme. As a result, I configured a large XML, dragged it countless times, and my mentality gradually exploded, so I gave up this scheme.

Vscode plug-in

I want to see if there are any vscode plug-ins that can prompt code. There is one that uses XSD for code verification, which seems to be provided by IBM.

But it’s a pity that it has been abandoned and can’t be used. Give up this scheme.

Online editor

Later, the online editor was used to do this because the company wanted to do an online editing of the XML configuration file of the Java project environment in March and April.

Then I tried to make one and learnedCodeMirror

CodeMirrorYou can configure tags to support XML code prompt, but it does not support XML code verification, so you need to do XML code verification yourself.

And because we usually use XSD to verify XML, we also need to convert XSD intoCodeMirrorTags configuration for.

Neither Baidu, Google nor GitHub can find the corresponding scheme, so they can only write their own code to implement it.

In the process, ICodeMirrorxsdhtmllintHave a deep understanding, and finally completed the project.

Because this is the code of the previous company, it will not be released here.

In short, in the process of learningCodeMirrorSuch a thing is usefulCodeMirrorThe idea of doing mod’s online editor.

Initial form: simple online XML Editor

Well, without nonsense, picking up the keyboard is no brain stem.

In the initial form, there was no file tree on the left, only a simple editor and a rule learning pop-up box.

There are three technologies involved:

  • CodeMirror
  • FileReader
  • xmldom

Using codemirror as editor

CodeMirrorThis is a packaged version of react, which is mainly usedreact-codemirror2Anyway, it’s just to match the document with the demo.

The only difficulty is a lot of onlineCodeMirrorMany configuration introductions are copied and reproduced. It’s still wrong. It’s outrageous.

In short, if you want to play, you’d better read the official documents(https://codemirror.net/)And the demo on the document, and then study it yourself. If you copy other people’s configuration, the water is very deep and you can’t grasp it.

Let me post a section of the configuration code of the editor component I encapsulated here. It is absolutely available anyway. Most of the functions of the editor are OK, but it is only applicable to editing XML.

The comments inside are more detailed, including common code folding and code formatting. I’m too lazy to talk about them one by one. You can refer to the official website for yourself.

I won’t post some of the referenced codes. If you are interested, you can go to the code warehouse mentioned above.

import { useEffect } from 'react'
import { Controlled as ControlledCodeMirror } from 'react-codemirror2'
import CodeMirror from 'codemirror'
import 'codemirror/lib/codemirror.css'
import 'codemirror/theme/ayu-dark.css'
import 'codemirror/mode/xml/xml.js'
//Cursor line code highlight
import 'codemirror/addon/selection/active-line'
//Folding code
import 'codemirror/addon/fold/foldgutter.css'
import 'codemirror/addon/fold/foldcode.js'
import 'codemirror/addon/fold/xml-fold.js'
import 'codemirror/addon/fold/foldgutter.js'
import 'codemirror/addon/fold/comment-fold.js'
//Code prompt completion and
import 'codemirror/addon/hint/xml-hint.js'
import 'codemirror/addon/hint/show-hint.css'
import './hint.css'
import 'codemirror/addon/hint/show-hint.js'
//Code verification
import 'codemirror/addon/lint/lint'
import 'codemirror/addon/lint/lint.css'
import CodeMirrorRegisterXmlLint from './xml-lint'
//Automatically type the end label when entering >
import 'codemirror/addon/edit/closetag.js'
//Notes
import 'codemirror/addon/comment/comment.js'

//Theme style used to adjust codemirror
import style from './index.less'

//Register XML code validation
CodeMirrorRegisterXmlLint(CodeMirror)

//Format correlation
CodeMirror.extendMode("xml", {
commentStart: "",
newlineAfterToken: function (type, content, textAfter, state) {
    return (type === "tag" && />$/.test(content) && state.context) ||
    /^ {
    //Each time tags changes, the verification rules will be changed again
    CodeMirrorRegisterXmlLint(CodeMirror, tags, onErrors)
}, [onErrors, tags])

//Start label
function completeAfter(cm, pred) {
    if (!pred || pred()) setTimeout(function () {
    if (!cm.state.completionActive)
        cm.showHint({
        completeSingle: false
        });
    }, 100);
    return CodeMirror.Pass;
}

//End tag
function completeIfAfterLt(cm) {
    return completeAfter(cm, function () {
    var cur = cm.getCursor();
    return cm.getRange(CodeMirror.Pos(cur.line, cur.ch - 1), cur) === "
    Automatically type the end element when
        Togglecomment: true, // enable comments
        //Folding code begin
        lineWrapping: true,
        foldGutter: true,
        gutters: ['CodeMirror-linenumbers', 'CodeMirror-foldgutter', 'CodeMirror-lint-markers'],
        //Folding code end
        extraKeys: {
            //Code prompt
            "' {
            cm.toggleComment()
            },
            //Save function
            "Ctrl-S": (cm) => {
            onSave()
            },
            //Format
            "Shift-Alt-F": (cm) => {
            const totalLines = cm.lineCount();
            cm.autoFormatRange({ line: 0, ch: 0 }, { line: totalLines })
            },
            //Tab is automatically converted to spaces
            "Tab": (cm) => {
            If (cm. Somethingselected()) {// overall indentation after selection
                cm.indentSelection('add')
            } else {
                cm.replaceSelection(Array(cm.getOption("indentUnit") + 1).join(" "), "end", "+input")
            }
            }
        },
        //Code prompt
        hintOptions: { schemaInfo: tags, matchInMiddle: true },
        lint: true
        }}
        editorDidMount={onGetEditor}
        onBeforeChange={onChange}
    />
    
)
}

export default XmlEditor

Learn XML and extract tags rules

When we useCodeMirrorWhen making a simple editor, you need to use tags to make an XML code prompt.

Obviously, different games have different XML rules, including the XML rules will change after the game is updated.

Therefore, we must ensure that there is a mechanism to constantly learn these XML rules, so here I do a pop-up window to learn XML file rules to do this.

Click constraint rule – > add constraint rule at the top left of the editor

A pop-up window will pop up:

adoptFileReaderRead the XML file in the specified folder, and then usexmldomTo parse the text of these XML files in turn and generate document objects.

Then analyze these document objects to get the final tags rules.

This step only needs to understand XML, which is actually quite basic, so I won’t talk about it.

In short, now that we have completed its initial form, each time you use it, you need to copy the content of the XML file you edited to this online editor. After editing, copy the completed text to the original XML file to save and overwrite.

Evolutionary form: Online XML editor with tree file structure and full file verification function

In fact, the use scenario of the above editor is very narrow and can only be used when writing a new XML.

A mod often has dozens, hundreds or even thousands of files, which cannot be pasted into the editor for verification one by one.

Therefore, we need to load all XML files of mod in this editor and perform a code verification.

There are two technologies involved:

  • FileReader
  • Web Worker

Left file tree

The file tree on the left usesAnt DesignAfter the tree component of is completed, we won’t talk about the configuration here.

When you click the open folder button

Same useFileReaderTo read the files in the mod folder.

howeverFileReaderWhat you get is a file array. To generate the tree structure on the left, you need to manually parse the path of each XML file and generate a tree structure accordingly.

Full file verification function

At the moment of opening the folder, we need to perform a code verification on all XML files. If the verification is wrong, we need to mark all relevant files and a series of folders of its parent and ancestor in red in the folder on the left.

On the surface, this function is very simple, but in fact, it has a lot of holes, because the amount of verification calculation is actually not small, especially when there are hundreds of thousands of files in your mod, it is very easy to make your JS blocked and the page unresponsive.

I used it hereWeb WorkerOpen a new thread to handle the verification process and return the results to me after the verification is completed.

In the process, IWeb WorkerWe also know more about the use of.

I always thought it was aNew worker (a JS file)It’s hard to play in this way in combination with the modular development of react.

But actually nowwebpackIn the configurationworker-loader, can be very convenient to useWeb Worker

First, our worker code can be written as follows:

import { lintFileTree } from '@/utils/files'

onmessage = ({ data }) => {
lintFileTree(data.fileTree, data.currentTags).then(content => {
    postMessage(content)
})
}

Then when we use this worker, we can see the following

import { useWebWorkerFromWorker } from 'react-webworker-hook'
import lintFileTreeWorker from '@/utils/webWorker/lintFileTree.webworker'

const worker4LintFileTree = new lintFileTreeWorker()

const [lintedFileTree, startLintFileTree] = useWebWorkerFromWorker(worker4LintFileTree)

Then you have another oneuseEffectRely on thislintedFileTree, if it changes, it will do some operations, so it’s like usinguseStateJust as easy.

Non recursive traversal tree

You can see that many of the things we use above are related to the tree, such as traversing the file tree to verify the code.

Or after we switch a constraint rule, we also need to traverse the whole file tree for re verification.

In the process of traversal, I used recursion to traverse the whole tree. The bad thing is that the memory can not be released during recursion, so later I changed an algorithm to traverse the whole tree in a non recursive way.

Indexdb saves the contents of the file

Because the contents of our mod files are relatively large, the memory occupation may be large, and it is impossible to put these file contents in memory all the time.

So I read the contents of the file and put them in orderIndexDBIn, only the contents of the currently edited file are displayed.

Only when necessary, such as full file verification or file switching, can you start fromIndexDBGet the file content again.

Extreme evolution form: break through the restriction of browser sandbox and realize the addition, deletion and modification of computer local files

Through the previous operations, we have finally completed a basically available online XML editor.

However, it has a fatal disadvantage, which is limited by the browser sandbox environment. After modifying the file, we can’t save it directly to the computer. Instead, we must manually copy the modified code one by one to the corresponding file.

This operation is cumbersome and complex, so that the functions of our editor may only be used to assist in code writing and batch verification.

I thought I could only do this before, but later I accidentally read a post on Zhihu and found that the version of chrome 86 + has a functional API:FileSystemAccess

In addition, unless it is in the local localhost environment, this API can only be called in the HTTPS environment, that is, you can’t call it on an HTTP website, even if you use chrome 86 + or edge86 +.

This API allows us to directly operate the files on the local computer, instead of reading only like FileReader or operating only in the browser sandbox like filesystem.

adoptFileSystemAccessWe can not only read and modify the files in the folder, but also add and delete files.

Therefore, I use this API to completely replace the previous points using FileReader, and right-click on the file tree to add and delete folders and files. (file renaming is not supported here, but in fact, we can simulate renaming by adding after deleting, but I’m too lazy to do it.)

At the same time, press the Save button or the shortcut key Ctrl + s to save the file directly.

Here is a useFileSystemAccessComponent code to open the folder:

import React from 'react'

    //Custom open folder component
    const FileInput = (props) => {
    const { children, onChange } = props
    const handleClick = async () => {
        const dirHandle = await window.showDirectoryPicker()
        dirHandle.requestPermission({ mode : "readwrite" })
        onChange(dirHandle)
    }
    return 
        {children}
    
    }

    export default FileInput

As long as the element wrapped by this component (such as a button) is clicked, it will be called immediatelyshowDirectoryPicker, request to open the folder.

After opening the folder, request the folder write permission through the obtained folder handle, and then transfer the folder handle to the outside to obtain the file tree structure.

The operation here is flawed because when requesting to open a folder, the browser will pop up a box to obtain the permission to read the folder from the user,

After opening, the second box will pop up directly to obtain write permission, that is, the box will pop up twice when opening the folder.

But I can only request all permissions at one time through this method, otherwise it’s not good to request permissions when I want to save.

However, the shortcomings do not hide the shortcomings. Through this API, not only the addition, deletion and modification of files are realized, but also theIndexDBUse of.

Because we can get the corresponding file content through the file handle at any time, it is not necessary to save the file content toIndexDBYes.

More features and details

I have just outlined the core functions of the technology. In fact, the editor has more than n details.

For example, the panel for adjusting the tags rules, such as the buttons on the toolbar, such as the simple encapsulation of DVA. For example, when analyzing XML, if the attribute value is a number, it will not be prompted, but will be ignored directly, because the number is often meaningless and the enumeration value is too large.

There are too many of these, but their applications are relatively basic, so I don’t want to repeat the details, otherwise this blog will become very long and difficult to highlight the core ideas.

Deficiency and summary

The shortcomings here are more due to laziness, such as the folder and file renaming function mentioned earlier, and the custom rules for adjusting tags rules. Modification and deletion are not supported.

Can be achieved, just too lazy to do it.

This thing has been done for several months. It doesn’t mean that I write it every night. I mainly write it when I have inspiration, or write it again when I find out where I can improve it.

Together, it’s like doing this every night for two or three weeks, and then when it’s more and more perfect and usable, it’s more and more lazy to do it.

Because the remaining operations are not very important and can be completed by brain repair, there are not many challenging places.

But overall, the usability of this thing is still very strong.

It can not only be used for the auxiliary writing of XML files for a series of games such as horse riding and slashing 2, great truth repair simulator and civilization 6, but also for those XML configurations without XSD rules and too complex, and even it can learn your customized XML rules.

This blog is over. If there are any omissions, I hope you will give me some advice.