Go JJ

Time:2020-6-30

brief introduction

In the previous two articles, we introduced libraries for quick reading of JSON valuesgjsonAnd a library for quickly setting JSON valuessjson。 Today, we introduce a method based ongjsonandsjsonA very useful command line tool forjj。 It is a command-line program written in go to quickly read and set JSON values.

Quick use

It can be used directly on the Macbrew install tidwall/jj/jjInstallation. Other systems can download the compiled executable program, the download address is https://github.com/tidwall/jj/releases 。

I choose to usego getInstallation:

$ go get github.com/tidwall/jj/cmd/jj

After the execution of the above command, compile the generatedjjThe program will be placed in the$GOPATH/binIn the catalog, I used to put$GOPATH/binJoin system executable directory$PATHTherefore, it can be used directly.

Simple read and setup (my environment is win10 + git bash)

$ echo '{"name":{"first":"li","last":"dj"}}' | jj name.last
dj

$ echo '{"name":{"first":"li","last":"dj"}}' | jj -v dajun name.last
{"name":{"first":"li","last":"dajun"}}

adoptKey pathTo specify the read / set location, the first command above reads the fieldname.last, return todj

-vOption specifies the value of the setting. The second command changes the fieldname.lastSet todajunTo output the JSON string after setting. Key path in the first two articles have a detailed introduction, unfamiliar can go back to have a look.

Read and set

Actually read and set the syntax and form with our previous introductiongjsonandsjsonIt’s basically the same, but it’s done on the command line.

Read the nonexistent field and returnnull

$ echo '{"name":{"first":"li","last":"dj"}}' | jj name.middle
null

Read a field of an object type and return the JSON representation of the object:

$ echo '{"name":{"first":"li","last":"dj"}}' | jj name
{"first":"li","last":"dj"}

Using the index (starting from 0) to read the elements of the array, illegal index will return null:

$ echo '{"fruits":["apple","orange","banana"]}' | jj fruits.1
orange

$ echo '{"fruits":["apple","orange","banana"]}' | jj fruits.3

Using the index to set the elements of the array, the following command sets the arrayfruitsThe second element of is set topear

$ echo '{"fruits":["apple","orange","banana"]}' | jj -v pear fruits.1
{"fruits":["apple","pear","banana"]}

use-1Or array length as an index, you can add an element after the array. If the index exceeds the length of the array, it will be increased by a certain numbernull

$ echo '{"fruits":["apple","orange","banana"]}' | jj -v strawberry fruits.-1
{"fruits":["apple","orange","banana","strawberry"]}

$ echo '{"fruits":["apple","orange","banana"]}' | jj -v grape fruits.3
{"fruits":["apple","orange","banana","grape"]}

$ echo '{"fruits":["apple","orange","banana"]}' | jj -v watermelon fruits.5
{"fruits":["apple","orange","banana",null,null,"watermelon"]}

Use options-DDelete the element on the specified key path. If the corresponding element does not exist, it has no effect

$ echo '{"name":"dj","age":18}' | jj -D age
{"name":"dj"}

$ echo '{"fruits":["apple","orange","banana"]}' | jj -D fruits.2
{"fruits":["apple","orange"]}

$ echo '{"fruits":["apple","orange","banana"]}' | jj -D fruits.5
{"fruits":["apple","orange","banana"]}

The first command removes the fieldage; the second command removes the arrayfruitsThe second element of; the third command removes the arrayfruitsBecause the array length is only 3, it has no effect.

file

jjSupports reading JSON strings from files and writing results to files. Use options-iSpecify input file, options-oSpecifies the output file. The following will be from the filefruits.txtRead the JSON string, take the second element of the array, and writeout.txtChinese:

$ jj -i fruits.txt -o out.txt fruits.1

fruits.txtThe contents of the document are as follows:

{"fruits":["apple","orange","banana"]}

Execute the command, and the output file contains:

orange

format

jjSupport to format the output JSON string. option-uRemove all space saving characters. option-pBeautify the format and make it easy to read.

$ echo '{"name":{"first": "li", "last":"dj"}, "age":18}' | jj -u name
{"first":"li","last":"dj"}

$ echo '{"name":{"first": "li", "last":"dj"}, "age":18}' | jj -p name
{
  "first": "li",
  "last": "dj"
}

performance

Command line tool with another JSONjqcomparison,jjIt is more than 10 times of its performance. becausejjThe validity of the JSON string is not verified, and it only cares about the value specified by the key path and stops once the value processing is complete. Here is a performance comparison: https://github.com/tidwall/jj#performance

purpose

jjA very convenient use is log processing. At present, many log libraries support JSON format, such as the one we introduced earlierlogrus。 We can use itjjFind the corresponding information in these logs. Let’s use it firstlogrusGenerate 20 logs of players’ login and offline:

package main

import "github.com/sirupsen/logrus"

func main() {
  logrus.SetFormatter(&logrus.JSONFormatter{})

  for i := 1; i <= 10; i++ {
    logrus.WithFields(logrus.Fields{
      "userid": i,
    }).Info("login")
    logrus.WithFields(logrus.Fields{
      "userid": i,
    }).Info("logoff")
  }
}

Build logs are stored inlog.txtIn the document:

{"level":"info","msg":"login","time":"2020-03-26T23:36:04+08:00","userid":1}
{"level":"info","msg":"logoff","time":"2020-03-26T23:36:04+08:00","userid":1}
{"level":"info","msg":"login","time":"2020-03-26T23:36:04+08:00","userid":2}
{"level":"info","msg":"logoff","time":"2020-03-26T23:36:04+08:00","userid":2}
{"level":"info","msg":"login","time":"2020-03-26T23:36:04+08:00","userid":3}
{"level":"info","msg":"logoff","time":"2020-03-26T23:36:04+08:00","userid":3}
{"level":"info","msg":"login","time":"2020-03-26T23:36:04+08:00","userid":4}
{"level":"info","msg":"logoff","time":"2020-03-26T23:36:04+08:00","userid":4}
{"level":"info","msg":"login","time":"2020-03-26T23:36:04+08:00","userid":5}
{"level":"info","msg":"logoff","time":"2020-03-26T23:36:04+08:00","userid":5}
{"level":"info","msg":"login","time":"2020-03-26T23:36:04+08:00","userid":6}
{"level":"info","msg":"logoff","time":"2020-03-26T23:36:04+08:00","userid":6}
{"level":"info","msg":"login","time":"2020-03-26T23:36:04+08:00","userid":7}
{"level":"info","msg":"logoff","time":"2020-03-26T23:36:04+08:00","userid":7}
{"level":"info","msg":"login","time":"2020-03-26T23:36:04+08:00","userid":8}
{"level":"info","msg":"logoff","time":"2020-03-26T23:36:04+08:00","userid":8}
{"level":"info","msg":"login","time":"2020-03-26T23:36:04+08:00","userid":9}
{"level":"info","msg":"logoff","time":"2020-03-26T23:36:04+08:00","userid":9}
{"level":"info","msg":"login","time":"2020-03-26T23:36:04+08:00","userid":10}
{"level":"info","msg":"logoff","time":"2020-03-26T23:36:04+08:00","userid":10}

Since each row is a separate JSON string, we can use thejjSupported JSON line features, using..The path identifies the rows...bringjjThink of these rows as elements of an array. We can read these elements.

Get the array length, return 20:

$ jj -i log.txt ..#
20

Read only theuseridInformation:

$ jj -i log.txt ..#.userid
[1,1,2,2,3,3,4,4,5,5,6,6,7,7,8,8,9,9,10,10]

Read only themsgInformation:

$ jj -i log.txt ..#.msg
["login","logoff","login","logoff","login","logoff","login","logoff","login","logoff","login","logoff","login","logoff","login","logoff","login","logoff","login","logoff"]

More complicated, if we want to see all of themuserid=1Log of:

$ jj -i log.txt ..#\(userid=1\)# -p
[  
  {
    "level": "info",
    "msg": "login",
    "time": "2020-03-26T23:36:04+08:00",
    "userid": 1
  },
  {
    "level": "info",
    "msg": "logoff",
    "time": "2020-03-26T23:36:04+08:00",
    "userid": 1
  }
]

Notice two points in the above command,(and)Is a special character in the shell that requires\Paraphrase. We use-pOptions make the results easier to read.

If we only need to find the first eligible log, we can remove the right most log#

$ jj -i log.txt ..#\(userid=1\) -p
{
  "level": "info",
  "msg": "login",
  "time": "2020-03-26T23:36:04+08:00",
  "userid": 1
}

To view all your login information:

$ jj -i log.txt ..#\(msg="login"\)# -p
[
  {
    "level": "info",
    "msg": "login",
    "time": "2020-03-26T23:36:04+08:00",
    "userid": 1
  },
  {
    "level": "info",
    "msg": "login",
    "time": "2020-03-26T23:36:04+08:00",
    "userid": 2
  },
  {
    "level": "info",
    "msg": "login",
    "time": "2020-03-26T23:36:04+08:00",
    "userid": 3
  },
  {
    "level": "info",
    "msg": "login",
    "time": "2020-03-26T23:36:04+08:00",
    "userid": 4
  },
  {
    "level": "info",
    "msg": "login",
    "time": "2020-03-26T23:36:04+08:00",
    "userid": 5
  },
  {
    "level": "info",
    "msg": "login",
    "time": "2020-03-26T23:36:04+08:00",
    "userid": 6
  },
  {
    "level": "info",
    "msg": "login",
    "time": "2020-03-26T23:36:04+08:00",
    "userid": 7
  },
  {
    "level": "info",
    "msg": "login",
    "time": "2020-03-26T23:36:04+08:00",
    "userid": 8
  },
  {
    "level": "info",
    "msg": "login",
    "time": "2020-03-26T23:36:04+08:00",
    "userid": 9
  },
  {
    "level": "info",
    "msg": "login",
    "time": "2020-03-26T23:36:04+08:00",
    "userid": 10
  }
]

summary

jjIs a very used JSON command-line tool, excellent performance. implementjj -hCheck out the other options.

If you find a fun and easy-to-use go language library, you are welcome to submit an issue on the go daily library GitHub

reference resources

  1. jj GitHub:https://github.com/tidwall/jj
  2. Go daily GitHub: https://github.com/darjun/go-daily-lib

I

My blog: https://darjun.github.io

Welcome to my WeChat official account, GoUpUp, learn together and make progress together.

Go JJ