Teach you to quickly parse JSON data without programming

Time:2021-4-7

JSON is a lightweight, language independent data storage format. Most programming languages can parse it, and it is also very friendly to programmers. We often use JSON format in communication / data interaction.

However, when we store the data, the JSON format stores the data in one line, so it will be difficult to read. Therefore, in order to make it easier to read, we can use some methods to format JSON data.

In various programming languages, there will be some corresponding libraries for us to parse JSON data, such as cjson in C and python json.tool And so on.

Is there any tool that can format / parse JSON data without programming on Linux platform?

The answer, of course, is yesjq

JQ is a tool to process JSON data from the command line. It can accept the JSON data in standard input, command pipeline or file, form the data structure we need after a series of filters and expression transformation, and output the results to the standard output. This characteristic of JQ makes it easy to call it in Shell script.

Installation of JQ tool

Some distributions already have JQ built-in, but some don’t. If there is no built-in tool, we need to install it manually.

The installation methods of each platform are as follows:

  • Arch Linux platform:
sudo pacman -S jq
  • Debian, Ubuntu, Linux, mint platform:
sudo apt-get install jq
  • Fedora:
sudo dnf install jq
  • OpenSUSE:
sudo zypper install jq

For the installation of other platforms, you need to check their official installation manual.

Using JQ tool to format JSON data

For example, we now have the following JSON data:

{"firstName":"Liangxu","lastName":"Yan","age":18,"address":{"streetAddress":"21 2nd Street","city":"Guangzhou","province":"Guangdong","postalCode":"510655"},"phoneNumber":[{"type":"home","number":"020 555-1234"},{"type":"company","number":"020 555-4567"}],"gender":{"type":"male"}}

It looks dizzy, doesn’t it? It’s not easy to read, is it?

Let’s save this file as liangxu.json File, and then use JQ tool to format it to make it easier for us to read

cat liangxu.json | jq '.'

Output results:

{
  "firstName": "Liangxu",
  "lastName": "Yan",
  "age": 18,
  "address": {
    "streetAddress": "21 2nd Street",
    "city": "Guangzhou",
    "province": "Guangdong",
    "postalCode": "510655"
},
  "phoneNumber": [
{
  "type": "home",
  "number": "020 555-1234"
},
{
  "type": "company",
  "number": "020 555-4567"
}
],
  "gender": {
  "type": "male"
  }
}

‘.’ is the simplest expression of JQ tool, it does not change the input, but it can output gracefully, easy to read and understand.

In the following cases, we use this data as the parsing object.

Using JQ tools to resolve specific fields

In the above example JSON data, if we want to parse the address field, we can use the JQ tool like this:

jq .address liangxu.json

Output results:

{
  "streetAddress": "21 2nd Street",
  "city": "Guangzhou",
  "province": "Guangdong",
  "postalCode": "510655"
}

Next, we will further resolve the address of the zip code, we cooperate with the pipeline to carry out.

cat liangxu.json | jq .address.postalCode

Output results:

"510655"

Please note that when using the JQ command, the filter is case sensitive, so when you parse a field, you must be exactly the same as the original field, otherwise you cannot parse it.

Using JQ tool to resolve the elements in the array

In JSON data, an array is a set of elements enclosed in square brackets. If we want to parse the elements in the array, we need to use the subscript in the array.

In the example JSON data, the content stored in the phonenumber field is an array. If we want to get all the elements in the array, we just need to add a square bracket. The following command is used:

jq .phoneNumber[] liangxu.json

Output results:

{
  "type": "home",
  "number": "020 555-1234"
}
{
  "type": "company",
  "number": "020 555-4567"
}

If we want to filter out the first element in the array, we can add the subscript [0]:

jq .phoneNumber[0] liangxu.json

Output results:

{
  "type": "home",
  "number": "020 555-1234"
}

Built in functions of JQ tools

JQ tool provides us with many built-in functions. Here are two of themkeysandhas

  • keys

Keys is used to get the key element in JSON and find all the keys in JSON data.

cat liangxu.json | jq 'keys'

Output results:

[
  "address",
  "age",
  "firstName",
  "gender",
  "lastName",
  "phoneNumber"
]
  • has

Has is used to judge whether there is a key in JSON data, and its output result is true or false.

cat liangxu.json | jq 'has("alvin")'

Output results:

false

Summary

The above is a very basic usage of JQ tool. JQ can not only meet the general common needs, but also include powerful functions such as operation, built-in function, condition comparison, variable declaration, self defining function, etc. If you are interested in this, you may as well learn from the official Manual of JQ.


Official account: good Linux

Teach you to quickly parse JSON data without programming

What’s the gain? I hope the old fellow will take three strikes to show this article to more people.