[mongodb] Mongo common field types

Time:2020-9-8

1. JSON and bson

JSON

JSON is a simple data representation, which is easy to understand, parse and remember. But on the other hand, because there are only null, Boolean, number, string, array and object data types, JSON has some limitations. For example, JSON does not have a date type, and JSON has only one numeric type, so it cannot distinguish between floating-point numbers and integers, let alone 32-bit numbers and 64 bit numbers. Furthermore, JSON cannot represent other generic types, such as regular expressions or functions.

BSON

Bson (binary serialized document format) is a kind of binary form storage format of JSON, referred to as binary JSON. Like JSON, it supports embedded document objects and array objects, but bson has some data types that JSON does not have, such as date and bindata types. It supports the following data types. Each data type corresponds to a number. In mongodb, you can use the $type operator to view the bson type of the corresponding document

Mongodb does not need to declare the data type, and can automatically match

Each bson type has an integer and string identifier, as shown in the following table:

Type Number Alias Notes
Double 1 “double”
String 2 “string”
Object 3 “object”
Array 4 “array”
Binary data 5 “binData”
Undefined 6 “undefined” Deprecated.
ObjectId 7 “objectId”
Boolean 8 “bool”
Date 9 “date”
Null 10 “null”
Regular Expression 11 “regex”
DBPointer 12 “dbPointer” Deprecated.
JavaScript 13 “javascript”
Symbol 14 “symbol” Deprecated.
JavaScript (with scope) 15 “javascriptWithScope” Deprecated in MongoDB 4.4.
32-bit integer 16 “int”
Timestamp 17 “timestamp”
64-bit integer 18 “long”
Decimal128 19 “decimal” New in version 3.4.
Min key -1 “minKey”
Max key 127 “maxKey”

2. Data type

2.1. Double type

By default, Mongo shell clients treat numbers as floating-point numbers.

2.2、 64-bit integer(long)

Bson has two types of integer data: 32-bit signed integer data (int) and 64 bit signed integer data (long)

The conversion function is numberlong(),

2.3、 32-bit integer (int)

32 bit integer (int) and 64 bit integer (long) are almost the same, but the conversion function changes from numberlong() toNumberInt()The accepted parameters are also treated as string type.

2.4、decimal

The data type decimal was introduced in Mongo 3.4. The new decimal value type is mainly used to record and process monetary data, such as financial data and tax rate data. Sometimes, some scientific calculations also use the decimal type.

Because the Mongo shell treats numbers as double by default, it also needs to be explicitConversion function numberdecimal (), which accepts a string value.

mongos> db.testnum01.insert({_id:231,calc:NumberDecimal("1000.55")})

explain:

int/long/decimal,The parameter accepts a type of stringIf it is a number (the default type is double), it can be used, but there is a risk of precision loss. The number will be changed to 15 digits (decimal point is not included)

2.5 digital type addition test

All of the above 4 types are number type, and the addition test of decimal and type numbers is carried out, if it is as follows:

Decimal and decimal / int / long are added, and the decimal places remain unchanged;

When decimal and double are added, the decimal places become 14 bits.

mongos> db.testnum01.find({_id:231})
{ "_id" : 231, "calc" : NumberDecimal("1004.55") }
mongos> db.testnum01.updateOne({_id:231},{$inc:{calc:2}})
{ "acknowledged" : true, "matchedCount" : 1, "modifiedCount" : 1 }
mongos> db.testnum01.find({_id:231})
{ "_id" : 231, "calc" : NumberDecimal("1006.55000000000000") }

2.6 object ID

Similar to a unique primary key, it contains 12 bytes: a total of 24 hexadecimal numbers, that is, 12 bytes.

{ "_id" : ObjectId("5f2a22f7aa56fc2fc978b159"), "calc" : 123456789012345680 }
5f2a22f7 aa56fc 2fc9 78b159
#"5f2a222f7" refers to the UNIX timestamp, the time when the data was generated
#"Aa56fc" refers to the machine code of a machine, and the machine number when the data is stored
#"2fc9" refers to the process ID, which is very useful when multiple processes store data
#"78b159" refers to a random number. It should be noted here that the number of random numbers may be repeated and not unique
#The above four identifiers are pieced together to form the unique objectid in the world
#Any language that supports mongodb will have one or more methods to transform the objectid
#The above four kinds of information can be obtained

#Note: this type cannot be serialized by JSON
This is the unique key of the primary key of a similar relational DB table generated by mongodb. It is specifically composed of 24 bits:
0-8 bytes are UNIX time stamps,
9-14 byte machine code, which indicates that the mongodb instance is located on different machines;
15-18 byte process ID, indicating different mongodb processes of the same machine.
19-24 bytes are random numbers

Since the created timestamp is saved in objectid, you do not need to save the timestamp field for your document,
You can get the creation time stamp of the document through “gettimestamp()” and return the timestamp

--Return timestamp
mongos> ObjectId("5f2a22f7aa56fc2fc978b159").getTimestamp()
ISODate("2020-08-05T03:09:43Z")
--Return string
mongos> ObjectId("5f2a22f7aa56fc2fc978b159").str
5f2a22f7aa56fc2fc978b159

2.7 string

UTF-8 string, remember it must be UTF-8 string

2.8 arrays

Array or list, where multiple values are stored in a single key (list)

"Hobby": ["badminton", "football", "Basketball"]

2.9 object dictionary

"course" : {"name" : "MongoDB","price" : 1000}

2.10 Null

Null data type, a special concept, none null

2.11 timestamp timestamp

"date" : 1528183743111

2.12 data

Stores the current date or time format

 "date" : ISODate("2019-01-05T15:28:33.705+08:00")

3. The problem of precision loss in using large integers in Mongo

Mongo shell uses large integer facets, but the default integer facet type is double precision floating-point numbers, resulting in loss of precision

  • Problem Description:

For example, if the length of a large number is greater than 16, the length of Mongo is greater than or equal to:

mongos> db.testnum01.insert([{_id:100,calc:12345678901111111111},{_id:101,calc:NumberLong("1234567890123456789")}])

What’s actually inserted is

> db.testnum01.find()
{ "_id" : 100, "calc" : 12345678901111112000 }
{ "_id" : 206, "calc" : 12345678902222221000 }
{ "_id" : 207, "calc" : 12345678903333333000 }
{ "_id" : 101, "calc" : NumberLong("1234567890123456789") }
  • analysis:

Since the Mongo shell is actually a JS engine, and in JavaScript, there is no int or long in the base type, all integer facets are actually represented as double precision floating-point numbers (IEEE754 format). In 64 bit double precision floating-point numbers, it is actually composed of 1 bit symbol bit, 11 bit order code bit and 52 bit tail digit.

The 11 bit co-1023 order code makes the double precision floating-point number provide the range of – 1.7e308 to + 1.7e308, and the tail of 52bit can represent 15-16 bits (some 16 bit integers have exceeded the range of 52bit).

Therefore, when we use integers directly in Mongo shell, they are in fact represented by double. When the integer number is more than 16 digits, some integers may not be accurately represented, so we can only use an integer close to the representation to replace it. For example, in the above example, only 16 digits can be effectively represented by the number stored in 20 bits, and the accuracy of the other 4 bits is lost.

  • resolvent:

Use the numberlong() function to construct the type of a long integer. Remember to add double quotation marks to the parameters passed in. Otherwise, the integer will be regarded as a double and the precision may be lost.

be careful: long type: 64bit, 8-byte signed type, maximum storage 2 ^ 63-1 = 9223372036854775807

More than 64 bits can be stored as strings: > db.testnum01 .insert({_ id:222,calc:”12345678901234567890″})

mongos> db.testnum01.insert({_id:200,calc:NumberLong(1234567890123456789)})
mongos> db.testnum01.insert({_id:201,calc:NumberLong("1234567890123456789")})
mongos> db.testnum01.find()
{ "_ ID ": 200," Calc ": numberlong (" 1234567890123456768 ")} // the accuracy of the last two bits is incorrect
{ "_id" : 201, "calc" : NumberLong("1234567890123456789") }

mongos>  db.testnum01 .insert({_ id:202, calc:NumberLong ("9223372036854775808")}) - unable to write
2020-08-05T13:39:48.422+0800 E QUERY    [js] Error: could not convert string to long long :
@(shell):1:35
mongos>  db.testnum01 .insert({_ id:202, calc:NumberLong ("9223372036854775807")}) - can be written to
WriteResult({ "nInserted" : 1 })
mongos>

Note that in addition to Mongo shell (JavaScript language environment), there will be similar problems in other programming languages that do not support long integers and use floating-point numbers instead of representation by default. You must pay attention to this problem when operating.

4. Determine the type / length of a field

//If the field type is 2 (string), it means there is such a field, or use $exists: true, and the length is greater than 100
mongos> db.testnum01.find({calc: {$type:2,$regex: /^.{100,}$/ }});
mongos> db.testnum01.find({calc: {$exists: true, $regex: /^.{10,}$/ }});
//The query field type is long, double
mongos> db.testnum01.find({calc:{$type:"double"}})    --db.testnum01.find({calc:{$type:1}})
mongos> db.testnum01.find({calc:{$type:"long"}})    --db.testnum01.find({calc:{$type:18}})

Recommended Today

Blog based on beego, go blog

Go Blog A beego based development, can quickly create personal blog, CMS system Include functions see Official website of go bloggo-blog.cn Demo siteleechan.online Update log time function January 23, 2020 New top post function February 2, 2020 New custom navigation function February 4, 2020 New site announcement function February 6, 2020 New link module February […]