The first time I heard about mongodb (1), I will show you the past and present life of mongodb from scratch

Time:2021-1-16

The first time I heard about mongodb (1), I will show you the past and present life of mongodb from scratch

preface:

Recently, when I was looking at mongodb related knowledge, when I first heard about mongodb, I was confused. What kind of database is this.

The first time I heard about mongodb (1), I will show you the past and present life of mongodb from scratch

But after studying, I found that this database is very good….

Now let’s talk about this database to see what it has.

The main line of this paper is as follows

① Introduction to mongodb

② What functions does mongodb support?

③ The storage engine of mongodb

④ Mongodb knowledge extension

Introduction to mongodb:

Mongndb is aDistributed file storage database(or document database);

Mongodb is a platform betweenRelational database and non relational databaseThe products between right and wrong relational databases have the richest functions, most like relational databases and closest to relational databases.

The first time I heard about mongodb (1), I will show you the past and present life of mongodb from scratch

Mongodb data logic structure:

First of all, let’s talk about the data logic level of mongodbDatabase, collection, document

The logical hierarchy of mongodb data is as follows:

The first time I heard about mongodb (1), I will show you the past and present life of mongodb from scratch

Comparison of data logical structure between mongodb and relational database

The first time I heard about mongodb (1), I will show you the past and present life of mongodb from scratch

From the figure above, we can know the corresponding relationship between mongodb and relational database in terms of data logical structure. Then we can map the relationship all at once

Database — database, table table — collection set, row record — Document

Let’s talk about the document in the picture above{ “name” : “lyl”, “age” : 25}What exactly is data that looks like JSON format?

Data storage structure of documents in mongodb:

The file data storage structure of mongodb isBSONBson isBinary JSONAbbreviation is a binary encoding serialization of documents similar to JSON.

The characteristics of bson are as follows

  • Lightweight
  • Ergodicity
  • High efficiency

Let’s take a look at the bson format document data of mongodb?

The first time I heard about mongodb (1), I will show you the past and present life of mongodb from scratch

Through the figure above, we have a basic understanding of mongodb as a whole. Next, let’s talk about its functions? See what it supportsBreak offIt’s what keeps it going.

Functions supported by mongodb:

Let’s post a large picture from the Internet: describe the main features of mongodb;

The first time I heard about mongodb (1), I will show you the past and present life of mongodb from scratch

In addition to the above features, mongodb also supports many powerful functions:

  • Bson document model:

    • Dynamic data model
    • Use efficient binary data storage, including large objects (such as video, etc.).
  • Dynamic query
  • Full text search
  • Aggregation operation:

    • Polymerization pipeline
    • Map – Reduce
    • Single purpose polymerization
  • Horizontal expansion (high expansion)

    • Master slave replication (building a master slave replication cluster)
    • Data fragmentation (building data fragmentation cluster)
    • Data fragmentation + master-slave replication = more powerful cluster system
  • High availability, support failure recovery
  • Support large file storage gridfs
  • Enterprise security
  • Powerful index support:

    • Single field index
    • Review index
    • Multi key index
    • Text index
    • Wildcard index
    • Geolocation index
    • Hash indices
    • Index properties:

      • TTL index
      • unique index
      • Partial index
      • Sparse index
      • Index intersection
  • Version 4.0 supports multi document transactions, and version 4.2 supports distributed transactions
  • Support multiple storage engines

    • MMAP storage engine
    • Mmapv1 storage engine
    • Wiredtiger storage engine
    • In memory storage engine

Note: the above is only the main functions supported by mongodb. If you want to know more about each function, you can consult the information by yourself.

Now let’s talk about the very important storage engine in mongodb, because the storage engine is the main component of mongodb responsible for managing data, and likeTransaction, concurrent lock, operation of storing data, etcIt’s all supported by the storage engine.

Mongodb’s storage engine:

Mongodb supports a variety of storage engines. Let’s talk about these storage engines of mongodb;

Native storage engine MMAP:

The full name of MMAP is memory mapped storage engineMemory mapped storage engine, inBefore version 3.0Used.

MMAP can map part or all of the contents of a disk file directly to the memory, so that the information location in the file will have the corresponding address space in the memory. At this time, the file can be read and written directly with a pointer instead of the read / write function. But this does not mean that the file is mapped to the physical memory. Only when this data is accessed, the operating system will use the page function Switch mode to physical memory.

When writing to mongodb, first put the data in the memory, and then refresh and save the data to the disk through sequential IO, which will greatly improve the performance of the write operation; and mongodb will hand over the memory management to the virtual memory manager of the operating system, which also greatly simplifies the work of mongodb.

Native upgrade mmapv1 storage engine:

stayMongodb version 3.0, launched mmapv1 storage engine, which is a new storage engine based on the original storage engine.

There is a very important log file in mmapv1 storage engineJournal pre write transaction logUse this log file to ensure data persistence and help database recovery when the database is hard closed.

Plug in wiredtiger storage engine:

Wiredtiger is inMongodb version 3.0Introduced, and inMongodb version 3.2It has become the default storage engine of mongodb.

Compared with mmapv1, wiredtiger storage engine is more powerful and has higher performance.

It’s also available in the wiredtiger storage engineJournal pre write transaction logTo ensure data persistence and data recovery in case of database failure.

In memory storage engine:

In memory storage engine stores database data inMemoryIn this system, only a small amount of metadata, diagnosis log and temporary data are stored in the hard disk file to avoid disk I / O operation and the query speed is very fast.

If mongodb uses in memory storage engine, is it very similar to redis in memory database.

Comparison between storage engines:

The following will be compared in various aspectsMMAPV1 、WiredTiger 、In-MemoryThese three kinds of storage engines make the difference between them clearer.

Click to see the larger picture:

The first time I heard about mongodb (1), I will show you the past and present life of mongodb from scratch

Note: when you read this article, please be sure to use your ownVersion of mongodbBecause the functions supported by different versions may be different, confirm whether the version you are using supports the functions you want.

Mongodb knowledge extension:

Why eat memory?

When using mongodb database, it is found that the available memory of the server installed with mongodb is decreasing all the time. Through searching, it is found that mongodb is constantly eating memory, and mongodb is occupying more and more memory.

Why does mongodb use so much memory?

Here are two aspects:

  1. As can be seen from the above description of the storage engine, for example, the write operation of the wiredtiger storage engine will first write to the cache, save the data in the memory, and then drop the data in the memory to disk through the mechanism, but eventually the data in the memory will be retained, only the dropped data will be marked.
  2. When a large number of concurrent requests are made to mongodb database, the memory used by mongodb database will also increase; because when there are more concurrent connections, a large number of threads will be created to process the connections, and these threads will also occupy the memory; in addition, after the connection is established, the memory of these packets will also occupy the memory to process the packets transmitted in the connection; but the memory occupied will be used in the request After falling, slowly release the memory.

How to control the use of memory:

Let’s talk about how to control mongodb’s use of memory in two aspects. I hope that the alarm of server memory exhaustion will not be triggered again.

  1. Reasonable allocationWiredTiger cacheSizeGBParameters:

    • If only mongodb is deployed on one machine and mongodb can use all available memory, the default configuration is OK.
    • If mongodb is deployed and other process services are running on a machine, it needs to be configured according to the memory quota allocated to mongodbcacheSizeGBIt can also be configured according to about 60% of the quota.
  2. Control the number of concurrent requests to mongodb and make reasonable TCP connections;

Mongodb application scenarios:

According to the features and supported functions of mongodb, let’s talk about its application scenarios

  • Application server log storage.

    • Mongodb’s high performance is enough to support more than 2-3 times TPS / QPS of relational database;
    • Disk data compression storage can reduce the number of disk IO and improve the performance of data reading;
    • Powerful index support
    • Dynamic data model, not limited by table structure;
  • Geographic information storage, through the geographic index, you can easily and quickly find out the specific location information.
  • As a cache database, it uses its in memory storage engine;
  • Website real-time data processing; it is very suitable for real-time insertion, update and query, and has the replication and high scalability of website real-time data storage.

——end

That’s the end of this article. This article only talks about the basic knowledge of mongodb with you. There is no specific description in the details. If you are interested, you can go to consult the information by yourself.

Finally, I’d like to chat with you again. I’m going to write a series of articles on mongodb recently. The titles of the series of articles have been thought out, as follows:

① Learn about mongodb from scratch

② First knowledge of mongodb (2), database installation and visualization tool installation

③ Know mongodb (3), common SQL statements and index knowledge of mongodb

④ Using java to connect and operate mongodb database

⑤ In love with mongodb (5), this article takes you to understand the actual operation and use of mongodb

❤ Like + comment + forward

If this article is helpful to you, please wave your little hand and praise it. Your support is the driving force of my continuous creation. Thank you!

You can search for the official account of Muzi thunder by VX, and insist on the high quality original Java technology articles.

And if you want to see the series of follow-up articles, please comment more, your support is the driving force of my continuous creation!

The first time I heard about mongodb (1), I will show you the past and present life of mongodb from scratch

reference material:

①、Mongodb Chinese manual | official document Chinese version

②、Introduction to mongodb

③、Mongodb storage engine (middle) — wiredtiger

Recommended Today

DK7 switch’s support for string

Before JDK7, switch can only support byte, short, char, int or their corresponding encapsulation classes and enum types. After JDK7, switch supports string type. In the switch statement, the value of the expression cannot be null, otherwise NullPointerException will be thrown at runtime. Null cannot be used in the case clause, otherwise compilation errors will […]