Enjoy the charm of NoSQL – Hello, redis!

Time:2019-11-14

What is NoSQL? Why NoSQL?

Since the era of Web1.0, we have been using relational databases. For example, MySQL and SQL Server Oracle are popular relational databases. But with the advent of Web2.0, massive data and highly concurrent database operations are generated, which greatly increases the load capacity of database servers and their related servers. For example If the double "11.11" period, its database operation can reach tens of millions per second, and the transactions processed by Alipay will be calculated in billions. In order to solve the above problems, non relational databases, namely NoSQL, came into being. The common nosqls on the market include Tokyo cabinet / pyramid, redis, Voldemort, Oracle BDB, Cassandra, HBase, riak, etc.
The following is the performance comparison of common NoSQL products:
! [picture description] [1]

Redis introduction

Redis is a remote memory database, which not only has strong performance, but also has replication features and unique data model to solve problems. Redis provides five different types of data structures, and all kinds of problems can be naturally mapped to these data structures: redis's data structure is dedicated to helping users solve problems, rather than requiring users to distort problems to adapt to the data base like other databases. In addition, through the features of replication, persistence and client side sharding, users can easily expand redis to a system that can contain hundreds of GB of data and process millions of requests per second.

Redis installation

I. redis download address
Redis doesn’t officially provide redis’s windows installation package, but there are related download addresses on GitHub, as follows:
https://github.com/ServiceSta…
You can also download it on my baidu online disk at http://pan.baidu.com/s/1gf2nuin. The version I downloaded is redis-64.3.0.503.zip (64 bit win system, redis
Version 3.0).

II. Redis installation and configuration

Find the downloaded package redis-64.3.0.503.zip, unzip it, rename it to redis-3.0, and put it in a folder (such as D: program files).

  1. Running redis server

Double click the redis-server.exe file (redis server side) in the D: program filesredis-3.0 directory, and the redis server will run as a window (but the window cannot be closed, otherwise, the redis service is not available). This mode of running redis server does not load the specified configuration file. If the following interface appears

  1. Running redis client

Double click the redis-cli.exe file (redis client) in the D: program filesredis-3.0 directory. If 127.0.0.1:6379 > is displayed, the client runs successfully.
Enter command: Keys*
View all keys
If “noauth authentication required.” is prompted, the redis server has set the password. Please enter the correct password before other operations.
Enter command: auth password
If the prompt is OK, the password is correct.
Tips: you can send redis-server.exe under the directory D: program filesredis-3.0 to the desktop shortcut for quick opening of redis client in the future.

  1. Install redis service to local service

Because the above method of starting redis server is a little complicated, and the redis service window cannot be closed. So here’s how to install redis service into the local service of windows system.
Copy the redis.windows.conf file in the D: program filesredis-3.0 directory, rename it to redis.conf, as the redis configuration file.
Open the command line of win system, and enter the following commands in turn:

D: (enter, switch to disk d)

CD program filesredis-3.0 (enter, switch to D: program filesredis-3.0 directory)

Redis server — service install redis.conf — loglevel verbose

After the operation, you can go to the local service management office of win system to view and operate redis service. (right click the computer icon – > Manage – > services and applications – > services)

  1. Set redis password

Edit the redis.conf configuration file in the D: program filesredis-3.0 directory, and find the following code:

requirepass foobared

Copy one line, remove the leading comment, and change foobared to the password you want to set, such as:
requirepass test
Save to exit.
Restart the redis service and the configuration file will take effect.

  1. How to uninstall redis local service

Open the win system command line and enter the following commands in turn:
d:
cd Program Filesredis-3.0\
redis-server –service-uninstall
You can uninstall the redis local service.

  1. Install redis Desktop Manager

Redis desktop manager is a visual redis database management tool, which is very simple to use. It is not introduced here.
Download address: http://pan.baidu.com/s/1i44axal
Here is the redis installation package and the redis client tool installation package download link:
Redis installation package download: link: https://pan.baidu.com/s/1o779tbo password: n8mw
Redis client tool installation package download: link: https://pan.baidu.com/s/1hsd2mx2 password: 40nd

Knowledge of jdies

Next, I will introduce the related operations of jedis in the form of a piece of code:

public class JedisDemo1 {
    /**
     *Jedis instance
     */
    @Test
    public void demo1()
    {
        //1. Set IP address and port
        Jedis jedis = new Jedis("127.0.0.1", 6379);
        //2. Save data
        jedis.set("name", "james");
        //3. Access to data
        String value = jedis.get("name");
        System.out.println(value);
        //4. Release resources
        jedis.close();
    }
    
    /**
     *Using connection pooling to manipulate jedis
     */
    @Test
    public void demo2()
    {
        //Configuration object of active link pool
        JedisPoolConfig config = new JedisPoolConfig();
        //Set the maximum number of connections
        config.setMaxTotal(30);
        //Set the maximum number of idle connections
        config.setMaxIdle(10);
        
        
        //Get connection pool
        JedisPool jedisPool = new JedisPool(config, "127.0.0.1", 6379);
        
        //Get core object jedis
        Jedis jedis = new Jedis();            
        try {
            //Get connection through connection pool
            jedis = jedisPool.getResource();
            //Set data
            Jedis. Set ("name", "Zhang San");
            jedis.set("age", "25");
            String name = jedis.get("name");
            System.out.println(name);
        } catch (Exception e) {
            e.printStackTrace();
        }finally{
            //Release resources
            if(jedis!=null){
                jedis.close();
            }
            if(jedisPool!=null){
                jedis.close();
            }
        }
    }
}

Download the source code of the project: link: https://pan.baidu.com/s/1c2nlskw password: 1iym

Redis 5 data structure

  1. string
    When using string, redisIn most casesIt doesn’t understand or parse its meaning. No matter using JSON, XML or plain text, it looks the same to redis. It’s just a string. It can only be used for general string operations such as strlen and append, and can’t be further operated on its content. The basic operation commands are set, get, strlen, getrange and append

    SET key value
    GET key
    STRLEN key
    GETRANGE key start end
    APPEND key value

  2. Hash
    When using hash, in my opinion, value itself is a set of key value pairs, but redis calls the key here as field (but why not call the hkeys command hfields command haha), that is, value is a set of field value pairs. The basic operation commands are hset, hget

    Hmset, hmget, hgetall, hkeys, and HDEL:
    HSET key field value
    HGET key field
    HMSET key field value [field value …]
    HMGET key field [field …]
    HGETALL key
    HKEYS key
    HDEL key field [field …]

  3. List

    When you use list, value is a string array. When you operate this group of strings, you can use pop and push operations as you do with stacks, but you can operate on both ends of the stacks; you can also use an index parameter as you do with arrays. The operation commands of list are slightly miscellaneous, mainly divided into two categories: those starting with L and those starting with R. l represents left or list, and performs some operations from the left end of the list, or some operations unrelated to the end. R represents right, and performs some operations from the right end of the list.
  4. Set

Set is used to store a set of values that are not repeated. It can also perform some set operations, just like the mathematical set, which is out of order. The basic operations are Sadd and

sismember:
SADD key member [member ...]
SISMEMBER key member

  1. Sorted Set

Sorted set is similar to set, but each element in sorted set has a score, which can be used for sorting and ranking. Basic operations include

zadd、zcount、zrank:
ZADD key score member [score member ...]
ZCOUNT key min max
ZRANK key member

Redis persistence

Redis provides different persistence options:

  • RDB persistence takes a point in time snapshot of your data set based on a specific time interval.
  • Aof persistence logs every time the server receives writes. When the server restarts, these writes will be executed again to reconstruct the original data set.
  • The command can only be added to the log in the same format as the redis protocol itself. When the log is too large, redis will overwrite the log.
  • If you want your data to exist only when the server is running, you can turn off the persistence function completely according to your requirements.
  • Aof and RDB can be used together in the same instance. Note that in this case, redis uses AOF files to reconstruct the original dataset when it restarts, because the dataset is more complete.

The most important thing is to understand the differences and trade-offs between RDB persistence and AOF persistence. Start with RDB:

RDB

Advantages of RDB:

RDB is a compressed single file that represents your redis data at a certain time. RDB files are ideal for backup. If you archive the RDB files every hour in the last 24 hours, and store one RDB snapshot every day in the last 30 days. In this way, you can easily recover various versions of datasets in case of data disaster.

  • RDB is suitable for disaster recovery. As a single compact file, it can be transmitted to the remote data center or to Amazon S3 (preferably encrypted).
  • RDB maximizes the performance of redis. Because for the redis parent process, if you want to perform persistence, the only thing you need to do is to create a child process, and the child process will do everything. The parent process does not need to perform disk I / O or other similar operations.
  • For large datasets, RDB makes restart faster than AOF.

Disadvantages of RDB:

  • If you want to minimize the possibility of data loss when redis stops working (such as power failure), RDB is not appropriate. You can set different savepoints where RDB will be generated (for example, after at least 5 minutes and 100 writes to the dataset). You can also set up multiple savepoints). However, you are usually set to generate an RDB snapshot every 5 minutes or more. In this case, if redis fails to shut down for any reason, you will lose the last few minutes of data.
  • RDB needs to write disk through subprocess through fork(). If the data set is large, fork () is very time-consuming, which may cause redis to stop providing services to clients in a few milliseconds or even one second, and the CPU performance is not good. Aof also needs fork (), but you can adjust the frequency of re logging without any sacrifice.

AOF

Advantages of AOF:

  • Aof is more reliable: you can have different file synchronization policies: no file synchronization, synchronization every second, synchronization every time you request. The default is synchronization per second, and the write performance is also good (file synchronization uses a background thread, when no file synchronization occurs, the main thread will perform write operations), and you will only lose write data within 1 second.
  • Aof logs can only be appended, so there is no search operation, and there will be no loss when power is off. Even if the last command is only half recorded for some reason (full disk or other reasons), the redis check AOF tool can easily repair it.

When the AOF file is too large, redis can automatically rewrite AOF in the background. Rewriting is completely secure. Because when redis continues to append content to the old file, the resulting new file contains the minimum command set needed to generate the current dataset. Once the new file is ready, redis will switch to the second file and append content to the new file.

  • Aof uses a format that is easy to understand and analyze, logging all operations one by one. You can easily import an AOF file. Even if you use the flush command to flush out all commands by mistake, if you haven’t rewritten the log during this period, you can still recover the data, just stop the server, remove the last command, and then restart the server.

Disadvantages of AOF

  • Compared with RDB files in the same data set, AOF files pass a lot.

Aof is slower than RDB due to the precise synchronization strategy. In general, the performance of file synchronization per second is still good, while AOF without file synchronization is as fast as RDB, even under high load. However, for a large number of potential problems, RDB can provide more guarantees, even for large-scale writing.

  • Previously, we detected very few bugs for some special commands (such as a command involving blocking, brpoplpush), which resulted in the AOF file generated not generating the same data set when reloading. This bug is uncommon. When we test, we automatically create arbitrary and complex datasets and reload them to check whether everything is OK. But this kind of problem hardly exists in RDB persistence. More clearly: redis’s AOF works by updating an existing state, similar to MySQL and mongodb, while RDB generates snapshots again and again, which is more robust in theory. However, (1) note that each time redis rewrites AOF, it starts from the actual data in the data set, which makes it stronger than simply appending (or reading old AOF instead of data in memory) to AOF. (2) up to now, we have not found a report that AOF data loss is detected in real use from users.

The above is a basic introduction to redis and some basic instructions. I believe you will love redis after you use it.