Redis — deployment operation

Time:2022-5-11

1.Redis

1.1 installation

  • Redis is already installed in Ubuntu virtual machine. The following steps can be skipped
  • The subsequent installation process is as follows: redis download link: X , refers to the version numberhttp://download.redis.io/releases/redis-x.x.x.tar.gz
  • Step 1: Download

    wget http://download.redis.io/releases/redis-x.x.x.tar.gz

    

  • Step 2: unzip

    tar xzf redis-x.x.x.tar.gz

  • Step3: move and record it in usr / local

    sudo mv ./redis-x.x.x /usr/local/redis/

  • Step4: enter redis

    cd /usr/local/redis/

  • Step5: generate

    sudo make

    

 

  • Step6: test, the transportation time will be shorter

    sudo make test

    

 

  • STEP7: install the redis command to/usr/local/bin/(record

    sudo make install

  • Step8: after the installation, we enter the directory/usr/local/binView in

    cd /usr/local/bin

    ls -all

    

Redis server redis server

Redis cli redis command line client

Redis benchmark redis performance test tool

Redis check AOF AOF file repair tool

Redis check RDB RDB file retrieval tool

  • Step9: configure ⽂ pieces, move to/etc/(record

Configuration part is recorded as/usr/local/redis/redis.conf

    sudo cp /usr/local/redis/redis.conf /etc/redis/

  • Step9: other supplements

Installing redis on MAC:https://brew.sh/

To install redis using Brew:https://www.cnblogs.com/cloudshadow/p/mac_brew_install_redis.html

 

1.2 configuration

  • Redis configuration information/etc/redis/redis.conflower

Check sudo VI / etc / redis / redis conf

  • Bind IP: if you need remote access, you can annotate this or bind a real IP
    • bind 127.0.0.1
  • End ⼝: 6379 by default
    • port 6379
  • Run as a daemon
    • If running as a daemon, it will not block in the command ⾏ similar to a service
    • If running as a daemon, the current terminal is blocked
    • Set to yes to indicate the daemon, and set to no to indicate the ⾮ daemon
    • It is recommended to set it to yes , daemon yes
  • Data ⽂ pieces
    • dbfilename dump.rdb

 

  • Data storage path
    • dir /var/lib/redis  
  • ⽇ record ⽂ piece
    • logfile “/var/log/redis/redis-server.log”
  • Database, 16 by default
    • database 16
  • Master slave replication, similar to dual computer backup
    • slaveof  host  port

 

 

1.3 server side and client side commands

1.3.1 server side

  • The command on the server side is redis server
  • You can make help view the help file
    • redis-server –help
  • Personal habits
    • PS aux | grep redis view redis server process
    • Sudo kill – 9 PID kill redis server
    • sudo redis-server /etc/redis/redis. Conf specifies the loaded configuration file

1.3.2 client

  • The command of the client is redis cli
  • You can make help view the help file
    • redis-cli –help
  • Connect to redis
    • redis-cli

      

  • Operation test command
    • ping
  • Switch database
  • There are 16 databases without names by default. They are identified by 0-15. The first database is selected by default when connecting to redis
    • select 10

 

1.4 data operation

 

Redis -- deployment operationRedis -- deployment operation

Key command

        1) View all keys: Keys*
        2) View the key with a in the name: keys a*
        3) Judge whether the key exists. If it exists, return 1; if not, return 0: exists key1
        4) View the type of value corresponding to the key: type key
        5) Delete key and corresponding value: del key1 key2
        6) Set the expiration time in seconds: expire key seconds
        7) View the effective time in seconds: TTL key

Key command 

Redis -- deployment operationstring
Redis -- deployment operationRedis -- deployment operation

Hash type:
    
    Hash is used to store objects. The structure of objects is attributes and values
    The value is of type string

    1. Add and modify
    
        Set a single attribute: hset key field value
        Set multiple attributes: hmset key field1 value1 field2 Value2
           
    2. Get
        
        Get all the attributes of the specified key: hkeys key
        Get the values of all attributes: hvals key
        Get the value of ⼀ attributes: hget key field
        Get the values of multiple attributes: hmget key field1 field2 

    3. Delete
        
        Delete the entire hash key and value, and make the ⽤ del command: HDEL key

hash

Redis -- deployment operationRedis -- deployment operation

List type
    
    The element type of the list is string
    Sort by insertion order

    1. Increase
        
        Insert data on the left side: lpush key value1 Value2
        Insert data on the right side: rpush key value1 Value2
        Insert ⼊ new element before or after the specified element:
               Linsert key before or after existing element new element
    
    2. Get
        
        Return list ⾥ elements within the specified range: range key start stop
        Set the value of the specified element: lset key index value
    
    3. Delete
        
        Delete the specified element
            Remove the element whose value is value in the first count of the list
            Count > 0: remove from top to bottom
            Count < 0: remove from tail to head
            Count = 0: remove all

        lrem key count value

    4. Cut and trim

        Trim (intercept) the elements within the [start stop] interval, and delete all the elements outside the interval
            ltrim key start stop

list

Redis -- deployment operationRedis -- deployment operation

Set type
    1) Ordered set
    2) Element is of type string
    3) Elements are unique and do not repeat
    4) Note: there is no modification operation for the collection


    1. Increase
        
        1) Add element: Sadd key member1 member2

    2. Get

        1) Return all elements: smembers key

    3. Delete

        1) Delete the specified element: SREM key values

set

Redis -- deployment operationRedis -- deployment operation

Zset type
    1) Sorted set
    2) Element is of type string
    3) Elements are unique and do not repeat
    4) Each element is associated with a score of double type,
         Represents the weight, which sorts the elements from ⼩ to ⼤ 
    5) Note: there is no modification

    1. Increase
    
        1) Add: zadd key Score1 member1 score2 member2

    2. Get
        
        1) Return elements within the specified range: zrange key start stop
        2) Gets the member whose permission value is between min and Max in the set of key A1
                zrangebyscore a1 5 6
        3) Gets the weight of the element Zhangsan in the set of key A2
                zscore a4 zhangsan
    
    3. Delete
        
        1) Delete the specified element: zrem key member1 member2
        2) Delete elements with weights in the specified range: zremrangebyscore key min max

zset

 

1.5 interaction with Python

Installation package

There are three ways to install redishttps://github.com/andymccurdy/redis-py

  • The first one: enter the virtual environment and connect with the installation package redis
    • pip install redis
  • The second is to enter the virtual environment and connect with the installation package redis
    • easy_install redis
  • The third way: download the source code of redis package to the Chinese official client and install the source code

Call module

  • Lead module
    • from redis import StrictRedis
  • This module providesStrictredis object, ⽤ connect to the redis server, and provide different methods according to different types for interactive operation

  

1.5.1 # strictredis object method

      • Create an object through init, and specify the parameters host and port to connect with the specified server and terminal. The default value of host is localhost, the default value of port is 6379, and the default value of DB is 0
      • sr = StrictRedis(host='localhost', port=6379, db=0)
        
        sr=StrictRedis()

         

      • According to different types, there are different instances. The method can be adjusted. It corresponds to the redis command learned before. The parameters required by the method are consistent with the parameters of the command
        • Redis -- deployment operationRedis -- deployment operation

          1、exists
          2、type
          3、delete
          4、expire
          5、getrange
          6、ttl

          keys

        • Redis -- deployment operationRedis -- deployment operation

          1、set
          2、setex
          3、mset
          4、append
          5、get
          6、mget
          7、key

          string

        • Redis -- deployment operationRedis -- deployment operation

          1、hset
          2、hmset
          3、hkeys
          4、hget
          5、hmget
          6、hvals
          7、hdel

          hash

        • Redis -- deployment operationRedis -- deployment operation

          1、lpush
          2、rpush
          3、linsert
          4、lrange
          5、lset
          6、lrem

          list

        • Redis -- deployment operationRedis -- deployment operation

          1、sadd
          2、smembers
          3、srem

          set

        • Redis -- deployment operationRedis -- deployment operation

          1、zadd
          2、zrange
          3、zrangebyscore
          4、zscore
          5、zrem
          6、zremrangebyscore

          zset

 

1.5.2 example string

      • Redis -- deployment operationRedis -- deployment operation

        ⽅ method set: add keys and values. If the addition is successful, it returns true. If the addition fails, it returns false
        The code is as follows:
        
        
        from redis import *
        if __name__=="__main__":
            try:
                #Create a strictredis object and establish a ⽴ connection with the redis server
                sr=StrictRedis()
                #Add the key name with the value itheima
                result=sr.set('name','itheima')
                #Output the response result. If the addition is successful, it returns true; otherwise, it returns false
                print(result)
            except Exception as e:
                print(e)

        String — add

      • Redis -- deployment operationRedis -- deployment operation

        1) ⽅ method get, add the value corresponding to the key, and return the corresponding value if the key exists,
             Returns none if the key does not exist
        
        The code is as follows:
        
        
        from redis import *
        if __name__=="__main__":
            try:
                #Create a strictredis object and establish a ⽴ connection with the redis server
                sr=StrictRedis()
                #Gets the value of the key name
                result = sr.get('name')
                #输出键的值,Returns none if the key does not exist
                print(result)
            except Exception as e:
                print(e)

        String — get

      • Redis -- deployment operationRedis -- deployment operation

        1) ⽅ method set: if the key already exists, enter ⾏ modify; if the key does not exist, enter ⾏ add
        
        The code is as follows:
        
        
        from redis import *
        if __name__=="__main__":
            try:
                #Create a strictredis object and establish a ⽴ connection with the redis server
                sr=StrictRedis()
                #Set the value of the key name. If the key already exists, enter ⾏ modify; if the key does not exist, enter ⾏ add
                result = sr.set('name','itcast')
                #Output the response result. If the operation is successful, it returns true; otherwise, it returns false
                print(result)
            except Exception as e:
                print(e)

        String — modify

      • Redis -- deployment operationRedis -- deployment operation

        1) ⽅ method delete: delete the key and its corresponding value. If the deletion is successful, the number of affected keys will be returned,
             Otherwise, return 0
        
        The code is as follows:
        
        
        from redis import *
        if __name__=="__main__":
            try:
                #Create a strictredis object and establish a ⽴ connection with the redis server
                sr=StrictRedis()
                #Set the value of the key name. If the key already exists, enter ⾏ modify; if the key does not exist, enter ⾏ add
                result = sr.delete('name')
                #Output the response result. If the deletion is successful, the number of affected keys will be returned; otherwise, 0 will be returned
                print(result)
            except Exception as e:
                print(e)

        String — delete

      • Redis -- deployment operationRedis -- deployment operation

        1) ⽅ method keys to obtain keys according to regular expressions
        
        The code is as follows:
        
        
        from redis import *
        if __name__=="__main__":
            try:
                #Create a strictredis object and establish a ⽴ connection with the redis server
                sr=StrictRedis()
                #Get all keys
                result=sr.keys()
                #Output the response result. All keys form a list. If there are no keys, an empty list will be returned
                print(result)
            except Exception as e:
                print(e)

        String — get key

         

1.6 build master-slave

1.6.1 master slave concept

      • One master can have multiple slaves, and one slave can have multiple slaves. In this way, a strong multi-level server cluster architecture is formed
      • Master is used to write data and slave is used to read data. According to statistics, the read-write ratio of the website is 10:1
      • Read / write separation can be realized through master-slave configuration
      •  

        Both master and slave are redis instances (redis services)

1.6.2 master slave configuration

      Configure master
      • View the IP address of the current host
        • ifconfig

 

      • modify/etc/redis/redis.conffile

           sudo vi redis.conf
           bind 192.168.26.128

      • Restart redis service

           sudo service redis stop
           sudo redis-server redis.conf

Configuration from

      • copy/etc/redis/redis.confDocuments

           sudo cp redis.conf ./slave.conf

      • modifyredis/slave.confDocuments

           sudo vi slave.conf    

      • Edit content

           bind 192.168.26.128   

           port 6378

           slaveof 192.168.26.128 6379  

      • Redis service

           sudo redis-server slave.conf

      • View master-slave relationship

           redis-cli -h 192.168.26.128 info Replication

1.6.3 data operation

      • Execute the ⾏ info command on the master and slave respectively to view the output information and enter the main client

           redis-cli -h 192.168.26.128 -p 6379  

      • Enter client from

           redis-cli -h 192.168.26.128 -p 6378

      • Write data on master

           set aa aa

           

      • Read data on slave

           get aa

1.6.4 interaction with Python

      • Redis -- deployment operationRedis -- deployment operation

        REDIS = {
            'Master':{
                'host':'192.168.56.100',
                'port': '6379',
                'db': 0
            },
            'Slave':{
                'host':'192.168.56.100',
                'port': '6378',
                'db': 0
            },
        }
        
        
        class MSRedis(object):
            '' read write separation client (only for commands used in the program) ''
            def __init__(self,conf):
                self.master = StrictRedis(**conf['Master'])
                self.slave = StrictRedis(**conf['Slave'])
                self.read_commands = [
                    'ttl', 'exist', 'expire', 'get', 'keys',
                    'hget', 'hgetall', 'hkeys', 'hmget',
                    'sismember', 'smembers', 'sdiff', 'sinter', 'sunion'
                    'zrevrange', 'zrevrangebyscore', 'zrevrank', 'zscore'
                ]
        
            def __getattribute__(self, name):
                if name in ['master', 'slave', 'read_commands']:
                    return object.__getattribute__(self, name)
                elif name in self.read_commands:
                    Print ('slave library selected ')
                    return self.slave.__getattribute__(name)
                else:
                    Print ('main library selected ')
                    return self.master.__getattribute__(name)
        
        
        rds = MSRedis(REDIS)
        
        res = rds.get('name2')
        # res = rds.set('name2','lisi')
        print(res)

        redis_test.py

            

1.7 building clusters

1.7.1 machine configuration 1

      • In the demonstration, 192.168.56.100 is the IP address of the current Ubuntu machine
      • Enter the “desktop” record on 192.168.56.100 and create the “conf” record
      • Create ⽂ 7000 pieces under conf y Conf, edit as follows
        • Redis -- deployment operationRedis -- deployment operation

          port 7000
          bind 192.168.56.100
          daemonize yes
          pidfile 7000.pid
          cluster-enabled yes
          cluster-config-file 7000_node.conf
          cluster-node-timeout 15000
          appendonly yes

          7000.conf

      • Create ⽂ part 7001 under conf y Conf, edit as follows
        • Redis -- deployment operationRedis -- deployment operation

          port 7001
          bind 192.168.56.100
          daemonize yes
          pidfile 7001.pid
          cluster-enabled yes
          cluster-config-file 7001_node.conf
          cluster-node-timeout 15000
          appendonly yes

          7001.conf

      • Create ⽂ part 7002 under conf y Conf, edit as follows
        • Redis -- deployment operationRedis -- deployment operation

          port 7002
          bind 192.168.56.100
          daemonize yes
          pidfile 7002.pid
          cluster-enabled yes
          cluster-config-file 7002_node.conf
          cluster-node-timeout 15000
          appendonly yes

          7002.conf

      • Summary: the configuration differences of the three pieces are port, pidfile and cluster config file
      • Enable the ⽤ configuration ⽂ piece to start the redis service
        • redis-server 7000.conf
          redis-server 7001.conf
          redis-server 7002.conf

           

      • The viewing process is shown below

1.7.2 machine configuration 2

      • In the demonstration, 192.168.56.100 is the IP address of the current Ubuntu machine
      • Enter the “desktop” record on 192.168.56.100 and create the “conf” record
      • Create ⽂ part 7003 under conf y Conf, edit as follows

        • Redis -- deployment operationRedis -- deployment operation

          port 7003
          bind 192.168.56.100
          daemonize yes
          pidfile 7003.pid
          cluster-enabled yes
          cluster-config-file 7003_node.conf
          cluster-node-timeout 15000
          appendonly yes

          7003.con  

      • Create ⽂ part 7004 under conf y Conf, edit as follows
        • Redis -- deployment operationRedis -- deployment operation

          port 7004
          bind 192.168.56.100
          daemonize yes
          pidfile 7004.pid
          cluster-enabled yes
          cluster-config-file 7004_node.conf
          cluster-node-timeout 15000
          appendonly yes

          7004.conf

      • Create ⽂ part 7005 under conf y Conf, edit as follows
        • Redis -- deployment operationRedis -- deployment operation

          port 7005
          bind 192.168.56.100
          daemonize yes
          pidfile 7005.pid
          cluster-enabled yes
          cluster-config-file 7005_node.conf
          cluster-node-timeout 15000
          appendonly yes

          7005.conf

      • Summary: the configuration differences of the three pieces are port, pidfile and cluster config file
      • Enable the ⽤ configuration ⽂ piece to start the redis service
        • redis-server 7003.conf
          redis-server 7004.conf
          redis-server 7005.conf
      • The viewing process is shown below

    

1.7.3 creating clusters

      • The installation package of redis includes redis trib RB, ⽤ for creating clusters
      • The next operation is carried out on the 192.168.56.100 machine
      • Copy the command so that it can be lowered in any record
      • sudo cp /usr/share/doc/redis-tools/examples/redis-trib.rb /usr/local/bin/
      • Install the ruby environment because redis trib RB is developed by ruby
      • sudo apt-get install ruby
      • Enter ⼊ y at the prompt, and then go back to ⻋ to continue the installation
      • Run the following command to create a cluster
      • redis-trib.rb create --replicas 1 192.168.56.100:7000 192.168.56.100:7001 192.168.56.100:7002 192.168.56.100:7003 192.168.56.100:7004 192.168.56.100:7005
      • If you execute this command, an error may be reported on some machines. The main reason is that the installed ruby is not the latest version!
      • China’s defense wall leads to the download of the latest version, so you need to set the source of gem
      • The solution is as follows
      • --First, check the address of your gem source
        Gem source - L -- if yes https://rubygems.org/ It needs to be replaced
        --The replacement instruction is
        gem sources --add https://gems.ruby-china.com/ --remove https://rubygems.org/
        --Install redis dependencies through gem
        sudo gem install redis
        --Then re execute the ⾏ command
      • redis-trib.rb create --replicas 1 192.168.56.100:7000 192.168.56.100:7001 192.168.56.100:7002 192.168.56.100:7003 192.168.56.100:7004 192.168.56.100:7005

         

      • The following master-slave information is prompted. Enter ⼊ yes and return ⻋
      • The prompt is completed and the cluster is built successfully

    

1.7.4 data verification

      • As can be seen from the above figure, the master servers currently built are 7000, 7001 and 7002, and the corresponding slave servers are 7003, 7004 and 7005
      • Connect 7002 on the 192.168.56.100 machine, and add the parameter – C to connect to the cluster
      • redis-cli -h 172.16.179.131 -c -p 7002
      • Write data
      • set name itheima
      • ⾃ jump to the 7003 server and write the data successfully
      • Data can be obtained in 7003, and if the written data is redirected to 7000 (load balancing)

 

    

1.7.5. On which server to write data: CRC16

      • During the design of redis cluster, it takes into account the de centralization and de middleware, that is, each node in the cluster is equal and equal, and each node saves its own data and the state of the whole cluster. Each node is connected to all other nodes, and these connections remain active, which ensures that we only need to connect to any node in the cluster to obtain the data of other nodes
      • Redis cluster does not use the traditional consistent hash to allocate data, but uses another type of hash slot to allocate data. Redis cluster assigns 16384 slots by default. When we set ⼀ keys, we will use the CRC16 algorithm to obtain the slot, and then divide the key to the nodes in the hash slot interval. The specific algorithm is: CRC16 (key)% 16384. Therefore, when we see set and get during the test, we directly jump to the node at the end of 7000
      • Redis cluster will store the data in a master node, and then synchronize the data between this master and its corresponding salve. When reading data, it also obtains data from the corresponding master node according to the consistency hash algorithm. Only when ⼀ masters hang up can ⼀ corresponding save nodes be started to act as masters
      • It should be noted that there must be three or more primary nodes, otherwise it will fail when creating the cluster, and when the number of surviving primary nodes is less than half of the total number of nodes, the whole cluster can provide services

   

1.7.6 interaction with Python

      • Redis -- deployment operationRedis -- deployment operation

        from rediscluster import *
        
        if __name__ == '__main__':
          try:
            #When all nodes are built, redis will enable the ⽤ CRC16 algorithm to write keys and values to a node
            startup_nodes = [
                {'host': '192.168.56.100', 'port': '7000'},
                {'host': '192.168.56.100', 'port': '7001'},
                {'host': '192.168.56.100', 'port': '7002'},
                {'host': '192.168.56.100', 'port': '7003'},
                {'host': '192.168.56.100', 'port': '7004'},
                {'host': '192.168.56.100', 'port': '7005'},
            ]
            #Building a strictrediscluster object
            src=RedisCluster(startup_nodes=startup_nodes,decode_responses=True) #decode_ Responses = true, the returned content will be automatically decoded
            #Set the data with the key name and the value itheima
            result=src.set('name','zhangsan')
            print(result)
            #Get key name
            name = src.get('name')
            print(name)
          except Exception as e:
            print(e)
        
        
        #1. The storage location does not need to be managed because it is automatically allocated to a server through hash
        #2. At present, our development only requires high performance, but we haven't yet
        #After the high availability crashes, the processing method of the crash is to hand over the high availability to the operation and maintenance department

        redis_cluster.py

         

Recommended Today

[issue 31] 360 background development practice – two rounds of technical aspects

one side self-introduction Project related usedatabase/sqlAnd usegormDifferences between Why use redis connection pool Basic knowledge Process / thread / coroutine difference Three characteristics of object-oriented (combined with go) How to limit the number of goroutine concurrencies: channel or waitgroup Questioning link What knowledge can go development add High concurrency / distributed Performance monitoring / performance […]