Redis cluster for redis services

Time:2021-1-26

In the last blog, we talked about the configuration of sentinel, a highly available component of redis. Please refer tohttps://www.cnblogs.com/qiuhom-1874/p/13429776.htmlIn the master-slave synchronization architecture of redis, sentinel mainly monitors whether the cluster master is normal. If the master is abnormal or down, sentinel will promote a slave to select a new master, so as to ensure the normal use of redis services. However, the problem of single machine writing in redis still exists. In the master-slave synchronization architecture of sentinel +, the program writing data is always the key to the success of redis Read and write requests are sent to the master (of course, if there is a statement router, the read and write requests can be separated). In this way, for the master, it undertakes all the write operations. Obviously, in this scenario where the write operations are very frequent, a single master certainly can’t bear such a great pressure. In order to solve the bottleneck problem of single master writing data, redis is officially in redis After version 3.0, the mechanism of redis cluster without central architecture has been introduced. The so-called cluster without central architecture means that every member of the cluster is a central node, and each member has the status metadata of the whole cluster, but each member only stores part of the data of the whole cluster. For redis, the cluster with central architecture is the same as the cluster without central architecture In terms of cluster architecture, all redis nodes are interconnected by (Ping mechanism). If a node in the cluster fails, more than half of the monitoring nodes in the whole cluster will fail before the real failure occurs. The client does not need a proxy to directly connect to redis, and the application needs to write all the redis server IP addresses. Redis cluster maps all redis nodes to 0-16383 slots. Read and write operations need to be performed on the specified redis nodes. Therefore, the number of redis nodes is equivalent to the number of times of redis concurrent expansion. Redis cluster allocates 16384 slots in advance. When a key value needs to be written in the redis cluster, the value after CRC16 (key) mod 16384 will be used to determine which slot to write the key value to, so as to determine which redis node to write to, so as to effectively solve the read-write bottleneck of redis single machine. As shown below

Tip: every time the client reads and writes, it needs to hash the read-write key through CRC16, and then modulus the hash result with 16384, and the final result will fall to a slot 0-16383. When we define the cluster in advance, we assign slot 0-5460 to master01, which means that as long as the key of the read-write data is calculated through crc16hash, the result% 16384 will be the best When the result is in the range of 0-5460, the read / write request is scheduled to master01; similarly, when the result is in the range of 5461-10922, the read / write request is scheduled to master02, and when the result is in the range of 10923-16383, the read / write request is scheduled to master03; From the above figure, we can also see that each master only saves part of the data of the whole cluster. If a Master goes down, it will cause the corresponding data of the slots on the corresponding master to be lost. In order to solve the single point problem of each master, we also need to be master-slave for each master, as shown in the figure below

Tip: in order to prevent data loss and write failure after a Master goes down, the corresponding slave should be promoted to master in time, which is similar to the sentinel function, but we don’t need to deploy sentinel, because redis Cluster supports master-slave switching. In this way, our cluster becomes a cluster of three masters and three slaves. Next, we prepare an experimental environment to deploy the above architecture;

Deploying redis cluster

Environmental description

role IP address port
Master01 192.168.0.41 6379
Slave01 192.168.0.42 6380
Master02 192.168.0.42 6379
Slave02 192.168.0.43 6380
Master03 192.168.0.43 6379
Slave03 192.168.0.41 6380

 

 

 

 

 

 

 

 

  

Prepare to start three servers, and start two redis instances on each server. The ports are 6379 (Master) and 6380 (slave). In order to prevent the master and slave from falling on the same node, they are purposely combined, as shown in the table above;

To create a redis cluster, we need to pay attention to that each redis node uses the same hardware configuration and password as much as possible; all redis servers must have no data;

Compile and install redis on node01. Please refer tohttps://www.cnblogs.com/qiuhom-1874/p/13378138.html

Create directory structure

[[email protected] ~]# tree /usr/local/redis/
/usr/local/redis/
├── 6379
│   ├── etc
│   │   ├── redis.conf
│   │   └── sentinel.conf
│   ├── logs
│   └── run
├── 6380
│   ├── etc
│   │   ├── redis.conf
│   │   └── sentinel.conf
│   ├── logs
│   └── run
└── bin
    ├── redis-benchmark
    ├── redis-check-aof
    ├── redis-check-rdb
    ├── redis-cli
    ├── redis-sentinel -> redis-server
    └── redis-server

9 directories, 10 files
[[email protected] ~]#

Tip: create 6379 and 6380 under the / usr / local / redis directory, and then create etc, logs, run and other subdirectories under them, as shown above

Modify configuration file

[[email protected] ~]# grep -E "^(bind|requirepass|masterauth|cluster)" /usr/local/redis/6379/etc/redis.conf 
bind 0.0.0.0
masterauth admin
requirepass admin
cluster-enabled yes
cluster-config-file redis-cluster_6379.conf
[[email protected] ~]# grep -E "^(bind|requirepass|masterauth|cluster)" /usr/local/redis/6380/etc/redis.conf   
bind 0.0.0.0
masterauth admin
requirepass admin
cluster-enabled yes
cluster-config-file redis-cluster_6380.conf
[[email protected] ~]# 

Tip: the contents of the configuration files of 6379 and 6380 are the same. The only difference is that the ports are different. It is mainly to modify the bind listening address, set the password, open the cluster and specify the name of the configuration file generated by the corresponding cluster;

Copy / usr / local / redis on node01 to node02 and node03 and save it to the / usr / local / directory

Tip: all the three hosts above me have done password free login, so no password is required to copy data from each other. Please refer to the description and configuration of password free loginhttps://www.cnblogs.com/qiuhom-1874/p/11783371.html

Start redis on node01, node02 and node03 respectively

So far, we have started six redis instances. Next, we will create a cluster. Redis 3 and 4 need to use the cluster management tool redis- trib.rb This tool is an official redis cluster management tool. It is integrated in the source code SRC directory of redis and is based on redis The cluster commands are encapsulated into a simple, convenient and practical operation tool, redis- trib.rb It was developed by the redis author with ruby, so we need to use redis- trib.rb First of all, we should solve the problem of ruby environment;

Install Ruby environment

[[email protected] ~]# rz
rz waiting to receive.
 zmodem trl+C ȡ

  100%   15621 KB 15621 KB/s 00:00:01       0 Errors

[[email protected] ~]# ls
redis-4.0.9.tar.gz  ruby-2.5.5.tar.gz
[[email protected] ~]# tar xf ruby-2.5.5.tar.gz  -C /usr/local/src/
[[email protected] ~]# cd /usr/local/src/
[[email protected] src]# cd ruby-2.5.5/
[[email protected] ruby-2.5.5]# ./configure 
checking for ruby... false
checking build system type... x86_64-pc-linux-gnu
checking host system type... x86_64-pc-linux-gnu
checking target system type... x86_64-pc-linux-gnu
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking for g++... g++
checking whether we are using the GNU C++ compiler... yes
…… Omit part of
checking for setjmp type... __builtin_setjmp
checking for prefix of external symbols... NONE
checking pthread.h usability... yes
checking pthread.h presence... yes
checking for pthread.h... yes
checking if make is GNU make... yes
.ext/include/x86_64-linux/ruby/config.h updated
configure: ruby library version = 2.5.0
configure: creating ./config.status
config.status: creating GNUmakefile
config.status: creating Makefile
config.status: creating ruby-2.5.pc
---
Configuration summary for ruby version 2.5.5

   * Installation prefix: /usr/local
   * exec prefix:         ${prefix}
   * arch:                x86_64-linux
   * site arch:           ${arch}
   * RUBY_BASE_NAME:      ruby
   * ruby lib prefix:     ${libdir}/${RUBY_BASE_NAME}
   * site libraries path: ${rubylibprefix}/${sitearch}
   * vendor path:         ${rubylibprefix}/vendor_ruby
   * target OS:           linux
   * compiler:            gcc
   * with pthread:        yes
   * enable shared libs:  no
   * dynamic library ext: so
   * CFLAGS:              ${optflags} ${debugflags} ${warnflags}
   * LDFLAGS:             -L. -fstack-protector -rdynamic \
                          -Wl,-export-dynamic
   * optflags:            -O3
   * debugflags:          -ggdb3
   * warnflags:           -Wall -Wextra -Wno-unused-parameter \
                          -Wno-parentheses -Wno-long-long \
                          -Wno-missing-field-initializers \
                          -Wno-tautological-compare \
                          -Wno-parentheses-equality \
                          -Wno-constant-logical-operand -Wno-self-assign \
                          -Wunused-variable -Wimplicit-int -Wpointer-arith \
                          -Wwrite-strings -Wdeclaration-after-statement \
                          -Wimplicit-function-declaration \
                          -Wdeprecated-declarations \
                          -Wno-packed-bitfield-compat \
                          -Wsuggest-attribute=noreturn \
                          -Wsuggest-attribute=format
   * strip command:       strip -S -x
   * install doc:         yes
   * man page type:       doc

---
[[email protected] ruby-2.5.5]#

Tip: redis- trib.rb This tool can be installed on any one of the cluster, and it does not need to be installed on every one;

compile

[[email protected] ruby-2.5.5]# make -j 2
        CC = gcc
        LD = ld
        LDSHARED = gcc -shared
        CFLAGS = -O3 -ggdb3 -Wall -Wextra -Wno-unused-parameter -Wno-parentheses -Wno-long-long -Wno-missing-field-initializers -Wno-tautological-compare -Wno-parentheses-equality -Wno-constant-logical-operand -Wno-self-assign -Wunused-variable -Wimplicit-int -Wpointer-arith -Wwrite-strings -Wdeclaration-after-statement -Wimplicit-function-declaration -Wdeprecated-declarations -Wno-packed-bitfield-compat -Wsuggest-attribute=noreturn -Wsuggest-attribute=format -std=gnu99 
        XCFLAGS = -D_FORTIFY_SOURCE=2 -fstack-protector -fno-strict-overflow -fvisibility=hidden -fexcess-precision=standard -DRUBY_EXPORT -fPIE
        CPPFLAGS =   -I. -I.ext/include/x86_64-linux -I./include -I. -I./enc/unicode/10.0.0
        DLDFLAGS = -fstack-protector -pie  
        SOLIBS = 
        LANG = en_US.UTF-8
        LC_ALL = 
        LC_CTYPE = 
gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-39)
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

compiling ./main.c
compiling dmydln.c
compiling miniinit.c
compiling dmyext.c
compiling miniprelude.c
making dummy probes.h
compiling bignum.c
compiling class.c
compiling compar.c
compiling compile.c
compiling complex.c
compiling cont.c
compiling debug.c
compiling debug_counter.c
compiling dir.c
compiling dln_find.c
compiling encoding.c
compiling enum.c
compiling enumerator.c
…… Omit part of
*** Fix the problems, then remove these directories and try again if you want.
make[1]: Leaving directory `/usr/local/src/ruby-2.5.5'
Generating RDoc documentation
Parsing sources...
100% [871/871]  vsnprintf.c                                                                           

Generating RI format into /usr/local/src/ruby-2.5.5/.ext/rdoc...

  Files:        871

  Classes:     1324 ( 565 undocumented)
  Modules:      286 ( 121 undocumented)
  Constants:   2181 ( 555 undocumented)
  Attributes:  1066 ( 251 undocumented)
  Methods:    10080 (2161 undocumented)

  Total:      14937 (3653 undocumented)
   75.54% documented

  Elapsed: 25.0s

[[email protected] ruby-2.5.5]#

Install Ruby environment

[[email protected] ruby-2.5.5]# make install
        CC = gcc
        LD = ld
        LDSHARED = gcc -shared
        CFLAGS = -O3 -ggdb3 -Wall -Wextra -Wno-unused-parameter -Wno-parentheses -Wno-long-long -Wno-missing-field-initializers -Wno-tautological-compare -Wno-parentheses-equality -Wno-constant-logical-operand -Wno-self-assign -Wunused-variable -Wimplicit-int -Wpointer-arith -Wwrite-strings -Wdeclaration-after-statement -Wimplicit-function-declaration -Wdeprecated-declarations -Wno-packed-bitfield-compat -Wsuggest-attribute=noreturn -Wsuggest-attribute=format -std=gnu99 
        XCFLAGS = -D_FORTIFY_SOURCE=2 -fstack-protector -fno-strict-overflow -fvisibility=hidden -fexcess-precision=standard -DRUBY_EXPORT -fPIE
        CPPFLAGS =   -I. -I.ext/include/x86_64-linux -I./include -I. -I./enc/unicode/10.0.0
        DLDFLAGS = -fstack-protector -pie  
        SOLIBS = 
        LANG = en_US.UTF-8
        LC_ALL = 
        LC_CTYPE = 
gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-39)
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

generating enc.mk
making srcs under enc
make[1]: Entering directory `/usr/local/src/ruby-2.5.5'
make[1]: Nothing to be done for `srcs'.
make[1]: Leaving directory `/usr/local/src/ruby-2.5.5'
generating transdb.h
transdb.h unchanged
generating makefiles ext/configure-ext.mk
ext/configure-ext.mk updated
make[1]: Entering directory `/usr/local/src/ruby-2.5.5'
make[1]: Leaving directory `/usr/local/src/ruby-2.5.5'
generating makefile exts.mk
exts.mk updated
make[1]: Entering directory `/usr/local/src/ruby-2.5.5'
make[2]: Entering directory `/usr/local/src/ruby-2.5.5/ext/fcntl'
make[2]: Leaving directory `/usr/local/src/ruby-2.5.5/ext/fcntl'
make[2]: Entering directory `/usr/local/src/ruby-2.5.5/ext/rubyvm'
…… Omit part of
installing binary commands:         /usr/local/bin
installing base libraries:          /usr/local/lib
installing arch files:              /usr/local/lib/ruby/2.5.0/x86_64-linux
installing pkgconfig data:          /usr/local/lib/pkgconfig
installing command scripts:         /usr/local/bin
installing library scripts:         /usr/local/lib/ruby/2.5.0
installing common headers:          /usr/local/include/ruby-2.5.0
installing manpages:                /usr/local/share/man/man1
installing extension objects:       /usr/local/lib/ruby/2.5.0/x86_64-linux
installing extension objects:       /usr/local/lib/ruby/site_ruby/2.5.0/x86_64-linux
installing extension objects:       /usr/local/lib/ruby/vendor_ruby/2.5.0/x86_64-linux
installing extension headers:       /usr/local/include/ruby-2.5.0/x86_64-linux
installing extension scripts:       /usr/local/lib/ruby/2.5.0
installing extension scripts:       /usr/local/lib/ruby/site_ruby/2.5.0
installing extension scripts:       /usr/local/lib/ruby/vendor_ruby/2.5.0
installing extension headers:       /usr/local/include/ruby-2.5.0/ruby
installing default gems from lib:   /usr/local/lib/ruby/gems/2.5.0 (build_info, cache, doc, extensions, gems, specifications)
                                    cmath 1.0.0
                                    csv 1.0.0
                                    fileutils 1.0.2
                                    ipaddr 1.2.0
                                    rdoc 6.0.1
                                    scanf 1.0.0
                                    webrick 1.4.2
installing default gems from ext:   /usr/local/lib/ruby/gems/2.5.0 (build_info, cache, doc, extensions, gems, specifications)
                                    bigdecimal 1.3.4
                                    date 1.0.0
                                    etc 1.0.0
                                    fcntl 1.0.0
                                    fiddle 1.0.0
                                    io-console 0.4.6
                                    json 2.1.0
                                    psych 3.0.2
                                    sdbm 1.0.0
                                    stringio 0.0.1
                                    strscan 1.0.0
installing bundled gems:            /usr/local/lib/ruby/gems/2.5.0 (build_info, cache, doc, extensions, gems, specifications)
                                    minitest 5.10.3
                                    rake 12.3.0
                                    xmlrpc 0.3.0
                                    did_you_mean 1.2.0
                                    net-telnet 0.1.1
                                    power_assert 1.1.1
                                    test-unit 3.2.7
installing rdoc:                    /usr/local/share/ri/2.5.0/system
installing capi-docs:               /usr/local/share/doc/ruby
[[email protected] ruby-2.5.5]#

Tip: the ruby environment is ready to install. Note that the version of ruby installed by Yum is a little low, so we choose to compile and install here;

Gem install redis package

[[email protected] ruby-2.5.5]# gem install redis
ERROR:  Loading command: install (LoadError)
        cannot load such file -- zlib
ERROR:  While executing gem ... (NoMethodError)
    undefined method `invoke_with_build_args' for nil:NilClass
[[email protected] ruby-2.5.5]#

Tip: Gem installation prompts that we don’t have zlib. In case of this problem, we use Yum to install zlib devel package, and then integrate zlib into Ruby environment;

[[email protected] ruby-2.5.5]# yum install zlib-devel
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: mirrors.aliyun.com
 * extras: mirrors.aliyun.com
 * updates: mirrors.aliyun.com
Resolving Dependencies
--> Running transaction check
---> Package zlib-devel.x86_64 0:1.2.7-18.el7 will be installed
--> Processing Dependency: zlib = 1.2.7-18.el7 for package: zlib-devel-1.2.7-18.el7.x86_64
--> Running transaction check
---> Package zlib.x86_64 0:1.2.7-17.el7 will be updated
---> Package zlib.x86_64 0:1.2.7-18.el7 will be an update
--> Finished Dependency Resolution

Dependencies Resolved

===========================================================================================================
 Package                    Arch                   Version                      Repository            Size
===========================================================================================================
Installing:
 zlib-devel                 x86_64                 1.2.7-18.el7                 base                  50 k
Updating for dependencies:
 zlib                       x86_64                 1.2.7-18.el7                 base                  90 k

Transaction Summary
===========================================================================================================
Install  1 Package
Upgrade             ( 1 Dependent package)

Total download size: 140 k
Is this ok [y/d/N]: y
Downloading packages:
Delta RPMs disabled because /usr/bin/applydeltarpm not installed.
(1/2): zlib-devel-1.2.7-18.el7.x86_64.rpm                                           |  50 kB  00:00:00     
  /2): zlib-1.2.7-18.el7.x86_64.rpm                                                 |  90 kB  00:00:00     
-----------------------------------------------------------------------------------------------------------
Total                                                                      509 kB/s | 140 kB  00:00:00     
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Updating   : zlib-1.2.7-18.el7.x86_64                                                                1/3 
  Installing : zlib-devel-1.2.7-18.el7.x86_64                                                          2/3 
  Cleanup    : zlib-1.2.7-17.el7.x86_64                                                                3/3 
  Verifying  : zlib-devel-1.2.7-18.el7.x86_64                                                          1/3 
  Verifying  : zlib-1.2.7-18.el7.x86_64                                                                2/3 
  Verifying  : zlib-1.2.7-17.el7.x86_64                                                                3/3 

Installed:
  zlib-devel.x86_64 0:1.2.7-18.el7                                                                         

Dependency Updated:
  zlib.x86_64 0:1.2.7-18.el7                                                                               

Complete!
[[email protected] ruby-2.5.5]#

Integrating zlib library into Ruby environment

[[email protected] ruby-2.5.5]# cd ext/zlib/
[[email protected] zlib]# ruby extconf.rb
checking for deflateReset() in -lz... yes
checking for zlib.h... yes
checking for crc32_combine() in zlib.h... yes
checking for adler32_combine() in zlib.h... yes
checking for z_crc_t in zlib.h... yes
creating Makefile
[[email protected] zlib]# make 
make: *** No rule to make target `/include/ruby.h', needed by `zlib.o'.  Stop.
[[email protected] zlib]#

Tip: in case of such a problem, we need to open the makefile generated above and put zlib. O: $(top_ Srcdir) / include / ruby. H is replaced by zlib. O: /. / include / ruby. H

Tip: modify makefile and make it again

"Makefile" 282L, 8468C written
[[email protected] zlib]# make
compiling zlib.c
linking shared-object zlib.so
[[email protected] zlib]# make install
/usr/bin/install -c -m 0755 zlib.so /usr/local/lib/ruby/site_ruby/2.5.0/x86_64-linux
[[email protected] zlib]#

Use gem to install redis package again

[[email protected] zlib]# gem install redis
ERROR:  While executing gem ... (Gem::Exception)
    Unable to require openssl, install OpenSSL and rebuild Ruby (preferred) or use non-HTTPS sources
[[email protected] zlib]#

Tip: it prompts us that we are lack of OpenSSL library. The processing method is to install OpenSSL devel package, and then integrate OpenSSL library into Ruby environment;

Install OpenSSL devel package

[[email protected] zlib]# yum install openssl-devel
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: mirrors.aliyun.com
 * extras: mirrors.aliyun.com
 * updates: mirrors.aliyun.com
Resolving Dependencies
--> Running transaction check
---> Package openssl-devel.x86_64 1:1.0.2k-19.el7 will be installed
--> Processing Dependency: openssl-libs(x86-64) = 1:1.0.2k-19.el7 for package: 1:openssl-devel-1.0.2k-19.el7.x86_64
--> Processing Dependency: krb5-devel(x86-64) for package: 1:openssl-devel-1.0.2k-19.el7.x86_64
--> Running transaction check
---> Package krb5-devel.x86_64 0:1.15.1-46.el7 will be installed
…… Omit part of
Installed:
  openssl-devel.x86_64 1:1.0.2k-19.el7                                                                     

Dependency Installed:
  keyutils-libs-devel.x86_64 0:1.5.8-3.el7                krb5-devel.x86_64 0:1.15.1-46.el7                
  libcom_err-devel.x86_64 0:1.42.9-17.el7                 libkadm5.x86_64 0:1.15.1-46.el7                  
  libselinux-devel.x86_64 0:2.5-15.el7                    libsepol-devel.x86_64 0:2.5-10.el7               
  libverto-devel.x86_64 0:0.2.5-4.el7                     pcre-devel.x86_64 0:8.32-17.el7                  

Dependency Updated:
  e2fsprogs.x86_64 0:1.42.9-17.el7                    e2fsprogs-libs.x86_64 0:1.42.9-17.el7               
  krb5-libs.x86_64 0:1.15.1-46.el7                    libcom_err.x86_64 0:1.42.9-17.el7                   
  libselinux.x86_64 0:2.5-15.el7                      libselinux-python.x86_64 0:2.5-15.el7               
  libselinux-utils.x86_64 0:2.5-15.el7                libsepol.x86_64 0:2.5-10.el7                        
  libss.x86_64 0:1.42.9-17.el7                        openssl.x86_64 1:1.0.2k-19.el7                      
  openssl-libs.x86_64 1:1.0.2k-19.el7                

Complete!
[[email protected] zlib]#

Integrating OpenSSL into Ruby environment

[[email protected] zlib]# cd ..
[[email protected] ext]# ls
bigdecimal        date      fcntl   json      psych     ripper        Setup.nt  -test-
cgi               dbm       fiber   nkf       pty       rubyvm        socket    win32
configure-ext.mk  digest    fiddle  objspace  racc      sdbm          stringio  win32ole
continuation      etc       gdbm    openssl   rbconfig  Setup         strscan   zlib
coverage          extmk.rb  io      pathname  readline  Setup.atheos  syslog
[[email protected] ext]# cd openssl/
[[email protected] openssl]# ls
depend             ossl_asn1.c    ossl_digest.c   ossl_ocsp.c      ossl_pkey_rsa.c     ossl_x509ext.c
deprecation.rb     ossl_asn1.h    ossl_digest.h   ossl_ocsp.h      ossl_rand.c         ossl_x509.h
extconf.rb         ossl_bio.c     ossl_engine.c   ossl_pkcs12.c    ossl_rand.h         ossl_x509name.c
exts.mk            ossl_bio.h     ossl_engine.h   ossl_pkcs12.h    ossl_ssl.c          ossl_x509req.c
History.md         ossl_bn.c      ossl.h          ossl_pkcs7.c     ossl_ssl.h          ossl_x509revoked.c
lib                ossl_bn.h      ossl_hmac.c     ossl_pkcs7.h     ossl_ssl_session.c  ossl_x509store.c
Makefile           ossl.c         ossl_hmac.h     ossl_pkey.c      ossl_version.h      ruby_missing.h
mkmf.log           ossl_cipher.c  ossl_kdf.c      ossl_pkey_dh.c   ossl_x509attr.c
openssl.gemspec    ossl_cipher.h  ossl_kdf.h      ossl_pkey_dsa.c  ossl_x509.c
openssl_missing.c  ossl_config.c  ossl_ns_spki.c  ossl_pkey_ec.c   ossl_x509cert.c
openssl_missing.h  ossl_config.h  ossl_ns_spki.h  ossl_pkey.h      ossl_x509crl.c
[[email protected] openssl]# ruby extconf.rb 
checking for t_open() in -lnsl... no
checking for socket() in -lsocket... no
checking for openssl/ssl.h... yes
checking for OpenSSL version is 1.0.1 or later... yes
checking for RAND_egd()... yes
…… Omit part of
checking for X509_get0_notBefore()... no
checking for SSL_SESSION_get_protocol_version()... no
checking for EVP_PBE_scrypt()... no
creating extconf.h
creating Makefile
[[email protected] openssl]# make
compiling openssl_missing.c
make: *** No rule to make target `/include/ruby.h', needed by `ossl.o'.  Stop.
[[email protected] openssl]#

Tip: this error is similar to inheriting the zlib library to the ruby environment just now. The processing method is the same as above. Modify the makefile file and add top to the position where the variables are defined in the makefile_ srcdir = ../..

Make again

Makefile" 1458L, 49182C written
[[email protected] openssl]# make
compiling ossl.c
compiling ossl_asn1.c
compiling ossl_bio.c
compiling ossl_bn.c
compiling ossl_cipher.c
compiling ossl_config.c
compiling ossl_digest.c
compiling ossl_engine.c
compiling ossl_hmac.c
compiling ossl_kdf.c
compiling ossl_ns_spki.c
compiling ossl_ocsp.c
compiling ossl_pkcs12.c
compiling ossl_pkcs7.c
compiling ossl_pkey.c
compiling ossl_pkey_dh.c
compiling ossl_pkey_dsa.c
compiling ossl_pkey_ec.c
compiling ossl_pkey_rsa.c
compiling ossl_rand.c
compiling ossl_ssl.c
compiling ossl_ssl_session.c
compiling ossl_x509.c
compiling ossl_x509attr.c
compiling ossl_x509cert.c
compiling ossl_x509crl.c
compiling ossl_x509ext.c
compiling ossl_x509name.c
compiling ossl_x509req.c
compiling ossl_x509revoked.c
compiling ossl_x509store.c
linking shared-object openssl.so
[[email protected] openssl]# make install
/usr/bin/install -c -m 0755 openssl.so /usr/local/lib/ruby/site_ruby/2.5.0/x86_64-linux
installing default openssl libraries
[[email protected] openssl]# 

Install redis package with gem again

[[email protected] openssl]# gem install redis
Fetching: redis-4.2.1.gem (100%)
Successfully installed redis-4.2.1
Parsing documentation for redis-4.2.1
Installing ri documentation for redis-4.2.1
Done installing documentation for redis after 0 seconds
1 gem installed
[[email protected] openssl]# 

Tip: at this point, the ruby environment is ready, and the redis package is also installed. Next, let’s add redis- trib.rb Connect to / usr / bin / and execute redis- trib.rb If it can be executed normally, it means that there is no problem with the ruby environment. If not, it needs to be adjusted

Soft connection redis- trib.rb To / usr / bin/

[[email protected] openssl]# find /usr/local/src/redis-4.0.9/ -name "redis-trib.rb"
/usr/local/src/redis-4.0.9/src/redis-trib.rb
[[email protected] openssl]# ln -s /usr/local/src/redis-4.0.9/src/redis-trib.rb /usr/bin/
[[email protected] openssl]# redis-trib.rb 
Usage: redis-trib   

  create          host1:port1 ... hostN:portN
                  --replicas 
  check           host:port
  info            host:port
  fix             host:port
                  --timeout 
  reshard         host:port
                  --from 
                  --to 
                  --slots 
                  --yes
                  --timeout 
                  --pipeline 
  rebalance       host:port
                  --weight 
                  --auto-weights
                  --use-empty-masters
                  --timeout 
                  --simulate
                  --pipeline 
                  --threshold 
  add-node        new_host:new_port existing_host:existing_port
                  --slave
                  --master-id 
  del-node        host:port node_id
  set-timeout     host:port milliseconds
  call            host:port command arg arg .. arg
  import          host:port
                  --from 
                  --copy
                  --replace
  help            (show this help)

For check, fix, reshard, del-node, set-timeout you can specify the host and port of any working node in the cluster.
[[email protected] openssl]#

Tip: you can see redis- trib.rb The help information can be printed out normally, which indicates that the current Ruby environment can support redis- trib.rb function;

Add the password to the library file of the redis package you just downloaded

Tip: you need to quote the corresponding password string here;

Creating clusters

[[email protected] ~]# redis-trib.rb create --replicas 1 192.168.0.41:6379 192.168.0.42:6379 192.168.0.43:6379 192.168.0.41:6380 192.168.0.42:6380 192.168.0.43:6380
>>> Creating cluster
>>> Performing hash slots allocation on 6 nodes...
Using 3 masters:
192.168.0.41:6379
192.168.0.42:6379
192.168.0.43:6379
Adding replica 192.168.0.42:6380 to 192.168.0.41:6379
Adding replica 192.168.0.43:6380 to 192.168.0.42:6379
Adding replica 192.168.0.41:6380 to 192.168.0.43:6379
M: 8c785e6ec3f8f7ff4fb7768765da8b8a93f26855 192.168.0.41:6379
   slots:0-5460 (5461 slots) master
M: e99b0b450e78719d63520cb6efc068d5e8d4d081 192.168.0.42:6379
   slots:5461-10922 (5462 slots) master
M: a7ace08c36f7d55c4f28463d72865aa1ff74829e 192.168.0.43:6379
   slots:10923-16383 (5461 slots) master
S: 62ece0b80b83c0f1f078b07fc1687bb8376f76b3 192.168.0.41:6380
   replicates a7ace08c36f7d55c4f28463d72865aa1ff74829e
S: dbfff4c49a94c0ee55d14401ccc9245af3655427 192.168.0.42:6380
   replicates 8c785e6ec3f8f7ff4fb7768765da8b8a93f26855
S: 91169e71359deed96f8778cf31c823dbd6ded350 192.168.0.43:6380
   replicates e99b0b450e78719d63520cb6efc068d5e8d4d081
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join......
>>> Performing Cluster Check (using node 192.168.0.41:6379)
M: 8c785e6ec3f8f7ff4fb7768765da8b8a93f26855 192.168.0.41:6379
   slots:0-5460 (5461 slots) master
   1 additional replica(s)
S: 91169e71359deed96f8778cf31c823dbd6ded350 192.168.0.43:6380
   slots: (0 slots) slave
   replicates e99b0b450e78719d63520cb6efc068d5e8d4d081
M: e99b0b450e78719d63520cb6efc068d5e8d4d081 192.168.0.42:6379
   slots:5461-10922 (5462 slots) master
   1 additional replica(s)
M: a7ace08c36f7d55c4f28463d72865aa1ff74829e 192.168.0.43:6379
   slots:10923-16383 (5461 slots) master
   1 additional replica(s)
S: dbfff4c49a94c0ee55d14401ccc9245af3655427 192.168.0.42:6380
   slots: (0 slots) slave
   replicates 8c785e6ec3f8f7ff4fb7768765da8b8a93f26855
S: 62ece0b80b83c0f1f078b07fc1687bb8376f76b3 192.168.0.41:6380
   slots: (0 slots) slave
   replicates a7ace08c36f7d55c4f28463d72865aa1ff74829e
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
[[email protected] ~]# 

Tip: from the above information, we can see that we have specified the addresses and ports of six redis instances. It has created three masters and three slaves for us, and allocated 16384 slots on the three masters averagely. If you can see the OK all 16384 slots covered, It indicates that the cluster is created successfully; — replicates is used to specify the number of replicas of each master in the cluster. 1 means that each master has one replica;

View cluster status information

[[email protected] ~]# redis-trib.rb info 192.168.0.41:6379                    
192.168.0.41:6379 (8c785e6e...) -> 0 keys | 5461 slots | 1 slaves.
192.168.0.42:6379 (e99b0b45...) -> 0 keys | 5462 slots | 1 slaves.
192.168.0.43:6379 (a7ace08c...) -> 0 keys | 5461 slots | 1 slaves.
[OK] 0 keys in 3 masters.
0.00 keys per slot on average.
[[email protected] ~]# redis-trib.rb check 192.168.0.41:6379     
>>> Performing Cluster Check (using node 192.168.0.41:6379)
M: 8c785e6ec3f8f7ff4fb7768765da8b8a93f26855 192.168.0.41:6379
   slots:0-5460 (5461 slots) master
   1 additional replica(s)
S: 91169e71359deed96f8778cf31c823dbd6ded350 192.168.0.43:6380
   slots: (0 slots) slave
   replicates e99b0b450e78719d63520cb6efc068d5e8d4d081
M: e99b0b450e78719d63520cb6efc068d5e8d4d081 192.168.0.42:6379
   slots:5461-10922 (5462 slots) master
   1 additional replica(s)
M: a7ace08c36f7d55c4f28463d72865aa1ff74829e 192.168.0.43:6379
   slots:10923-16383 (5461 slots) master
   1 additional replica(s)
S: dbfff4c49a94c0ee55d14401ccc9245af3655427 192.168.0.42:6380
   slots: (0 slots) slave
   replicates 8c785e6ec3f8f7ff4fb7768765da8b8a93f26855
S: 62ece0b80b83c0f1f078b07fc1687bb8376f76b3 192.168.0.41:6380
   slots: (0 slots) slave
   replicates a7ace08c36f7d55c4f28463d72865aa1ff74829e
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
[[email protected] ~]# 

Tip: from the above information, we can see that the 6379 ports of 192.168.0.41/42/43 are all master and have a slave node;

Confirm master status

[[email protected] ~]# redis-cli -h 192.168.0.41
192.168.0.41:6379> AUTH admin
OK
192.168.0.41:6379> CLUSTER INFO
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:6
cluster_my_epoch:1
cluster_stats_messages_ping_sent:606
cluster_stats_messages_pong_sent:586
cluster_stats_messages_sent:1192
cluster_stats_messages_ping_received:581
cluster_stats_messages_pong_received:606
cluster_stats_messages_meet_received:5
cluster_stats_messages_received:1192
192.168.0.41:6379> info replication
# Replication
role:master
connected_slaves:1
slave0:ip=192.168.0.42,port=6380,state=online,offset=854,lag=0
master_replid:303943ae1e9f54e3525abda2fd1efff905bf08e3
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:854
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:854
192.168.0.41:6379> quit
[[email protected] ~]# redis-cli -h 192.168.0.42
192.168.0.42:6379> auth admin
OK
192.168.0.42:6379> info replication
# Replication
role:master
connected_slaves:1
slave0:ip=192.168.0.43,port=6380,state=online,offset=896,lag=0
master_replid:f4e1b75b8f30be622868814fc4618a4584f3aa32
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:896
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:896
192.168.0.42:6379> CLUSTER INFO
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:6
cluster_my_epoch:2
cluster_stats_messages_ping_sent:662
cluster_stats_messages_pong_sent:645
cluster_stats_messages_meet_sent:5
cluster_stats_messages_sent:1312
cluster_stats_messages_ping_received:644
cluster_stats_messages_pong_received:667
cluster_stats_messages_meet_received:1
cluster_stats_messages_received:1312
192.168.0.42:6379> quit
[[email protected] ~]# redis-cli -h 192.168.0.43
192.168.0.43:6379> AUTH admin
OK
192.168.0.43:6379> CLUSTER INFO
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:6
cluster_my_epoch:3
cluster_stats_messages_ping_sent:683
cluster_stats_messages_pong_sent:787
cluster_stats_messages_meet_sent:4
cluster_stats_messages_sent:1474
cluster_stats_messages_ping_received:786
cluster_stats_messages_pong_received:687
cluster_stats_messages_meet_received:1
cluster_stats_messages_received:1474
192.168.0.43:6379> info replication
# Replication
role:master
connected_slaves:1
slave0:ip=192.168.0.41,port=6380,state=online,offset=952,lag=1
master_replid:7cbc7d52452717cd49a0e543f6efdccafe5fe1bd
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:952
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:952
192.168.0.43:6379> quit
[[email protected] ~]# 

Prompt: you can see that the cluster status is OK, and 6379 of 41 corresponds to 6380 of 42, 6379 of 42 corresponds to 6380 of 43, 6379 of 43 corresponds to 6380 of 41, just the master and slave are staggered;

View the corresponding relationship of each node in the cluster

Tip: the node above corresponds to the ID number. At this point, the redis cluster runs normally;

Verification: write any data on the master to see what happens?

Tip: when writing K1 to master01, it prompts us to write to 192.168.0.43:6379. This is because the slot we write K1 to is 12706, which is on master03, so we can’t write to master01 normally. This also tells us that in order to use redis cluster, the client must be able to understand the redis protocol intelligently and return the information through the cluster, Schedule the corresponding request to the corresponding master for execution;

Verify read request

Tip: from the above information, we can make sure that the data of the three master nodes are completely isolated from each other, that is to say, master01 does not know whether there is corresponding data on master02, it only knows that it should go to master02 to check; this is what we said before, each node has all the status information (metadata) of the whole cluster, and each node only has the status information (metadata) There are some data in this node;

Verification: take down any master node to see if the corresponding slave can be promoted to master and take over the master?

Tip: you can see that when master01 goes down, the corresponding slave is promoted to master;

Check whether the configuration file of master01 has changed?

[[email protected] ~]# cat /usr/local/redis/redis-cluster_6379.conf 
91169e71359deed96f8778cf31c823dbd6ded350 192.168.0.43:[email protected] master - 0 1596648345000 8 connected 5461-10922
dbfff4c49a94c0ee55d14401ccc9245af3655427 192.168.0.42:[email protected] slave 8c785e6ec3f8f7ff4fb7768765da8b8a93f26855 0 1596648345510 9 connected
62ece0b80b83c0f1f078b07fc1687bb8376f76b3 192.168.0.41:[email protected] slave a7ace08c36f7d55c4f28463d72865aa1ff74829e 0 1596648343893 4 connected
e99b0b450e78719d63520cb6efc068d5e8d4d081 192.168.0.42:[email protected] slave 91169e71359deed96f8778cf31c823dbd6ded350 0 1596648344903 8 connected
8c785e6ec3f8f7ff4fb7768765da8b8a93f26855 192.168.0.41:[email protected] myself,master - 0 1596648345000 9 connected 0-5460
a7ace08c36f7d55c4f28463d72865aa1ff74829e 192.168.0.43:[email protected] master - 0 1596648345000 3 connected 10923-16383
vars currentEpoch 9 lastVoteEpoch 0
[[email protected] ~]# ps -ef |grep redis
root      15980      1  0 Aug05 ?        00:00:17 redis-server 0.0.0.0:6380 [cluster]
root      30867      1  0 01:19 ?        00:00:02 redis-server 0.0.0.0:6379 [cluster]
root      30949  14690  0 01:30 pts/0    00:00:00 grep --color=auto redis
[[email protected] ~]# redis-cli -a admin
127.0.0.1:6379> info replication
# Replication
role:master
connected_slaves:1
slave0:ip=192.168.0.42,port=6380,state=online,offset=4476,lag=0
master_replid:ed690ba2766e6e334ffc298fc6aba8be80465aa8
master_replid2:776b5f996d78171e7c5ab5419d65c6c14a253bda
master_repl_offset:4476
second_repl_offset:4071
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:3595
repl_backlog_histlen:882
127.0.0.1:6379> quit
[[email protected] ~]# kill -9 30867
[[email protected] ~]# redis-cli -h 192.168.0.42 -p 6380
192.168.0.42:6380> AUTH admin
OK
192.168.0.42:6380> info replication
# Replication
role:master
connected_slaves:0
master_replid:d46c0a3761b61352b1fc75986ab3343e04a89b83
master_replid2:ed690ba2766e6e334ffc298fc6aba8be80465aa8
master_repl_offset:4518
second_repl_offset:4519
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:4071
repl_backlog_histlen:448
192.168.0.42:6380> quit
[[email protected] ~]# cat /usr/local/redis/redis-cluster_6379.conf
91169e71359deed96f8778cf31c823dbd6ded350 192.168.0.43:[email protected] master - 0 1596648345000 8 connected 5461-10922
dbfff4c49a94c0ee55d14401ccc9245af3655427 192.168.0.42:[email protected] slave 8c785e6ec3f8f7ff4fb7768765da8b8a93f26855 0 1596648345510 9 connected
62ece0b80b83c0f1f078b07fc1687bb8376f76b3 192.168.0.41:[email protected] slave a7ace08c36f7d55c4f28463d72865aa1ff74829e 0 1596648343893 4 connected
e99b0b450e78719d63520cb6efc068d5e8d4d081 192.168.0.42:[email protected] slave 91169e71359deed96f8778cf31c823dbd6ded350 0 1596648344903 8 connected
8c785e6ec3f8f7ff4fb7768765da8b8a93f26855 192.168.0.41:[email protected] myself,master - 0 1596648345000 9 connected 0-5460
a7ace08c36f7d55c4f28463d72865aa1ff74829e 192.168.0.43:[email protected] master - 0 1596648345000 3 connected 10923-16383
vars currentEpoch 9 lastVoteEpoch 0
[[email protected] ~]# grep slaveof /usr/local/redis/6379/etc/redis.conf 
# Master-Slave replication. Use slaveof to make a Redis instance a copy of
# slaveof  
[[email protected] ~]#

Tip: from the above information, it can be seen that the corresponding configuration information has not changed before and after master01 is down and slave is promoted to master. In order to verify whether the configuration file has changed, I first put master01 online, then offline the corresponding slave, and restore master01 to master role;

Put the master online again and see what changes have taken place in its configuration file? Will the role still be master?

[[email protected] ~]# redis-server /usr/local/redis/6379/etc/redis.conf     
[[email protected] ~]# ss -tnl
State       Recv-Q Send-Q                                        Local Address:Port                                                       Peer Address:Port              
LISTEN      0      511                                                       *:6379                                                                  *:*                  
LISTEN      0      511                                                       *:6380                                                                  *:*                  
LISTEN      0      128                                                       *:22                                                                    *:*                  
LISTEN      0      100                                               127.0.0.1:25                                                                    *:*                  
LISTEN      0      511                                                       *:16379                                                                 *:*                  
LISTEN      0      511                                                       *:16380                                                                 *:*                  
LISTEN      0      128                                                      :::22                                                                   :::*                  
LISTEN      0      100                                                     ::1:25                                                                   :::*                  
[[email protected] ~]# cat /usr/local/redis/redis-cluster_6379.conf 
91169e71359deed96f8778cf31c823dbd6ded350 192.168.0.43:[email protected] master - 1596649311329 1596649311326 8 connected 5461-10922
8c785e6ec3f8f7ff4fb7768765da8b8a93f26855 192.168.0.41:[email protected] myself,slave dbfff4c49a94c0ee55d14401ccc9245af3655427 0 1596649311326 9 connected
e99b0b450e78719d63520cb6efc068d5e8d4d081 192.168.0.42:[email protected] slave 91169e71359deed96f8778cf31c823dbd6ded350 0 1596649311331 8 connected
a7ace08c36f7d55c4f28463d72865aa1ff74829e 192.168.0.43:[email protected] master - 0 1596649311332 3 connected 10923-16383
62ece0b80b83c0f1f078b07fc1687bb8376f76b3 192.168.0.41:[email protected] slave a7ace08c36f7d55c4f28463d72865aa1ff74829e 0 1596649311331 4 connected
dbfff4c49a94c0ee55d14401ccc9245af3655427 192.168.0.42:[email protected] master - 0 1596649311332 10 connected 0-5460
vars currentEpoch 10 lastVoteEpoch 0
[[email protected] ~]# redis-cli 
127.0.0.1:6379> AUTH admin
OK
127.0.0.1:6379> info replication
# Replication
role:slave
master_host:192.168.0.42
master_port:6380
master_link_status:up
master_last_io_seconds_ago:5
master_sync_in_progress:0
slave_repl_offset:4686
slave_priority:100
slave_read_only:1
connected_slaves:0
master_replid:d46c0a3761b61352b1fc75986ab3343e04a89b83
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:4686
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:4519
repl_backlog_histlen:168
127.0.0.1:6379> 

Tip: as you can see, after putting master01 online again, it will be automatically downgraded to slave, and the configuration file needs to be changed from master to slave. This should be because the redis cluster triggers to synchronize the configuration in memory to the corresponding file after it detects 192.168.0.41:6379 is online, so that the corresponding configuration file is also modified. At this point, the test of a redis cluster is completed;

Recommended Today

Socket communication between Java and python

Code service code, call the corresponding service according to the code Status code, return the status code according to the execution result Using JSON string communication between sockets Java socket client and python socket server Java socket client package com.brewin.hammer.system.util.socket; import java.io.InputStream; import java.io.OutputStream; import java.io.PrintWriter; import java.net.Socket; import java.util.HashMap; import java.util.Map; import com.alibaba.fastjson.JSON; /** […]