An example of Spring Boot configuring Redis Sentinel

Time:2019-10-8

Original address: https://chanjarster.github.io…

This article introduces an example of Spring Book connecting Redis Sentinel.

The source code associated with this article: GitHub

Essential information

Topology (M stands for redis-master, S for redis-sentinel, R for redis-slave and C for Spring Boot Application):

     +---------------------------+
     |                           |
  +-----+       +-----+       +-----+
  |  M  |-------|  S  |-------|  R  |
  +-----+       +-----+       +-----+
     |             |
     |          +-----+
     +----------|  C  |
                +-----+

application.yamlTo configure:

spring:
  redis:
#    host: redis-master
#    port: 6379
    password: abc
    sentinel:
      master: springboot
      nodes:
        - redis-sentinel:26379

Note that there is no need to configure master’s host and port, and this information will be obtained from Redis Sentinel.

Demonstration steps

Package and build the image:mvn clean install dockerfile:build

Get intodockerDirectory, Executiondocker-compose up -d

Observe the log of Spring Boot Application:docker logs -f docker_spring-boot_1It will be found to be executed every three secondsINCR foo

07:53:49.205  INFO  hello.Application                    : INCR foo: 1
07:53:52.212  INFO  hello.Application                    : INCR foo: 2
07:53:55.213  INFO  hello.Application                    : INCR foo: 3
07:53:58.216  INFO  hello.Application                    : INCR foo: 4
07:54:01.217  INFO  hello.Application                    : INCR foo: 5

Stop redis-master:docker stop docker_redis-master_1You will see problems with the Edits link of Spring Boot Application:

07:54:37.206  INFO  hello.Application                    : INCR foo: 17
07:54:40.204  INFO  hello.Application                    : INCR foo: 18
07:54:42.238  INFO  i.l.core.protocol.ConnectionWatchdog : Reconnecting, last destination was /10.0.19.4:6379
07:54:52.247  WARN  i.l.core.protocol.ConnectionWatchdog : Cannot reconnect: io.netty.channel.ConnectTimeoutException: connection timed out: /10.0.19.4:6379
...
07:55:22.560  INFO  i.l.core.protocol.ConnectionWatchdog : Reconnecting, last destination was 10.0.19.4:6379
07:55:22.842  WARN  i.l.core.protocol.ConnectionWatchdog : Cannot reconnect: io.netty.channel.AbstractChannel$AnnotatedNoRouteToHostException: Host is unreachable: /10.0.19.4:6379
...
07:55:29.582  INFO i.l.core.protocol.ConnectionWatchdog  : Reconnecting, last destination was 10.0.19.4:6379
07:55:32.353  WARN i.l.core.protocol.ConnectionWatchdog  : Cannot reconnect: io.netty.channel.AbstractChannel$AnnotatedNoRouteToHostException: Host is unreachable: /10.0.19.4:6379
...

Waiting for about 60 seconds, Redis Sentinel intervened, promoted redis-slave to master, and resumed the link:

07:55:43.860  INFO i.l.core.protocol.ConnectionWatchdog  : Reconnecting, last destination was 10.0.19.4:6379
07:55:43.882  INFO i.l.core.protocol.ReconnectionHandler : Reconnected to 10.0.19.6:6379
07:55:43.887  INFO hello.Application                     : INCR foo: 20
07:55:43.889  INFO hello.Application                     : INCR foo: 21
07:55:43.891  INFO hello.Application                     : INCR foo: 22
07:55:43.892  INFO hello.Application                     : INCR foo: 23

At this point, the topology becomes like this:

     +-------------//------------+
     |                           |
  +-----+       +-----+       +-----+
  |  M  |--//---|  S  |-------| [M] |
  +-----+       +-----+       +-----+
     |             |             |
     |          +-----+          |
     +----//----|  C  |----------+
                +-----+

Clean containers:docker-compose down

Problems after Master Restart

This problem has nothing to do with Spring Boot, which is Redis itself. If we restart the previously stopped master, sentinel will not perceive the master because the master’s IP has changed (see comment):

You can watch the master’s INFO after reboot:

$ docker exec docker_redis-master_1 redis-cli -a abc INFO replication
# Replication
role:master
...

You can see that it’s master after it starts, not slave. In this way, there are two masters, which is a problem.

BTW, redis configuration can use hostname, such asslaveof redis-master。 But redis-sentinel, which uses ip, even if you configure hostname, is ultimately ip. Execute the following command to see sentinel’s configuration:

$ docker exec docker_redis-sentinel_1 cat /bitnami/redis-sentinel/conf/sentinel.conf | grep springboot

sentinel monitor springboot 10.0.25.2 6379 1
sentinel down-after-milliseconds springboot 60000
sentinel auth-pass springboot abc
sentinel config-epoch springboot 0
sentinel leader-epoch springboot 0
sentinel known-slave springboot 10.0.25.5 6379

Solution 1: Use host network

Use host network to deploy redis-master, redis-slave, and use<host-ip>:<container-port>To access them, because the host’s IP is relatively fixed, can alleviate this problem.

There is another limitation with host network: you cannot start two containers of the same container-port on the same host.

Solution 2: publish port

Publish the ports of redis-master and redis-slave to the host, in redis.config.slave-announce-ipandslave-announce-portSet it to host-ip and host-port, and finally use<host-ip>:<host-port>Access also solves this problem by utilizing the fixed IP characteristics of host.