Redis cluster environment configuration of springboot series

Time:2020-8-7

The previous posts on redis are all based on stand-alone redis. However, in the actual production environment, the possibility of using a redis cluster should be greater than that of a stand-alone version of redis. How to operate a cluster’s redis? What is the difference between its configuration and that of a single machine, and what should be noted?

This article mainly introduces the integration of redis cluster in springboot project, and explains the problems in the process, and gives the corresponding solutions

<!– more –>

1. Environment related

First of all, you need to install the redis cluster environment. Please refer to the blog post: redis cluster setup manual

Then initialize the springboot project, and the corresponding POM structure is as follows

<parent>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-parent</artifactId>
    <version>2.1.7</version>
    <relativePath/> <!-- lookup parent from update -->
</parent>

<properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
    <spring-cloud.version>Finchley.RELEASE</spring-cloud.version>
    <java.version>1.8</java.version>
</properties>

<dependencies>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-data-redis</artifactId>
    </dependency>
    <dependency>
        <groupId>org.apache.commons</groupId>
        <artifactId>commons-pool2</artifactId>
    </dependency>
</dependencies>

<build>
    <pluginManagement>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </pluginManagement>
</build>
<repositories>
    <repository>
        <id>spring-milestones</id>
        <name>Spring Milestones</name>
        <url>https://repo.spring.io/milestone</url>
        <snapshots>
            <enabled>false</enabled>
        </snapshots>
    </repository>
</repositories

It should be noted that we have introduced two packages, one of which is necessaryspring-boot-starter-data-redis, an official package of the start tool for operating redis. With it, we can easily use redistemplate to operate redis directly

The other iscommonos-pool2This package is mainly used when we configure the connection pool of redis, otherwise a class not found exception will be thrown

2. Environmental integration and mining pit

Here, I set up the redis cluster on a CentOS machine in the LAN. From the configuration file at the back, we can see that (why do we do this? It is mainly to lead to the latter question.)

1. application.yml

First, we will use the default configuration method to obtain ourRedisTemplateTo achieve the fastest access to the redis cluster

spring:
  redis:
    password:
    cluster:
      nodes: 192.168.0.203:7000,192.168.0.203:7001,192.168.0.203:7002
      max-redirects: 3
    lettuce:
      pool:
        max-idle: 16
        max-active: 32
        min-idle: 8

The redis cluster we built does not have a master / slave (otherwise, six instances are required). In order to save time, there is no password set (this is strictly prohibited in the production environment)

2. Use test

Because we use the default configuration, we can directly obtain the bean object of redistemplate to operate the redis cluster

@SpringBootApplication
public class Application {

    public Application(RedisTemplate redisTemplate) {
        redisTemplate.opsForValue().set("spring-r-cluster-1", 123);
        redisTemplate.opsForValue().set("spring-r-cluster-2", 456);
        redisTemplate.opsForValue().set("spring-r-cluster-3", 789);
    }

    public static void main(String[] args) {
        SpringApplication.run(Application.class);
    }
}

a. Deny connection

After the above execution, the first error reported is connection rejection. I can connect successfully on the machine (203) where the redis cluster is located, but the local connection error is reported

Redis cluster environment configuration of springboot series

There are generally two reasons for the above problems. One is that the port cannot be accessed externally due to the firewall, and the other is the configuration of redis

Confirmation method of firewall

  • To determine whether the firewall is on:firewall-cmd --stateIf the prompt “not running” indicates that it is not turned on
  • To view firewall rules:firewall-cmd --list-all

Then you can add ports according to the actual scene

#Permanently open public access to port 7000
sudo firewall-cmd --zone=public --add-port=7000/tcp --permanent
sudo firewall-cmd --reload

Of course, in the intranet test environment, you can directly close the firewall

//Disable firewall
systemctl disable firewalld
systemctl stop firewalld
systemctl status firewalld

//Enable firewall
systemctl enable firewalld
systemctl start firewalld
systemctl status firewalld

Redis configuration

If it is confirmed that it is not a firewall problem, the configuration of redis needs to be modifiedredis.confThere is a linebind 127.0.0.1The configuration is enabled by default, which means that only the local machine is allowed to access, and other machines have no access to it

The solution is to modify the configuration and restart it

bind 0.0.0.0

b. Unable to connect to 127.0.0.1:7001

When executing the previous test case, a strange exception was thrown as follows

Redis cluster environment configuration of springboot series

The key stack information is as follows

Caused by: org.springframework.data.redis.RedisSystemException: Redis exception; nested exception is io.lettuce.core.RedisException: io.lettuce.core.RedisConnectionException: Unable to connect to 127.0.0.1:7001
    at org.springframework.data.redis.connection.lettuce.LettuceExceptionConverter.convert(LettuceExceptionConverter.java:74) ~[spring-data-redis-2.0.9.RELEASE.jar:2.0.9.RELEASE]
    at org.springframework.data.redis.connection.lettuce.LettuceExceptionConverter.convert(LettuceExceptionConverter.java:41) ~[spring-data-redis-2.0.9.RELEASE.jar:2.0.9.RELEASE]
    at org.springframework.data.redis.PassThroughExceptionTranslationStrategy.translate(PassThroughExceptionTranslationStrategy.java:44) ~[spring-data-redis-2.0.9.RELEASE.jar:2.0.9.RELEASE]
    at org.springframework.data.redis.FallbackExceptionTranslationStrategy.translate(FallbackExceptionTranslationStrategy.java:42) ~[spring-data-redis-2.0.9.RELEASE.jar:2.0.9.RELEASE]
    at org.springframework.data.redis.connection.lettuce.LettuceConnection.convertLettuceAccessException(LettuceConnection.java:257) ~[spring-data-redis-2.0.9.RELEASE.jar:2.0.9.RELEASE]
    at org.springframework.data.redis.connection.lettuce.LettuceStringCommands.convertLettuceAccessException(LettuceStringCommands.java:718) ~[spring-data-redis-2.0.9.RELEASE.jar:2.0.9.RELEASE]
    at org.springframework.data.redis.connection.lettuce.LettuceStringCommands.set(LettuceStringCommands.java:143) ~[spring-data-redis-2.0.9.RELEASE.jar:2.0.9.RELEASE]
    at org.springframework.data.redis.connection.DefaultedRedisConnection.set(DefaultedRedisConnection.java:231) ~[spring-data-redis-2.0.9.RELEASE.jar:2.0.9.RELEASE]
    at org.springframework.data.redis.core.DefaultValueOperations$3.inRedis(DefaultValueOperations.java:202) ~[spring-data-redis-2.0.9.RELEASE.jar:2.0.9.RELEASE]
    at org.springframework.data.redis.core.AbstractOperations$ValueDeserializingRedisCallback.doInRedis(AbstractOperations.java:59) ~[spring-data-redis-2.0.9.RELEASE.jar:2.0.9.RELEASE]
    at org.springframework.data.redis.core.RedisTemplate.execute(RedisTemplate.java:224) ~[spring-data-redis-2.0.9.RELEASE.jar:2.0.9.RELEASE]
    at org.springframework.data.redis.core.RedisTemplate.execute(RedisTemplate.java:184) ~[spring-data-redis-2.0.9.RELEASE.jar:2.0.9.RELEASE]
    at org.springframework.data.redis.core.AbstractOperations.execute(AbstractOperations.java:95) ~[spring-data-redis-2.0.9.RELEASE.jar:2.0.9.RELEASE]
    at org.springframework.data.redis.core.DefaultValueOperations.set(DefaultValueOperations.java:198) ~[spring-data-redis-2.0.9.RELEASE.jar:2.0.9.RELEASE]
    at com.git.hui.boot.redis.cluster.Application.<init>(Application.java:14) [classes/:na]
    at com.git.hui.boot.redis.cluster.Application$$EnhancerBySpringCGLIB$$ac0c03ba.<init>(<generated>) ~[classes/:na]
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[na:1.8.0_171]
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[na:1.8.0_171]
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[na:1.8.0_171]
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[na:1.8.0_171]
    at org.springframework.beans.BeanUtils.instantiateClass(BeanUtils.java:170) ~[spring-beans-5.0.8.RELEASE.jar:5.0.8.RELEASE]
    ... 19 common frames omitted
Caused by: io.lettuce.core.RedisException: io.lettuce.core.RedisConnectionException: Unable to connect to 127.0.0.1:7001
    at io.lettuce.core.LettuceFutures.awaitOrCancel(LettuceFutures.java:125) ~[lettuce-core-5.0.4.RELEASE.jar:na]
    at io.lettuce.core.cluster.ClusterFutureSyncInvocationHandler.handleInvocation(ClusterFutureSyncInvocationHandler.java:118) ~[lettuce-core-5.0.4.RELEASE.jar:na]
    at io.lettuce.core.internal.AbstractInvocationHandler.invoke(AbstractInvocationHandler.java:80) ~[lettuce-core-5.0.4.RELEASE.jar:na]
    at com.sun.proxy.$Proxy44.set(Unknown Source) ~[na:na]
    at org.springframework.data.red

Through the breakpoint, we can see that the IP / port of the node in the cluster is accurate, but an exception indicates that one cannot be connected127.0.0.1:7001The main reason for this problem is that when creating a redis cluster, the following command is used to set the cluster nodes

redis/src/redis-cli --cluster create 127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002

There is no problem with the redis cluster created in the above way, but in the integration of springbot, the node information obtained through the redis cluster is127.0.0.1:7000… then causes the above problem, so one solution is to specify the IP address when creating the cluster

First data and configuration, and then re-establish the cluster relationship

#Delete data configuration
rm xxx/data/*

redis/src/redis-cli  --cluster create 192.168.0.203:7000 192.168.0.203:7001 192.168.0.203:7002

Then test OK again

Redis cluster environment configuration of springboot series

3. Jedis configuration

The previous configuration uses lettuce as the bridge tool of redis by default. If we want to use jedis at the bottom, what can we do?

First, add the jedis dependency to the POM dependency

<dependency>
    <groupId>redis.clients</groupId>
    <artifactId>jedis</artifactId>
</dependency>

Basically, the configuration in the YML file is OK. In the actual project, the connection pool is slightly changed, which does not affect reading. It is not posted here

Next is the definitionRedisConnectionFactoyTo replace the default

The following configuration is basically the same as the jedis configuration of redis in the 181101 springboot advanced article. It should be noted that we use theRedisClusterConfigurationReplacedRedisStandaloneConfiguration

@Configuration
public class RedisAutoConfig {

    @Bean
    public RedisConnectionFactory redisConnectionFactory(JedisPoolConfig jedisPool,
            RedisClusterConfiguration jedisConfig) {
        JedisConnectionFactory factory = new JedisConnectionFactory(jedisConfig, jedisPool);
        factory.afterPropertiesSet();
        return factory;
    }

    @Configuration
    public static class JedisConf {
        @Value("${spring.redis.cluster.nodes:127.0.0.1:7000,127.0.0.1:7001,127.0.0.1:7002}")
        private String nodes;
        @Value("${spring.redis.cluster.max-redirects:3}")
        private Integer maxRedirects;
        @Value("${spring.redis.password:}")
        private String password;
        @Value("${spring.redis.database:0}")
        private Integer database;

        @Value("${spring.redis.jedis.pool.max-active:8}")
        private Integer maxActive;
        @Value("${spring.redis.jedis.pool.max-idle:8}")
        private Integer maxIdle;
        @Value("${spring.redis.jedis.pool.max-wait:-1}")
        private Long maxWait;
        @Value("${spring.redis.jedis.pool.min-idle:0}")
        private Integer minIdle;

        @Bean
        public JedisPoolConfig jedisPool() {
            JedisPoolConfig jedisPoolConfig = new JedisPoolConfig();
            jedisPoolConfig.setMaxIdle(maxIdle);
            jedisPoolConfig.setMaxWaitMillis(maxWait);
            jedisPoolConfig.setMaxTotal(maxActive);
            jedisPoolConfig.setMinIdle(minIdle);
            return jedisPoolConfig;
        }

        @Bean
        public RedisClusterConfiguration jedisConfig() {
            RedisClusterConfiguration config = new RedisClusterConfiguration();

            String[] sub = nodes.split(",");
            List<RedisNode> nodeList = new ArrayList<>(sub.length);
            String[] tmp;
            for (String s : sub) {
                tmp = s.split(":");
                //Fixme does not consider the case of exception configuration
                nodeList.add(new RedisNode(tmp[0], Integer.valueOf(tmp[1])));
            }

            config.setClusterNodes(nodeList);
            config.setMaxRedirects(maxRedirects);
            config.setPassword(RedisPassword.of(password));
            return config;
        }
    }
}

Then the rest is the same. At this time, the underlying connection of redistemplate becomes jedis

Redis cluster environment configuration of springboot series

3. Others

0. Project & related blog

  • Project: https://github.com/liuyueyi/spring-boot-demo
  • moduel : https://github.com/liuyueyi/spring-boot-demo/tree/master/spring-boot/124-redis-cluster

Related blog

  • Redis cluster building manual
  • Redis & springboot basic configuration
  • Basic configuration of jedis & springboot

1. A gray blog

It’s not as good as a letter. The above contents are all from one family. Due to limited personal ability, there are inevitably omissions and mistakes. If you find a bug or have better suggestions, you are welcome to criticize and correct, and thank you

The following is a gray personal blog, recording all the blog articles in study and work. Welcome to visit

  • Personal blog https://blog.hhui.top
  • A grey blog spring blog http://spring.hhui.top

Redis cluster environment configuration of springboot series