Handling method of redis connection timeout exception

Time:2020-10-18

0. Problem description

Using jedis to connect to redis for data query operation, the normal code operation is OK, but the following errors will be reported from time to time:

Exception in thread “main” redis.clients.jedis.exceptions.JedisConnectionException: java.net.SocketTimeoutException: Read timed out
 at redis.clients.util.RedisInputStream.ensureFill(RedisInputStream.java:202)
 at redis.clients.util.RedisInputStream.read(RedisInputStream.java:181)
 at redis.clients.jedis.Protocol.processBulkReply(Protocol.java:181)
 at redis.clients.jedis.Protocol.process(Protocol.java:155)
 at redis.clients.jedis.Protocol.processMultiBulkReply(Protocol.java:206)
 at redis.clients.jedis.Protocol.process(Protocol.java:157)
 at redis.clients.jedis.Protocol.processMultiBulkReply(Protocol.java:206)
 at redis.clients.jedis.Protocol.process(Protocol.java:157)
 at redis.clients.jedis.Protocol.read(Protocol.java:215)
 at redis.clients.jedis.Connection.readProtocolWithCheckingBroken(Connection.java:340)
 at redis.clients.jedis.Connection.getRawObjectMultiBulkReply(Connection.java:285)
 at redis.clients.jedis.Connection.getObjectMultiBulkReply(Connection.java:291)
 at redis.clients.jedis.BinaryJedis.hscan(BinaryJedis.java:3390)
 at com.ict.mcg.filter.DuplicateClueFilterV2.hscan(DuplicateClueFilterV2.java:867)
 at com.ict.mcg.filter.DuplicateClueFilterV2.collectRecentCluekeywords(DuplicateClueFilterV2.java:487)
 at com.ict.mcg.main.GetCluesMain.run_online(GetCluesMain.java:208)
 at com.ict.mcg.main.GetCluesMain.main(GetCluesMain.java:1685)
Caused by: java.net.SocketTimeoutException: Read timed out
 at java.net.SocketInputStream.socketRead0(Native Method)
 at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
 at java.net.SocketInputStream.read(SocketInputStream.java:171)
 at java.net.SocketInputStream.read(SocketInputStream.java:141)
 at java.net.SocketInputStream.read(SocketInputStream.java:127)
 at redis.clients.util.RedisInputStream.ensureFill(RedisInputStream.java:196)
 … 16 more

The reason can be determined as follows java.net.SocketTimeoutException : read timed out, that is, the network connection is abnormal;

1. Possible causes

1.1 high utilization of server resources including memory, disk, CPU, etc

After checking the status information of the redis deployment machine, it is found that the overall machine is running well

1.2 firewall is set on the server, resulting in connection failure

Because the normal code flow can run through, there is no problem with firewall settings;

1.3 the bind monitoring host of redis configuration file is not configured properly

The configuration of host corresponding to bind in redis configuration file is as follows:


# By default Redis listens for connections from all the network interfaces
# available on the server. It is possible to listen to just one or multiple
# interfaces using the "bind" configuration directive, followed by one or
# more IP addresses.
#
# Examples:
#
# bind 192.168.1.100 10.0.0.1
# bind 127.0.0.1

The default bind host of bind is 0.0.0.0, which means that you can listen to every available network interface


bind 0.0.0.0

Our configuration file is also configured normally, and the normal code flow is running normally, which can also prove this point;

1.4 use and configuration of jedis

At present, the connection pool configuration of jedis is as follows:

private static JedisPool getPool() {
    if (pool == null) {
      JedisPoolConfig config = new JedisPoolConfig();
      //To control how many jedis instances can be allocated to a pool by pool.getResource () to obtain;
      //If the value is - 1, it means there is no restriction; if the pool has already allocated maxactive jedis instances, then the status of the pool is exhausted.
      config.setMaxActive(10);
      //Controls the maximum number of jedis instances with idle status in a pool.
      config.setMaxIdle(2);
      //Represents the maximum waiting time when a jedis instance is introduced. If the waiting time is exceeded, the jedisconnectionexception will be thrown directly;
      config.setMaxWait(1000 * 200000);
      //Whether to perform the validate operation in advance when borrow a jedis instance; if it is true, all the jedis instances obtained are available;
      config.setTestOnBorrow(true);
      config.setTestOnReturn(true);

      //Currently, redis has only one server
      pool = new JedisPool(config, "localhost", 6379);
    }
    return pool;
  }

  private static Jedis getJedis() {
    Jedis jedis = null;
    int count = 0;
    do {
      try {
        pool = getPool();
        jedis = pool.getResource();
      } catch(Exception e) {
//    System.out.println(e.getMessage());
        e.printStackTrace();
        pool.returnBrokenResource(jedis);
      }

      count++;
    } while (jedis==null && count < 3);

    return jedis;
  }

In the logic of building jedispool, only the config.setMaxWait (1000 * 200000);, which is the maximum waiting time for the introduction of a new jedis instance, and there is no other related connection timeout configuration. Therefore, check the source code of jedispool and find the following:


public JedisPool(final Config poolConfig, final String host) {
    this(poolConfig, host, Protocol.DEFAULT_PORT, Protocol.DEFAULT_TIMEOUT, null, Protocol.DEFAULT_DATABASE);
  }

  public JedisPool(String host, int port) {
    this(new Config(), host, port, Protocol.DEFAULT_TIMEOUT, null, Protocol.DEFAULT_DATABASE);
  }

  public JedisPool(final String host) {
    this(host, Protocol.DEFAULT_PORT);
  }

  public JedisPool(final Config poolConfig, final String host, int port,
      int timeout, final String password) {
    this(poolConfig, host, port, timeout, password, Protocol.DEFAULT_DATABASE);
  }

  public JedisPool(final Config poolConfig, final String host, final int port) {
    this(poolConfig, host, port, Protocol.DEFAULT_TIMEOUT, null, Protocol.DEFAULT_DATABASE);
  }

  public JedisPool(final Config poolConfig, final String host, final int port, final int timeout) {
    this(poolConfig, host, port, timeout, null, Protocol.DEFAULT_DATABASE);
  }

  public JedisPool(final Config poolConfig, final String host, int port, int timeout, final String password,
          final int database) {
    super(poolConfig, new JedisFactory(host, port, timeout, password, database));
  }

As can be seen from the above code, jedispool has multiple overloaded constructors, and a timeout parameter needs to be passed in the constructor as the connection timeout time. If it is not passed, the Protocol.DEFAULT_ As the default timeout, timeout continues to trace the source code


public final class Protocol {

  public static final int DEFAULT_PORT = 6379;
  public static final int DEFAULT_TIMEOUT = 2000;
  public static final int DEFAULT_DATABASE = 0;

  public static final String CHARSET = "UTF-8";

  public static final byte DOLLAR_BYTE = '$';
  public static final byte ASTERISK_BYTE = '*';
  public static final byte PLUS_BYTE = '+';
  public static final byte MINUS_BYTE = '-';
  public static final byte COLON_BYTE = ':';

  private Protocol() {
 // this prevent the class from instantiation
  }

We can draw a conclusion that the default timeout time of the connection in the jedispool is 2 seconds. However, the jedispool constructor we call just adopts this configuration. As long as the connection is not successful for two seconds, the connection of redis will be disconnected and an error will be reported. This may happen when the database request concurrency is relatively large. Therefore, the following changes are made when creating the jedispool , pass in a large timeout:


pool = new JedisPool(config, ParamUtil.REDIS_ADDRESS[0], ParamUtil.REDIS_PORT, 1000 * 10);

2. Summary

Encounter problems or check more, look at the source code, see the configuration of the source code, carefully check the problem one by one!

This article about redis connection timeout exception handling is introduced here. For more information about redis connection timeout exception, please search the previous articles of developeppaer or continue to browse the related articles below. I hope you can support developpaer more in the future!