Reactive spring: responsive redis interaction

Time:2021-2-22

This article shares how to implement redis responsive interaction mode in spring.

This paper will simulate a user service and use redis as the data storage server.
This article involves two Java beans, users and interests

public class User {
    private long id;
    private String name;
    //Label
    private String label;
    //Receiving address longitude
    private Double deliveryAddressLon;
    //Receiving address dimension
    private Double deliveryAddressLat;
    //Latest sign in date
    private String lastSigninDay;
    //Integral
    private Integer score;
    //Rights and interests
    private List<Rights> rights;
    ...
}

public class Rights {
    private Long id;
    private Long userId;
    private String name;
    ...
}

start-up

Introduce dependency

    <dependency>
      <groupId>org.springframework.boot</groupId>
      <artifactId>spring-boot-starter-data-redis-reactive</artifactId>
    </dependency>

Add redis configuration

spring.redis.host=192.168.56.102
spring.redis.port=6379
spring.redis.password=
spring.redis.timeout=5000

Springboot startup

@SpringBootApplication
public class UserServiceReactive {
    public static void main(String[] args) {
        new SpringApplicationBuilder(
                UserServiceReactive.class)
                .web(WebApplicationType.REACTIVE).run(args);
    }
}

After the application starts, spring will automatically generate a reactive redistemplate (its underlying framework is lettuce).
Reactive redistemplate is similar to redistemplate, but it provides asynchronous and responsive redis interaction.
Again, responsive programming is asynchronous. After the reactive redistemplate sends a redis request, it will not block the thread. The current thread can perform other tasks.
After the redis response data is returned, the reactive redistemplate dispatches the thread to process the response data.
Responsive programming can achieve asynchronous calls and process asynchronous results in an elegant way, which is its greatest significance.

serialize

Reactive redistemplate uses JDK serialization by default, which can be configured as JSON serialization

@Bean
public RedisSerializationContext redisSerializationContext() {
    RedisSerializationContext.RedisSerializationContextBuilder builder = RedisSerializationContext.newSerializationContext();
    builder.key(StringRedisSerializer.UTF_8);
    builder.value(RedisSerializer.json());
    builder.hashKey(StringRedisSerializer.UTF_8);
    builder.hashValue(StringRedisSerializer.UTF_8);

    return builder.build();
}

@Bean
public ReactiveRedisTemplate reactiveRedisTemplate(ReactiveRedisConnectionFactory connectionFactory) {
    RedisSerializationContext serializationContext = redisSerializationContext();
    ReactiveRedisTemplate reactiveRedisTemplate = new ReactiveRedisTemplate(connectionFactory,serializationContext);
    return reactiveRedisTemplate;
}

builder.hashValue Method specifies the serialization method of redis list values. Since redis list values in this article only store strings, they are still set to s tringRedisSerializer.UTF_ 8。

Basic data type

Reactive redistemplate supports redis string, hash, list, set, ordered set and other basic data types.
In this paper, hash is used to save user information, list is used to save user rights, and other basic data types are not used in this paper.

public Mono<Boolean>  save(User user) {
    ReactiveHashOperations<String, String, String> opsForHash = redisTemplate.opsForHash();
    Mono<Boolean>  userRs = opsForHash.putAll("user:" + user.getId(), beanToMap(user));
    if(user.getRights() != null) {
        ReactiveListOperations<String, Rights> opsForRights = redisTemplate.opsForList();
        opsForRights.leftPushAll("user:rights:" + user.getId(), user.getRights()).subscribe(l -> {
            logger.info("add rights:{}", l);
        });
    }
    return userRs;
}

The beantomap method is responsible for converting the user class into a map.

HyperLogLog

Redis hyperlog structure can count the number of different elements in a collection.
Use hyperlog to count the number of users logged in every day

public Mono<Long>  login(User user) {
    ReactiveHyperLogLogOperations<String, Long> opsForHyperLogLog = redisTemplate.opsForHyperLogLog();
    return opsForHyperLogLog.add("user:login:number:" + LocalDateTime.now().toString().substring(0, 10), user.getId());
}

BitMap

Redis bitmap represents the value or state of an element through a bit. Because bit is the smallest unit of computer storage, it will save space.
Use bitmap to record whether the user has signed in this week

public void addSignInFlag(long userId) {
    String key = "user:signIn:" + LocalDateTime.now().getDayOfYear()/7 + (userId >> 16);
    redisTemplate.opsForValue().setBit(
            key, userId & 0xffff , true)
    .subscribe(b -> logger.info("set:{},result:{}", key, b));
}

The upper 48 bits of userid are used to divide users into different keys, and the lower 16 bits are used as offset parameter of bitmap.
The offset parameter must be greater than or equal to 0 and less than 2 ^ 32 (bit mapping is limited to 512 MB).

Geo

Redis geo can store and calculate geographic location information.
For example, find the warehouse information within a given range

public Flux getWarehouseInDist(User u, double dist) {
    ReactiveGeoOperations<String, String> geo = redisTemplate.opsForGeo();
    Circle circle = new Circle(new Point(u.getDeliveryAddressLon(), u.getDeliveryAddressLat()), dist);
    RedisGeoCommands.GeoRadiusCommandArgs args =
            RedisGeoCommands.GeoRadiusCommandArgs.newGeoRadiusArgs().includeDistance().sortAscending();
    return geo.radius("warehouse:address", circle, args);
}

warehouse:addressThis collection needs to save the warehouse location information.
The reactivegeooperations # radius method can find the elements in the collection whose geographical location is within a given range. It also supports the operations of adding elements to the collection and calculating the geographical distance between two elements in the collection.

Lua

Reactive redistemplate can also execute Lua scripts.
The user sign in logic is completed by Lua script: if the user does not sign in today, it is allowed to sign in, and the score is increased by 1. If the user has signed in today, it is refused.

public Flux<String> addScore(long userId) {
    DefaultRedisScript<String> script = new DefaultRedisScript<>();
    script.setScriptSource(new ResourceScriptSource(new ClassPathResource("/signin.lua")));
    List<String> keys = new ArrayList<>();
    keys.add(String.valueOf(userId));
    keys.add(LocalDateTime.now().toString().substring(0, 10));
    return redisTemplate.execute(script, keys);
}

signin.lua The content is as follows

local score=redis.call('hget','user:'..KEYS[1],'score')
local day=redis.call('hget','user:'..KEYS[1],'lastSigninDay')
if(day==KEYS[2])
    then
    return '0'
else
    redis.call('hset','user:'..KEYS[1],'score', score+1,'lastSigninDay',KEYS[2])
    return '1'
end

Stream

Redis stream is a new data type added in redis 5.0. This type can implement message queue, and provide message persistence and active / standby replication functions. It can also remember the access location of each client and ensure that messages are not lost.

Redis uses Kafka’s design for reference. There can be multiple consumption groups in a stream and multiple consumers in a consumption group.
If a consumer in a consumption group consumes a message in the stream, the message will not be consumed by other consumers in the consumption group. Of course, it can also be consumed by a consumer in other consumption groups.

The following defines a stream consumer, which is responsible for processing the received equity data

@Component
public class RightsStreamConsumer implements ApplicationRunner, DisposableBean {
    private static final Logger logger = LoggerFactory.getLogger(RightsStreamConsumer.class);

    @Autowired
    private RedisConnectionFactory redisConnectionFactory;

    private StreamMessageListenerContainer<String, ObjectRecord<String, Rights>> container;
    //Stream queue
    private static final String STREAM_KEY = "stream:user:rights";
    //Consumption group
    private static final String STREAM_GROUP = "user-service";
    //Consumers
    private static final String STREAM_CONSUMER = "consumer-1";

    @Autowired
    @Qualifier("reactiveRedisTemplate")
    private ReactiveRedisTemplate redisTemplate;

    public void run(ApplicationArguments args) throws Exception {

        StreamMessageListenerContainer.StreamMessageListenerContainerOptions<String, ObjectRecord<String, Rights>> options =
                StreamMessageListenerContainer.StreamMessageListenerContainerOptions.builder()
                        . batchsize (100) // the maximum number of counts pulled in a batch
                        .executor( Executors.newSingleThreadExecutor ()) // thread pool
                        .pollTimeout( Duration.ZERO )// blocking polling
                        .targetType( Rights.class )// target type (type of message content)
                        .build();
        //Create a message listening container
        container = StreamMessageListenerContainer.create(redisConnectionFactory, options);

        //Preparestreamandgroup finds the stream information. If it does not exist, it creates a stream
        prepareStreamAndGroup(redisTemplate.opsForStream(), STREAM_KEY , STREAM_GROUP)
                .subscribe(stream -> {
            //Create a consumer for the stream and bind the processing class
            container.receive(Consumer.from(STREAM_GROUP, STREAM_CONSUMER),
                    StreamOffset.create(STREAM_KEY, ReadOffset.lastConsumed()),
                    new StreamMessageListener());
            container.start();
        });
    }

    @Override
    public void destroy() throws Exception {
        container.stop();
    }

    //Find the stream information, if it does not exist, create the stream
    private Mono<StreamInfo.XInfoStream> prepareStreamAndGroup(ReactiveStreamOperations<String, ?, ?> ops, String stream, String group) {
        //The info method queries the stream information. If the stream does not exist, the underlying layer will report an error, and the onerrorresume method will be called.
        return ops.info(stream).onErrorResume(err -> {
            logger.warn("query stream err:{}", err.getMessage());
            //The creategroup method creates a stream
            return ops.createGroup(stream, group).flatMap(s -> ops.info(stream));
        });
    }

    //Message processing object
    class  StreamMessageListener implements StreamListener<String, ObjectRecord<String, Rights>> {
        public void onMessage(ObjectRecord<String, Rights> message) {
            //Processing messages
            RecordId id = message.getId();
            Rights rights = message.getValue();
            logger.info("receive id:{},rights:{}", id, rights);
            redisTemplate.opsForList().leftPush("user:rights:" + rights.getUserId(), rights).subscribe(l -> {
                logger.info("add rights:{}", l);
            });
        }
    }
}

Here’s how to send a message

public Mono<RecordId> addRights(Rights r) {
    String streamKey = "stream:user:rights";//stream key
    ObjectRecord<String, Rights> record = ObjectRecord.create(streamKey, r);
    Mono<RecordId> mono = redisTemplate.opsForStream().add(record);
    return mono;
}

Create a message record object objectrecord, and send the message record through reactive stream operations.

Sentinel、Cluster

Reactive redistemplate also supports redis sentinel and cluster modes. You only need to adjust the configuration.
Sentinel is configured as follows

spring.redis.sentinel.master=mymaster
spring.redis.sentinel.nodes=172.17.0.4:26379,172.17.0.5:26379,172.17.0.6:26379
spring.redis.sentinel.password=

spring.redis.sentinel.nodesThe configuration is sentinel node IP address and port, not redis instance node IP address and port.

The cluster configuration is as follows

spring.redis.cluster.nodes=172.17.0.2:6379,172.17.0.3:6379,172.17.0.4:6379,172.17.0.5:6379,172.17.0.6:6379,172.17.0.7:6379
spring.redis.lettuce.cluster.refresh.period=10000
spring.redis.lettuce.cluster.refresh.adaptive=true

For example, node2 in redis cluster is the slave node of node1, and this information will be cached in lettuce. When node1 goes down, redis cluster will upgrade node2 to the master node. But lettuce does not automatically switch the request to node2 because its buffer is not refreshed.
openspring.redis.lettuce.cluster.refresh.adaptiveConfiguration, lettuce can refresh the cluster cache information of redis cluster regularly, dynamically change the node condition of the client, and complete the fail over.

There is no solution for reactivereditemplate to implement pipeline and transaction.

Official document: https://docs.spring.io/spring… :reactive
Article complete code: https://gitee.com/binecy/bin-…

If you think this article is good, please pay attention to my WeChat official account, and the series articles are continuously updated. Your attention is the driving force of my persistence!
Reactive spring: responsive redis interaction