Using Zset of redis to count online user information in springboot

Time:2021-6-3

Counting the number of online users is a common requirement for applications. If it is necessary to accurately count whether the user is online or offline, I think only the client and server can maintain a long TCP connection. If the application itself is not an IM application, the cost of this method is very high.

Today’s applications tend to useHeartbeat To identify whether the user is online. After the user logs in, a message is pushed to the server every other period of time to indicate that the current user is online. The server can define a time difference, for example:If a heartbeat message is received from the client within 5 minutes, it is regarded as an online user

Realization of online user statistics

Database based implementation

The simplest way is to add a date and time field of the last heartbeat package to the user tablelast_active. After receiving the heartbeat, the server updates this field to the latest time.

If you want toQuery the number of active users in the last 5 minutes, you can simply complete it with one sentence of SQL.

SELECT COUNT(1) AS `online_user_count` FROM `user` WHERE `last_active` BETWEEN  '2020-12-22 13:00:00' AND '020-12-22 13:05:00';

The disadvantages are also obvious. In order to improve the efficiency of retrieval, we have to pay attention to itlast_activeField to add index, and because of the heartbeat update, will lead to frequent re maintenance of the index tree, the efficiency is extremely low.

Implementation based on redis

This is an ideal way to achieve, redis based on memory to read and write, performance is naturally much better than relational database, and it providesZsetIt is very convenient to build a statistical service for online users.

Zset of redis

There won’t be much about redis here. Here’s a brief explanationzset. It is an orderly processsetSet, each element in the set consists of two things

  • Since member is a set, it is an element in the set and cannot be repeated
  • score   Since it is ordered, it is used for sortingweightfield

Partial operation of Zset

Add element

ZADD key score member [score member ...]

Add one or more elements to the collection at a time, ifmemberIf it already exists, the currentscore Make the overlay

Count all the elements

ZCARD key

Count the number of elements whose score value is between min and max

ZCOUNT key min max

Delete elements with score values between min and max

ZREMRANGEBYSCORE key min max

An example

I’m going to use onezsetStore the ranking of programming languages in my heart. This key is calledlang

Add information to return the number of newly added elements

> zadd lang 999 php 10 java 9 go 8 python 7 javascript
"5"

View the number of additions

> zcard lang
"5"

Look at the number of elements scored between 8 and 10, there are 3

> zcount lang 8 10
"3"

Delete the elements with a score of 8-1000, and return the number of deleted elements

> ZREMRANGEBYSCORE lang 8 1000
"4"

Realization of online user service

got itzsetAfter that, you can achieve aOnline usersOur statistical service is ready.

Realization idea

The client sends a heartbeat to the server every 5 minutes, and the server obtains the user’s ID according to the session, which is used as thezsetOfmember
DepositzsetscoreWhen the same user sends the heartbeat for the second time, it will update its corresponding time stampscoreSince the update is in memory, the speed is quite fast.

zadd users 1608616915109 10000

We need to count the number of online users. In essence, we need to count the number of users who have sent heartbeat in the last five minuteszcountIt can be easily calculated. Through the program to obtain the current timestamp, asmaxScore5 minutes from the timestampminScore

zcount users 1608616615109 1608616915109 

Because some users may not have logged in for a long time, they can log in through theZREMRANGEBYSCOREClean up. Get the current time stamp through the program, subtract 5 minutes as the time stampmaxScore, using0, asminScore, which means to clean up all users who have not sent heartbeat packets for more than 5 minutes.

ZREMRANGEBYSCORE users 0 1608616615109 

Implementation code

import java.time.Duration;
import java.time.Instant;
import java.time.LocalDateTime;
import java.time.ZoneId;

import javax.annotation.Resource;

import org.springframework.data.redis.core.StringRedisTemplate;
import org.springframework.stereotype.Component;

/**
 * 
 * 
 *Online user statistics
 * 
 * @author Administrator
 *
 */
@Component
public class OnlineUserStatsService {
    
    private static final String ONLINE_USERS = "onlie_users";

    @Resource
    private StringRedisTemplate stringRedisTemplate;

    /**
     *Add user online information
     * @param userId
     * @return 
     */
    public Boolean online(Integer userId) {
        return this.stringRedisTemplate.opsForZSet().add(ONLINE_USERS, userId.toString(), Instant.now().toEpochMilli());
    }
    
    /**
     *Get the number of online users in a certain period of time
     * @param duration
     * @return
     */
    public Long count(Duration duration) {
        LocalDateTime now = LocalDateTime.now();
        return this.stringRedisTemplate.opsForZSet().count(ONLINE_USERS, 
                                    now.minus(duration).atZone(ZoneId.systemDefault()).toInstant().toEpochMilli(), 
                                    now.atZone(ZoneId.systemDefault()).toInstant().toEpochMilli());
    }
    
    /**
     *Get the number of all online users, regardless of time
     * @return
     */
    public Long count() {
        return this.stringRedisTemplate.opsForZSet().zCard(ONLINE_USERS);
    }
    
    /**
     *Clear user data that has not been online for a certain period of time
     * @param duration
     * @return
     */
    public Long clear(Duration duration) {
        return this.stringRedisTemplate.opsForZSet().removeRangeByScore(ONLINE_USERS, 0, 
                LocalDateTime.now().minus(duration).atZone(ZoneId.systemDefault()).toInstant().toEpochMilli());
    }
}

Use examples

@Resource
private OnlineUserStatsService onlineUserStatsService;

@Test
public void test() {
    
    //The user with ID 1 sent a heartbeat packet
    boolean result = this.onlineUserStatsService.online(1);
    System.out.println("online=" + result);
    
    //Get the number of users who have sent heartbeat packets within 5 minutes, that is, the number of online users
    Long count = this.onlineUserStatsService.count(Duration.ofMinutes(5));
    System.out.println("oneline count=" + count);
    
    //Get the number of users who have sent heartbeat packets
    count = this.onlineUserStatsService.count();
    System.out.println("all count=" + count);
    
    //Clear users who have not sent heartbeat packets for more than one day
    Long clear = this.onlineUserStatsService.clear(Duration.ofDays(1));
    System.out.println("clear=" + clear);
}

Memory consumption analysis

Can pass http://www.redis.cn/redis_ Memory / budget redis memory consumption

I’m not familiar with redis’s memory allocation. I just filled in some data according to my own idea, so what I understand here may be wrong. But I don’t think it’s too late to prove – the fact that using Zset in this scenario consumes very little memory

imagineonlie_usersNeed to store the status information of 100 million users, each elementscoreandmemberIf you need 10 bytes of storage, you need about 20g of memory. 20g of memory is not a big problem for today’s servers.

Using Zset of redis to count online user information in springboot

last

  • Heartbeat protocol does not necessarily need HTTP, if the client supports UDP, it is very suitable, and can save some system overhead.
  • zsetYou don’t have to use your keyString, you can modify the serialization method to store the user ID in the form of fixed bytes. When the user ID is too large, you can save some storage space.
String userId = "10010";
System.out.println(userId.getBytes().length); //  Store as string = > 5 bytes required

byte[] bin = ByteBuffer.allocate(4).putInt(Integer.valueOf(userId)).array();
System.out.println(bin.length);                     //  Serialized as byte storage = > 4 bytes required

System.out.println(ByteBuffer.wrap(bin).getInt());     //  Deserialize to id = > 10010

First: https://springboot.io/t/topic/3157

Recommended Today

Mybatis best tutorial introductory notes

1、 Introduction to mybatis 1. Mybatis execution steps Read configuration filemybatis-config.xml, mainly to obtain database connection and running environment information Load the mapping file mapper XML, that is, the SQL mapping file, needs to be in mybatis config Only when loaded in XML can it be executed. Create sqlsessionfactory Create sqlsession from sqlsessionfactory Use the […]