3. How to design the check-in system of ten million days?

Time:2022-5-8

Suppose there is a million check-in system, which records the user‘s check-in record, signed record 1 and not signed record 0. If we use redis string storage, 1000000 * 365 keys will be saved a year, which will occupy a lot of memory.

In order to solve this problem, redis provides a bitmap data structure, so that the daily check-in record occupies only one bit, 365 days is 365 bits, and 46 bytes (a slightly longer string) can be fully accommodated, which greatly saves storage space.

Bitmap stores continuous binary numbers (0 and 1). Through bitmap, only one bit is needed to represent the corresponding value or state of an element, and the key is the corresponding element itself. We know that eight bits can form a byte, so bitmap itself will greatly save storage space.

1. Common commands

byte[] bytes = "hzy".getBytes();
for (byte b : bytes) {
 System.out.println(Integer.toBinaryString(b));
}

results of enforcement
1101000
1111010
1111001

We use the bitmap to set a string “HZY”. The basic command is “setbit key [offset] [value]”. We first get the binary data corresponding to the ascll Code: 0110100 01111001

3. How to design the check-in system of ten million days?

picture
##Zero deposit and lump sum withdrawal
127.0.0.1:6379> setbit test 1 1
(integer) 0
127.0.0.1:6379> setbit test 2 1
(integer) 0
127.0.0.1:6379> setbit test 4 1
(integer) 0
127.0.0.1:6379> get test
"h"

127.0.0.1:6379> setbit test 9 1
(integer) 0
127.0.0.1:6379> setbit test 10 1
(integer) 0
127.0.0.1:6379> setbit test 11 1
(integer) 0
127.0.0.1:6379> setbit test 12 1
(integer) 0
127.0.0.1:6379> setbit test 14 1
(integer) 0
127.0.0.1:6379> get test
"hz"

127.0.0.1:6379> setbit test 17 1
(integer) 0
127.0.0.1:6379> setbit test 18 1
(integer) 0
127.0.0.1:6379> setbit test 19 1
(integer) 0
127.0.0.1:6379> setbit test 20 1
(integer) 0
127.0.0.1:6379> setbit test 23 1
(integer) 0
127.0.0.1:6379> get test
"hzy"

##Zero storage and zero retrieval
127.0.0.1:6379> getbit test 1
(integer) 1

##Whole storage and zero retrieval
127.0.0.1:6379> set test2 h
OK
127.0.0.1:6379> getbit test2 1
(integer) 1

2.bitcount&bitpos

We can count the total number of days users have signed in through bitcount. Use the bitpos instruction to find out the date on which the user first signed in. If the range parameter [start, end] is specified, you can count the number of days the user signed in within a certain time range. The day after which the user starts signing in.

However, it should be noted that the start and end parameters are byte indexes, that is, the specified bit range must be a multiple of 8 and cannot be specified arbitrarily. Because of this design, we cannot directly calculate how many days the user has checked in in a month. Instead, we must take out all the byte contents covered in this month (getrange can take out the substring of the string) and then make statistics in memory. This is very cumbersome.

Bitcount command, bitcount key [start] [End]

127.0.0.1:6379> set name hzy
OK
127.0.0.1:6379> bitcount name
(integer) 13
127.0.0.1:6379 > bitcount name 0 0 # the number of 1 in the first character
(integer) 3
127.0.0.1:6379 > bitcount name 0 1 # the number of 1 in the first two characters
(integer) 8

Bitpos command, bitpos key bit [start] [End]

Binary data corresponding to ascll code corresponding to HZY: 0110100 01111001

127.0.0.1:6379 > bitpos name 0 # the subscript of the first 0
(integer) 0
127.0.0.1:6379 > bitpos name 1 # the subscript of the first 1
(integer) 1
127.0.0.1:6379 > bitpos name 1 2 10 # from the third character, the subscript of the first 1 (that is, Hz does not count, only y participates)
(integer) 17

3.bitfield

We set (setbit) and get (getbit) means that the value of the location is a single bit. If you want to operate multiple bits at a time, you must use the pipeline to process it. However, version 3.2 of redis provides the bitfield instruction.

bitfield key [GET type offset] [SET type offset value] [INCRBY type offset increment] [OVERFLOW WRAP|SAT|FAIL]

##The simulated user signs in on July 15, 2021, and the offset starts from 0
127.0.0.1:6379> setbit userid:sign:202107 14 1
(integer) 0
##The simulated user signs in on July 16, 2021, and the offset starts from 0
127.0.0.1:6379> setbit userid:sign:202107 15 1
(integer) 0
##The simulated user signs in on July 31, 2021, and the offset starts from 0
127.0.0.1:6379> setbit userid:sign:202107 30 1
(integer) 0
##Get the user's check-in data in July 2021
127.0.0.1:6379> bitfield userid:sign:202107 get u31 0
1) (integer) 98305
//This is pseudo code. I wrote the key. In a real scenario, userid is the user's ID
public static void main(String[] args) {
 LocalDate date = LocalDate.now();
 Map<String, Boolean> signMap = new TreeMap<>();
 List<Long> list = jedis.bitfield("userid:sign:202107", "GET", String.format("u%d", date.lengthOfMonth()), "0");
 if (list != null && list.size() > 0) {
  //From low to high, 0 means not signed, and 1 means signed
  long v = list.get(0) == null ? 0 : list.get(0);
  for (int i = date.lengthOfMonth(); i > 0; i--) {
   LocalDate d = date.withDayOfMonth(i);
   signMap.put(formatDate(d, "yyyy-MM-dd"), v >> 1 << 1 != v);
   v >>= 1;
  }
 }
 System.out.println(signMap.toString());
}
private static String formatDate(LocalDate date, String pattern) {
 return date.format(DateTimeFormatter.ofPattern(pattern));
}
//Execution results
{2021-07-01=false, 2021-07-02=false, 2021-07-03=false, 
2021-07-04=false, 2021-07-05=false, 2021-07-06=false, 
2021-07-07=false, 2021-07-08=false, 2021-07-09=false, 
2021-07-10=false, 2021-07-11=false, 2021-07-12=false, 
2021-07-13=false, 2021-07-14=false, 2021-07-15=true, 
2021-07-16=true, 2021-07-17=false, 2021-07-18=false, 
2021-07-19=false, 2021-07-20=false, 2021-07-21=false, 
2021-07-22=false, 2021-07-23=false, 2021-07-24=false, 
2021-07-25=false, 2021-07-26=false, 2021-07-27=false, 
2021-07-28=false, 2021-07-29=false, 2021-07-30=false, 
2021-07-31=true}

Note: I is a signed integer and u is an unsigned integer. For example, U8 is an 8-bit unsigned integer. The signed integer can obtain 64 bits at most and the unsigned integer can obtain 63 bits at most

We use the bitfield key set sub instruction to change the last string y to uppercase y, and the ASCII code of Y is 89

127.0.0.1:6379> set name hzy 
OK 
127.0.0.1:6379> bitfield name set u8 16 89 
1) (integer) 121 
127.0.0.1:6379> get name 
"hzY"