Understand redis, the core data type

Time:2021-7-25

String string

tring  Types are binary safe, i.e  string Can contain any data.

The ordinary string in redis adopts raw encoding, that is, the original encoding method. This encoding method will dynamically expand the capacity and reduce the overhead of frequent memory allocation by pre allocating redundant space in advance.

When the string length is less than 1MB, it is allocated according to twice the required length. If it exceeds 1MB, it is pre allocated according to the additional capacity of 1MB each time.

Redis  The numbers in are also stored as string type, but the encoding method is different from ordinary string. The numbers are encoded by integer, and the string content is directly set as the binary byte sequence of integer value.

Redis’s string type can be used when storing common strings, serialized objects, counters and other scenarios. The corresponding instructions for string data types include set, get, Mset, incr, decr, etc.

List list

list List is a fast two-way linked list, which stores a series of string values of string type

For conventional pop and push elements, the performance is very high, and the time complexity is O (1), because the list is directly appended or popped. However, for random insertion, random deletion and random range acquisition, the location needs to be determined by polling the list, and the performance is relatively low.

When operating the list, you can use lpush, lpop, rpush, rpop and lrange to perform routine queue in and out and range acquisition operations. In some special scenarios, you can also use lset and linsert to perform random insertion operations and lrem to delete specified elements; Finally, when consuming the message list, blpop and brpop can also be used for blocking acquisition, so that when there are no elements in the list temporarily, you can quietly wait for the insertion of new elements without additional continuous query.

Set set

Set is an unordered set of string type. The elements in set are unique, that is, duplicate elements will not appear in set. Collections in redis are generally implemented through dict hash table, so insert, delete and query elements can be directly located according to the element hash value, and the time complexity is O (1).

operation

  • sismember  The instruction determines whether an element exists in the set data structure corresponding to the key. If it exists, it returns 1, otherwise it returns 0;
  • sdiff  Instruction to perform subtraction on on a plurality of set sets;
  • sinter  Instructions execute intersection on multiple sets;
  • sunion  Instructions execute union on multiple sets;
  • spop  Command pops up a random element;
  • srandmember  Instruction returns one or more random elements.

In the social system, it can be used to store the list of friends you follow, to judge whether you follow, and to make friends’ recommendations. In addition, the uniqueness of set can be used to make accurate statistics on the source service and source IP of the service.

Sorted set

In an ordered collection, each element is associated with a score score score value of type double. The ordered set is sorted from small to large by this score value. In an ordered set, elements are not allowed to be repeated, but the score score value is allowed to be repeated.

operation

  • zscan  Instruction: get the elements in the ordered set in order;
  • zscore  Instruction: get the score value of the element;
  • zrangeInstruction: returns the elements within the specified score range through the specified score;
  • When the score value of an element changes, you can also add or subtract the score value of the element through the zincrby instruction.
  • adopt  zinterstore、zunionstore  The instruction takes the intersection and union of multiple ordered sets, and then saves the new ordered set into a new key. If there are duplicate elements, the scores of the duplicate elements are added, and then used as the score value of the element in the new set.

An ordered set can be used to count the ranking list and refresh the list in real time. It can also be used to record students’ scores, so as to easily obtain the list of students within a certain score range. It can also be used to add weight values to the system statistics, so as to display them on the dashboard in real time.

Hash hash

slightly

bitmap

A bitmap is a continuous string of binary numbers, and the underlying is actually based on  string  For encapsulated storage

Instruction operation by bit bit. The position of each bit in the bitmap is the offset. You can use setbit and bitfield to set each bit in the bitmap to 0 or 1, bitcount to count the number of bits set to 1 in the bitmap, and bitop to sum, or, XOR and other operations on multiple bitmaps.

Understand redis, the core data type

bitmap Bitmap is characterized by bit setting, summation, or and other operations, which are very efficient, and the storage cost is very low. If it is used to store object label attributes, one bit can store one label. You can use bitmap to save the login status of users in the last n days. Use 1 bit every day, and set 1 when logging in.

Personalized recommendation is very important in social applications. You can set a series of tags for news and feeds, such as military, entertainment, video, pictures, text, etc. use bitmap to store these tags and set 1 on the corresponding tag bit. For users, a similar method can be used to record multiple attributes of users, and multi-dimensional statistics can be easily carried out according to labels. Important instructions for bitmap bitmaps include: setbit, getbit, bitcount, bitfield, bitop, bitpos, etc.

Use experience

Statistics of user login: login within 1 2 3 5 days
bitmap : 1 1 1 0 1

Geo location

When storing a location point, first use  Geohash  The algorithm maps and codes the two-dimensional longitude and latitude of the location into a one-dimensional 52 bit integer value, takes the location name and longitude and latitude coding score as key value pairs, and stores them in the sorted set corresponding to the classification key.

When it is necessary to calculate the people near a location point a, first take the specified location a as the center point and the distance as the radius to calculate the range of 8 directions of geo hash, and then poll all location points within the azimuth range in turn. As long as the distance from these location points to the center position a is within the required distance range, it is the target location point. After polling all location points in the range, reorder to get all targets near location point a.

Using geoadd, add the location name (such as person, vehicle and store name) and the corresponding geographic location information to the specified location classification key;

Use geopos to easily query the location information of a name;

Use georadius to obtain all elements near the specified location and within the specified distance;

Redis geo geographical location uses geohash to convert a large number of two-dimensional longitude and latitude to one-dimensional integer values, which can facilitate geographical location query, distance measurement and range search. However, due to the large number of geographical locations, there may be a large number of elements under a geographical classification key. During geo design, it is necessary to plan in advance to avoid excessive expansion of a single key.

Redis’s geo location data structure has many application scenarios, such as querying the specific location of a place, checking the distance from the current location to the destination, and checking nearby people, restaurants, cinemas, etc. In geo geographic location data structure, important instructions include geoadd, geopos, geodist, georadius, georadius by member, etc.

Use geodist to get the distance between two specified positions.

Hyperlog cardinality statistics

hyperLogLog  It is a data type used for cardinality statistics. When a large number of elements are input for statistics, it only needs a small memory. Hyperlog does not save metadata, but only records the estimated quantity of elements to be counted. This estimated quantity is an approximate value with 0.81% standard deviation. In most business scenarios, for massive data, the error of less than 1% is acceptable.

In redis’s hyperlog statistics, if the number of counts is small, it is stored in a sparse matrix. With the increase of counts, the space occupied by the sparse matrix will gradually increase. When the threshold is exceeded, it will be changed to a dense matrix. The space occupied by the dense matrix is fixed, about 12KB bytes.

Through the hyperloglog data type, you can use pfadd to add new elements to the cardinality statistics. You can use pfcount to obtain the approximate cardinality number stored in the hyperloglog structure. You can also use hypermerge to merge multiple hyperloglogs into a hyperloglog structure, so that you can easily obtain the combined cardinality number.

The characteristic of hyperlog is that the statistical process does not record independent elements and occupies very little memory. It is very suitable for statistics of massive data. In large and medium-sized systems, the statistics of daily and monthly UV, that is, the number of independent visitors, or the number of independent entries searched by a large number of users can be processed with the hyperlog data type.

Recommended Today

Implementation example of go operation etcd

etcdIt is an open-source, distributed key value pair data storage system, which provides shared configuration, service registration and discovery. This paper mainly introduces the installation and use of etcd. Etcdetcd introduction etcdIt is an open source and highly available distributed key value storage system developed with go language, which can be used to configure sharing […]