Redis actual combat: geo helps me meet nearby goddesses

Time:2021-10-26

Yard old wet, read your bookSkillfully using data types to realize 100 million level data statisticsAfter that, I learned how to use different data types with ease(String、Hash、List、Set、Sorted Set、HyperLogLogBitmap)To solve the statistical problems of different scenarios.

The product manager said that he has an idea to provide the majority of boys and girls with an opportunity to connect with each other

Let the boys and girls at this most beautiful ageTwelve hourYou can meet that in theTa

So I want to develop oneApp, users can find the one nearby after logging inTa, connect to each other.

How can I achieve discoverypeople nearby? I also hope to pass thisApp Meet the goddess

In my memory, one night after work, she moved lightly from the crowd, and her tall and slim figure was like an elegant note floating in the space. Her eyes are full of clear sunshine and vitality, and the stars of the Milky way are printed in her eyes.

Opening message

Exercise your expression skills, especially at work. Many people say that “those who work are not as good as those who do PPT”. In fact, bosses are not stupid. Why do they recognize those who do PPT more?

Because they consider problems from the perspective of the boss, what he needs is a “solution”. Think more from the perspective of a creator, rather than limited to the perspective of a programmer;

Think more about what value this thing provides to people, rather than “how do I realize it”. Of course, how to implement it is necessary, but it is usually not the most important.

What is lbs oriented application

Longitude and latitude is a combination of longitude and latitudeCoordinate system。 Also known as geographic coordinate system, it is a spherical coordinate system that uses the sphere of three-dimensional space to define the space on the earth, which can mark the space on the earthAny location(7 digits after the decimal point, the accuracy can be up to 1 cm).

The range of longitude is (- 180, 180), the range of latitude is (- 90, 90), the positive and negative of latitude is bounded by the equator, the positive and negative of North and south, and the positive and negative of longitude is bounded by the prime meridian (Greenwich Observatory, UK), the positive and negative of East and West.

people nearbyThat’s what they often sayLBS(location based services), which focuses on the user’s current geographic location data and provides accurate encounter services for users.

people nearbyThe core idea is as follows:

  1. Take “I” as the center and search nearby TA;
  2. Calculate the distance between others and me based on my current geographical location;
  3. Sort by the distance between “I” and others to filter out the users closest to me.

MySQL implementation

Calculate “people nearby”, calculate other data near this coordinate through a coordinate, and sort them according to the distance. How to start?

Take the user as the center and draw a circle with a radius of 1000m, then the users in the circular area are the “nearby people” we want to meet.

Store latitude and longitude inMySQL

CREATE TABLE `nearby_user` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `Name ` varchar (255) default null comment ',
  `Longitude ` double default null comment 'longitude',
  `Latitude ` double default null comment ',
  `create_ time` datetime DEFAULT NULL ON UPDATE CURRENT_ Timestamp comment 'creation time',
  PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;

However, you can’t traverse all the longitude and latitude of the goddess and your own longitude and latitude data. The amount of calculation is too large in sorting according to the distance.

We can filter out the limited “goddess” coordinate data through the region, and then calculate the full distance of the data in the rectangular region and then sort it. In this way, the amount of calculation is significantly reduced.

How to divide rectangular areas?

A square on the circular coat filters the data according to the maximum and minimum values of the user’s longitude and latitude (longitude and latitude + distance) as the filtering criteria, and it is easy to search the “goddess” information in the square.

Redis actual combat: geo helps me meet nearby goddesses

What about the extra areas?

The distance from the extra users in this area to the dot must be greater than the radius of the circle, so we calculate the distance between the user center point and all users in the square,Filter out all users whose distance is less than or equal to the radius, all users in the circular area meet the requirementspeople nearby

In order to meet the high-performance rectangular region algorithm, the data table needs to add a composite index to the longitude and latitude coordinates(longitude, latitude)This maximizes query performance.

actual combat

A third-party class library is used to obtain the maximum and minimum longitude and latitude of the external rectangle according to longitude and latitude and distance, and calculate the distance according to longitude and latitude:

<dependency>
     <groupId>com.spatial4j</groupId>
     <artifactId>spatial4j</artifactId>
     <version>0.5</version>
</dependency>

After getting the circumscribed rectangle, theMaximum and minimum longitude and latitude valuesSearch the users in the square area, and then eliminate the users beyond the specified distance, which is the final resultpeople nearby

/**
 *Get people x meters nearby
 *
 *@ param distance search distance range unit: km
 *@ param userlng longitude of the current user
 *@ param userlat latitude of the current user
 */
public String nearBySearch(double distance, double userLng, double userLat) {
  //1. Get external square
  Rectangle rectangle = getRectangle(distance, userLng, userLat);
  //2. Get all users located in the square
  List<User> users = userMapper.selectUser(rectangle.getMinX(), rectangle.getMaxX(), rectangle.getMinY(), rectangle.getMaxY());
  //3. Eliminate redundant users whose radius exceeds the specified distance
  users = users.stream()
    .filter(a -> getDistance(a.getLongitude(), a.getLatitude(), userLng, userLat) <= distance)
    .collect(Collectors.toList());
  return JSON.toJSONString(users);
}

//Get circumscribed rectangle
private Rectangle getRectangle(double distance, double userLng, double userLat) {
  return spatialContext.getDistCalc()
    .calcBoxByDistFromPt(spatialContext.makePoint(userLng, userLat), 
                         distance * DistanceUtils.KM_TO_DEG, spatialContext, null);
}

     /***
     *The distance between two points in a sphere
     *@ param longitude longitude 1
     *@ param latitude latitude 1
     *@ param userlng longitude 2
     *@ param userlat latitude 2
     *@ return return distance in KM
     */
    private double getDistance(Double longitude, Double latitude, double userLng, double userLat) {
        return spatialContext.calcDistance(spatialContext.makePoint(userLng, userLat),
                spatialContext.makePoint(longitude, latitude)) * DistanceUtils.DEG_TO_KM;
    }

Because the sorting of distance between users is implemented in business code, you can see that SQL statements are also very simple.

SELECT * FROM nearby_user
WHERE 1=1
AND (longitude BETWEEN #{minlng} AND #{maxlng})
AND (latitude BETWEEN #{minlat} AND #{maxlat})

However, the database query performance is limited after all. If “people nearby” query requests are very many, this may not be a good solution in high concurrency situations.

Failed to try redis hash

Let’s analyze the characteristics of LBS data:

  1. Each goddess has an ID number, and each ID corresponds to longitude and latitude information.
  2. “Otaku” landingapp When you get the “heart girl”,app Find the nearby “goddess” according to the longitude and latitude of “otaku”.
  3. After obtaining the list of “goddess” IDS matching the location, obtain the “goddess” information corresponding to the ID from the database and return it to the user.

The data feature is that a goddess (user) corresponds to a group of longitude and latitude, which reminds me of redis’s hash structure. That is, a key (goddess ID) corresponds to a value (longitude and latitude).

Redis actual combat: geo helps me meet nearby goddesses

Hash It seems that it can be realized, but in addition to recording longitude and latitude, LBS applications also need to query the range of data in the hash set and convert it into distance sorting according to longitude and latitude.

The data of hash set is out of order, which is obviously undesirable

Sorted set is beginning to show signs

Is the sorted set type appropriate? Because it can be sorted.

Sorted SetType is also akey Corresponding to onevalueKey element content, andValue ` is the weight score of the element.

Sorted Set You can sort the elements according to their weight scores, which seems to meet our needs.

For example, the element of sorted set is goddess ID, and the weight score corresponding to the element is longitude and latitude information.

Redis actual combat: geo helps me meet nearby goddesses

Here’s the problem. The weight value of the sorted set element is a floating point number, and the longitude and latitude are longitude and latitude. What should I do? Can I convert latitude and longitude into a floating point number?

The idea is right. In order to compare longitude and latitude, redis adopts the geohash code widely used in the industry to encode longitude and latitude respectively, and finally combines the longitude and latitude codes into a final code.

In this way, the longitude and latitude are converted into a value, andRedis’s geo type underlying data structure usesSorted SetTo achieve

Let’s take a lookGeoHashHow to encode longitude and latitude.

Geohash code

For geohash, please refer to:https://en.wikipedia.org/wiki…

GeoHash The algorithm maps the two-dimensional longitude and latitude data to one-dimensional integers, so that all elements will be mounted on a line, and the distance between the two-dimensional coordinates close to each other will be very close to the points after mapping to one-dimensional coordinates.

When we want to calculate “people nearby”, we first map the target position to this line, and then get the nearby points on this one-dimensional line.

Geohash coding will encode a longitude value into an n-bit binary value. Let’s do n times of two partition operations for the longitude range [- 180180], where n can be customized.

During the first second partition, the longitude range [- 180180] will be divided into two sub intervals [- 180,0) and [0180] (I call them left and right partitions).

At this point, we can check whether the longitude value to be encoded falls in the left partition or the right partition. asIf it falls in the left partition, we use 0 to represent it; If it falls in the right partition, it is represented by 1

thus,Every time we finish the second partition, we can get the 1-bit coded value (either 0 or 1).

Then make a second partition for the partition to which the longitude value belongs. At the same time, check whether the longitude value falls in the left or right partition after the second partition again. Make another 1-bit coding according to the rules just now. After completing the second partition n times, the longitude value can be expressed by an n bit number.

All map element coordinates are placed in a unique grid. The smaller the grid, the more accurate the coordinates. Then these squares are integer coded. The closer the squares are, the closer the coding is.

After coding, the coordinates of each map element will become an integer. Through this integer, the coordinates of the element can be restored. The longer the integer, the smaller the loss of the restored coordinate value. For the function of “people nearby”, the loss of a little accuracy is negligible.

For example, for a longitude value equal to169.99Carry out 4-bit coding (n = 4, make 4 partitions), and divide the longitude interval [- 180180] into left partition [- 180,0) and right partition [0180].

  1. 169.99 belongs to the right division, using1Indicates the first partition code;
  2. After the first division, 169.99 is further divided into [0, 90) and [90, 180], and 169.99 is still in the right section, coding ‘1’.
  3. Divide [90, 180] into [90, 135) and [135, 180], this time in the left partition, code ‘0’.

In this way, we finally get a 4-bit code.

The coding idea of latitude is the same as that of longitude, so I won’t repeat it.

Combined longitude and latitude coding

If the calculated longitude and latitude codes are11011 and00101 `, bit 0 of the target code takes the value 1 of bit 0 of longitude as the target value, bit 1 of the target code takes the value 0 of bit 0 of latitude as the target value, and so on:

Redis actual combat: geo helps me meet nearby goddesses

In this way, longitude and latitude (35.679114.020) can be used1010011011Represents, and this value can be used asSortedSetThe weight value of is used to sort.

Redis geo implementation

Geo type takes the geohash encoded combined value of longitude and latitude as the score weight of sorted set element. What are the instructions of redis’s geo?

We need to save the girl Id and the corresponding latitude and longitude of the login app into the sorted set.

For more geo type instructions, refer to:https://redis.io/commands#geo

GEOADD

Redis providesGEOADD key longitude latitude memberCommand to record a set of longitude and latitude information and the corresponding “goddess ID” into the geo type collection, as follows: record the longitude and latitude information of multiple users (cangai Kong and bodono Jieyi) at one time.

Geoadd girl: localization 13.361389 38.115556 "cangai Kong" 15.087269 37.502669 "bodono clothing"

GEORADIUS

I logged in to the app to get my longitude and latitude information. How can I find other users within a certain range centered on this longitude and latitude?

Redis GEO Type providesGEORADIUS Instruction: it will find other elements within a certain range centered on this longitude and latitude according to the entered longitude and latitude position.

Assuming your longitude and latitude is (15.087269 37.502669), you need to obtain the “goddess” in the vicinity of 10 km and return it to the LBS application:

GEORADIUS girl:locations 15.087269 37.502669 km ASC COUNT 10

ASC The “goddess” information can be sorted from near to far according to its own longitude and latitude.

COUNT Option means to specify the number of “goddesses” returned to prevent too many “goddesses” nearby and save bandwidth resources.

If you think you need more goddesses, there can be no limit, but you need to pay attention to your body and eat more eggs.

What about deleting the latitude and longitude of the goddess after the user goes offline?

That’s a good question,GEOType is based onSorted SetImplemented, so it can be borrowedZREMCommand to delete geographic location information.

For example, delete the location information of “AOI Kong”:

Zrem girl: localization "AOI Kong"

Summary

Instead of designing a new underlying data structure, geo directly uses the sorted set collection type.

Geo type uses geohash coding method to realize the conversion from longitude and latitude to element weight score in sorted set. Two key mechanisms are interval division of two-dimensional map and interval coding.

After a group of longitude and latitude falls in an interval, it is represented by the coded value of the interval, and the coded value is taken as the weight score of the sorted set element.

In a map application, there may be millions of car data, restaurant data and human data. If redis’s geo data structure is used, they will all be placed in a Zset set.

In the redis cluster environment, collections may migrate from one node to another. If the data of a single key is too large, it will have a great impact on the migration of the cluster. In the cluster environment, the amount of data corresponding to a single key should not exceed 1m, otherwise the cluster migration will be stuck and the normal operation of online services will be affected.

Therefore, it is recommended that geo’s data be deployed using a separate redis cluster instance.

If the amount of data is more than 100 million or more, geo data needs to be split by country, province, city, and even by district in populous megacities.

This can significantly reduce the size of a single Zset set.

Redis actual combat: geo helps me meet nearby goddesses

Giant shoulder

  1. https://segmentfault.com/a/11…
  2. https://juejin.cn/book/684473…
  3. https://cloud.tencent.com/dev…
  4. Redis core technology and actual combat

Recommended Today

Swift advanced (XV) extension

The extension in swift is somewhat similar to the category in OC Extension can beenumeration、structural morphology、class、agreementAdd new features□ you can add methods, calculation attributes, subscripts, (convenient) initializers, nested types, protocols, etc What extensions can’t do:□ original functions cannot be overwritten□ you cannot add storage attributes or add attribute observers to existing attributes□ cannot add parent […]