Design and practice of coupon system architecture

Time:2022-5-29

Coupons and shopping malls are coupled in one system. With the intensification of marketing activities, the usage scenarios of coupons have increased, exposing problems:

(1) The issuance of massive coupons has reached the bottleneck of coupon single library and single table storage.

(2) The high coupling with the mall system directly affects the performance of the whole station interface of the mall.

(3) The iterative update of coupons is limited by the version arrangement of the mall.

For multi category coupons, there is no ability to deposit general coupons at the technical level.

To solve the above problems, the coupon system architecture is designed as follows:

Design and practice of coupon system architecture

How to migrate coupons from the mall system and be compatible with the connected business parties and historical data is also a major technical challenge. There are two schemes for system migration: downtime migration and non downtime migration.

The non-stop migration scheme is as follows:

Before migration, the operator shall stop the background operations related to coupons to avoid generating static data of coupons.

Static data: data generated in the background of coupons, independent of users.

Dynamic data: coupon data related to users, including the relationship data of coupons, coupons and orders received by users.

Configure the current database switch to write only, that is, the coupon data is written to the mall Library (Old Library).

The coupon system goes online and migrates static data through scripts. After migration, verify the accuracy of static data migration.

Configure the current database switch to double write, that is, the online data is written to the mall library and the new coupon library at the same time. At this time, the data source provided by the service is still the mall library.

Migrate dynamic data. After migration, verify the accuracy of dynamic data migration.

Switch the data source, and switch the data source provided by the service to the new library. Verify whether the service is correct. If there is a problem, switch back to the mall data source.

Close double write and the coupon system migration is completed.

The system request topology is as follows:

Design and practice of coupon system architecture

System design:

1. Coupon sub database and sub table:

There are mature open source solutions for sub database and sub table, which will not be introduced here. Referring to the previous project experience, the self-developed framework provided by the middleware team of the company is adopted. The principle is to introduce the self-developed mybatis plug-in, calculate different database table suffixes according to the customized routing strategy, and locate them to the corresponding database table.

The user coupon is associated with the user ID, and the user ID is an important field throughout the whole system. Therefore, the user ID is used as the routing factor for sub database and sub table. This can ensure that the same user routes to the same database table, which is not only conducive to data aggregation, but also convenient for user data query.

It is assumed that there are n databases and M tables in total, and the routing strategy for the databases and tables is:

Library suffix databasesuffix = hash (userid) / m%n

Table suffix tablesuffix = hash (userid)% m

Design and practice of coupon system architecture

2. Design of coupon issuing method

Unified coupon collection interface:

To ensure the accuracy of the verification of receiving vouchers

When collecting coupons, you need to strictly verify whether the various attributes of the coupon are met, such as the receiving object and various restrictions. Among them, the key is the verification of inventory and received quantity. Because in the case of high concurrency, it is necessary to ensure the accuracy of quantity verification, otherwise it is easy to cause users to over receive.

There is such a scenario: user a sends a request to receive coupon C twice in a row, and coupon C restricts each user to receive one. The first request has passed the verification of the quantity of collected coupons. If there is no restriction when the user’s coupons have not been deposited in the library, the second request will also pass the verification of the quantity of collected coupons. In this way, user a will successfully receive two coupons C, resulting in excessive receipt.

To solve this problem, the coupon adopts the distributed lock scheme, and the implementation of the distributed lock depends on redis. Try to obtain the distributed lock before verifying the number of coupons received by the user. Release the lock after the coupon is issued successfully to ensure that the user will not receive the same coupon over time. In the above scenario, after the user successfully obtains the distributed lock for the first time, the obtained distributed lock will be released or released over time for the first time. Otherwise, the user will fail to obtain the distributed lock for the second time. This ensures that user a will only successfully receive one.

Inventory deduction

Inventory deduction is required for voucher collection. There are two common inventory deduction schemes:

Scheme 1: database deduction.

When deducting inventory, directly update the inventory field in the database.

Of the schemeadvantageIt is simple and convenient. When checking inventory, you can directly check the inventory to obtain real-time inventory. It also has database transaction guarantee, without considering the problem of data loss and inconsistency.

shortcomingObviously, there are two main points:

1) Inventory is a single field in the database. When updating inventory, all requests need to wait for row lock. Once the concurrency is large, many requests will be blocked here, resulting in request timeout and system avalanche.

2) Frequent database requests are time-consuming and will consume a lot of database connection resources.

Scheme II:Implement inventory deduction based on redis.

Put the inventory into the cache and deduct the inventory by using the incrby feature of redis.

Of the schemeadvantageIt breaks through the bottleneck of the database, with high speed and high performance.

shortcomingThe system process is complex, and the cache loss or downtime data recovery needs to be considered, which is easy to cause inconsistency of inventory data.

Considering the current and foreseeable future traffic peak, system maintainability and practicability of the coupon system, the coupon system adopts the improvement scheme of scheme I. The improvement scheme is to disperse the single inventory field into multiple inventory fields, disperse the row locks of the database, and reduce the row lock bottleneck of the database in the case of large concurrency.

Design and practice of coupon system architecture

After the inventory quantity is updated, the inventory will be evenly distributed into m copies and initialized and updated into the inventory record table. After collecting vouchers, the user randomly selects a certain inventory field (m in total) that has been allocated in the inventory record table to update. If the update is successful, the inventory deduction is successful. At the same time, the scheduled task will periodically synchronize the received inventory. Compared with scheme 1, this scheme breaks through the bottleneck of database single line lock and is simple to implement without considering the problems of data loss and inconsistency.

Receive multiple vouchers with one click

In the voucher collection scenario of the connected business party, there is a case where users can receive multiple vouchers with one click. Therefore, the unified voucher collecting interface needs to support users to collect vouchers with one click. In addition to receiving multiple vouchers of the same voucher template, it also supports receiving multiple vouchers of different voucher templates. Generally speaking, receiving multiple coupons with one click refers to receiving multiple coupons with different coupon templates. In the implementation process, the following points should be noted:

(1) Performance issues

Receive multiple coupons. If each coupon is verified, inventory deducted, and warehoused separately, the bottleneck of interface performance is the number of coupons. The more the number, the worse the performance. So how to ensure high performance when there are many coupons? Two measures are mainly taken:

a.Batch operation

From the point of view of the issuance process, the bottleneck lies in the receipt of securities. Collecting coupons is real-time (in case of asynchronism, coupons cannot be sent to the user account in real time, which will affect the user experience and the conversion rate of coupons). The more coupons, the more IO times with the database during warehousing, the worse the performance. Batch warehousing can ensure only one IO with the database, which is not affected by the number of vouchers. As mentioned above, the user’s coupon data is divided into databases and tables, and the coupon assets of the same user are saved in the same database and table, so the same user can realize batch receipt.

b.Limit the number of coupons received in a single time

Set the threshold value and return directly after exceeding the quantity to ensure that the system is within the safe range.

2) In case of high concurrency, users will not over collect

If a user initiates a request in the mall, he / she can receive four coupons a/b/c/d with one click, and the activity system will issue a coupon to the user at the same time. The two coupon receiving requests are simultaneous. Among them, coupon a restricts each user to receive only one coupon. According to the above, distributed locks are used to ensure the accuracy of verification. The keys of the distributed locks requested twice are:

User id+a\_ Id+b\_ Id+c\_ Id+d\_ ID

User id+a_ ID

In this case, the distributed locks requested twice do not work because the lock keys are different, and there is still the possibility of error in the quantity verification. In order to avoid the phenomenon of users’ over picking in the process of batch picking up vouchers, the acquisition of distribution lock is modified in the process of batch picking up vouchers. In the above example, to receive four a/b/c/d vouchers with one click, you need to obtain four distributed locks in batch. The lock keys are:

User id+a_ ID

User id+b_ ID

User id+c_ ID

User id+d_ ID

Failure to acquire any of the locks indicates that the user is receiving one of the tickets and needs to wait (within the timeout period). The next step can be performed only after all distributed locks are obtained successfully.

If a user initiates a request in the mall, he / she can receive four coupons a/b/c/d with one click, and the activity system will issue a coupon to the user at the same time. The two coupon receiving requests are simultaneous. Among them, coupon a restricts each user to receive only one coupon. According to the above, distributed locks are used to ensure the accuracy of verification. The keys of the distributed locks requested twice are:

User id+a\_ Id+b\_ Id+c\_ Id+d\_ ID

User id+a_ ID

In this case, the distributed locks requested twice do not work because the lock keys are different, and there is still the possibility of error in the quantity verification. In order to avoid the phenomenon of users’ over picking in the process of batch picking up vouchers, the acquisition of distribution lock is modified in the process of batch picking up vouchers. In the above example, to receive four a/b/c/d vouchers with one click, you need to obtain four distributed locks in batch. The lock keys are:

User id+a_ ID

User id+b_ ID

User id+c_ ID

User id+d_ ID

Failure to acquire any of the locks indicates that the user is receiving one of the tickets and needs to wait (within the timeout period). The next step can be performed only after all distributed locks are obtained successfully.

Interface idempotency

The unified voucher collecting interface shall ensure idempotency (idempotency: the results of one request or multiple requests initiated by the user for the same operation are consistent). In case of network timeout and abnormal conditions, if the result of collecting coupons is not returned in time, the business party will retry collecting coupons. If the interface does not guarantee idempotency, it will cause over issuance. There are many schemes to realize idempotence. The coupon system uses the unique index of the database to ensure idempotence.

In the early days, coupon collecting did not support idempotence, and the table design did not consider idempotence.

thatThe first question to consider: in which table is the unique index added?

There are two options: existing tables or new tables.

The existing table is adopted, and the table association does not need to be added. However, as mentioned above, because there are databases and tables, a large number of tables need to add unique fields, and they need to be compatible with historical data. It is necessary to ensure the uniqueness of the new fields of historical data.

The method of creating a new table does not need to be compatible with historical data, but the defect is also obvious. It adds a layer of table Association, which has a great impact on performance and existing logic. After comprehensive consideration, we chose to add a unique field to the existing table, which is more conducive to ensuring performance and subsequent maintainability.

The second question to consider: how can historical data and business parties be compatible?A unique field has been added to the historical data. You need to fill in a unique value, otherwise you cannot add a unique index. We use script to brush data, construct unique values and refresh them into each row of historical data. The business party to which the coupon has been connected has not passed in a unique code. In this case, the coupon side generates a unique code as a substitute to ensure compatibility.

Directional issuance of securities

Directional issuance of coupons is used by the operation to issue coupons to specific groups in the background. Targeted issuance of vouchers can make up for the problem that users take the initiative to collect vouchers, and the crowd coverage is not accurate and wide. Through targeted coupon issuance, specific groups can be accurately covered and the order conversion rate can be improved. During the promotion period, the targeted issuance of coupons by a wide range of people can also carry the dual tasks of activity push and price reduction promotion.

The targeted issuance of bonds mainly depends on the selection of the crowd and the design of the issuance process. The overall process is as follows:

1) Remove transaction. The transaction logic is too heavy, which is not necessary for directional issuance. Failed to issue coupons, record the failed coupons, and ensure that the failure can be retried.

2) Lightweight calibration. Directional issuance restricts the type of coupons, and avoids the configuration of attributes that need to be strictly verified by restricting the configuration. Unlike the lengthy verification logic of users’ active receipt, the verification of directional issuance is very light, which greatly improves the performance of issuance.

3) Batch insert. Batch coupon insertion reduces database IO times, eliminates database bottlenecks, and improves coupon issuance speed. Targeted coupon issuance is aimed at different users. User coupons are divided into different databases and tables. In order to realize batch insertion, it is necessary to calculate the database table suffixes corresponding to different users in memory, and then insert them in batch after data collection. The maximum number of inserts is m, and M is the total number of database tables.

4) Core parameters can be dynamically configured. For example, the number of vouchers issued in a single time, the number of libraries read in a single time, and the number of users included in the message body sent to the message center can control the peak speed and average speed of directional issuance.

Coupon code exchange

The distribution method of off-site marketing coupons is different from that of other coupons, which can be exchanged by coupon code. The coupon code is exported from the background and distributed to the user through SMS or activity. The user will get the corresponding coupon after exchanging according to the coupon code. There are certain rules for the composition of the coupon code. On the basis of the rules, the security should be ensured. This security is mainly the accuracy of the coupon code verification to prevent the re conversion of the redeemed coupon code and the malicious conversion of invalid coupon code.

Refined marketing capability design

Through the combination and configuration of labels, coupons provide the ability of fine marketing to achieve thousands of people and thousands of faces of coupons. Tags can be divided into quasi real-time and real-time. It is worth noting that some real-time tags require preconditions, such as user authorization for regional attributes.

Relationship between coupons and commodities

Coupons need to be associated with commodities. They can be associated with all commodities or some commodities. In order to flexibly meet the configuration of coupon related products for the operation, the coupon system has two ways of association:

a. Blacklist.

Available products = all products – blacklist products.

The blacklist is applicable to a wide range of commodities available for the coupon. Excluding all commodities from the blacklist is the usable range of the coupon.

b. White list.

Available items = white list items.

The white list is applicable to the case that the range of available commodities of the coupon is relatively small, and the available commodities of the coupon are directly configured.

In addition, there is the configuration of super blacklist. Blacklist and white list are only valid for a single coupon, and super blacklist is valid for all coupons. The current coupon system provides commodity level association. Subsequent coupons will support the association of commodity classification dimensions. The classification dimension + commodity dimension can be more flexibly associated with coupons and commodities.

High performance guarantee

There are many coupon docking systems, and there are high traffic scenarios. The external interface provided by the coupon needs to ensure high performance and stability.

Multilevel cache

In order to improve the query speed and reduce the pressure on the database, and to cope with the scenarios where the instantaneous high traffic brings hot keys (for example, switching traffic to the specific commodity vendor details page after the live broadcast of the press conference and the hot activity commodity vendor details page will bring instantaneous high traffic to the coupon system), the coupon adopts a multi-level cache method.

Database read / write separation

In addition to the above-mentioned sub database and sub table, the coupons also perform read-write separation on this basis. The master database is responsible for executing the data update request, and then synchronizing the data changes to all slave databases in real time. The slave database is used to share the query request and solve the problem that database writing affects the query. There is a delay in master-slave synchronization. Under normal circumstances, the delay does not exceed 1ms. There is a time-consuming process for coupon collection or status change. The master-slave delay is not perceived by the user.

Dependent on external interface isolation fusing

The coupon internally relies on the system of a third party. In order to prevent the chain effect caused by the unavailability of the service of the relying party, which eventually leads to the avalanche of the coupon service, the coupon isolates and fuses the external interface.

User dimension coupon field redundancy

Querying user related coupon data is one of the most frequent operations for querying coupons. The user’s coupon data is divided into databases and tables, and cannot be queried in association with the coupon rule table. In order to reduce the number of IO, some fields of coupon rules are redundant in the user’s coupon table. There are many fields in the coupon rule table, and there cannot be many redundant fields. Balance the performance and the number of fields.