- If a complete pipeline involves more than four sub modules, how to quickly check the problem;
- It is inevitable for any sub module to have an exception. How to ensure the correctness of business data;
- Whether the error can be found in time and the wrong business data can be terminated to continue the downward execution;
- How to prevent errors from spreading downward due to the uneven coding level of project members;
Two, modular programming
Modular programming is a kind of software design technology, which emphasizes that the function of the program is divided into independent and interchangeable modules, so that each module contains everything necessary to perform one aspect of the required function. The module interface represents the elements provided and required by the module. The elements defined in the interface can be detected by other modules. The implementation contains the working code corresponding to the elements declared in the interface.
With modular programming, concerns can be separated, so that modules can perform logically discrete functions and interact with each other through well-defined interfaces. Generally, modules form a directed acyclic graph (DAG). In this case, the circular dependencies between modules are considered to indicate that the modules should be single. When the modules do form DAG, they can be arranged in a hierarchical structure. The lowest level module is independent and does not depend on other modules, while the higher level module depends on the lower level module. A specific program or library is the top module in its own hierarchy, but it can also be regarded as the lower module of a higher level program, library or system.
3、 Untrusted verification thinking
- The writer said: it is better to believe that it has something than to believe that it has nothing.
- Database experts say: This is a pessimistic lock.
- The tester said: a destructive test.
- R & D personnel who suffer losses: they should always be skeptical when they get the data, and verify it before using it. Check carefully, do not carry the pot.
- The service modules are split reasonably to ensure that the functions of the sub modules are relatively single and independent, and different sub modules are serially connected by globally unique service codes;
- The required service data of the sub module is obtained by the globally unique service code;
- The sub module actively verifies the result data generated by the previous business module. If the verification is qualified, continue to process, if the verification fails, terminate the process and record the failure reason;
4、 Case analysis
- Case background
There is a distribution business in e-commerce system, especially in the era of Internet red, which is also a distribution mode. For example, a certain distribution organization has 3000 orders per month, and the actual receipts of these orders flow into the distribution organization. These 3000 orders have not been paid in the e-commerce system and are in the open state. The distribution organization regularly makes payment to the e-commerce company to settle the order flow.
Parameter 1: 3000 orders.
Parameter 2: distribution organization a.
Parameter 3: distribution organization a provides n cash flow records
- Modular split
To realize the settlement function of this batch of orders, the e-commerce system can be developed based on modular programming. The main process references are as follows:
Statement: refers to the business order number that the distribution organization makes payment and settlement to the e-commerce company on a regular basis.
Serial number: Bank flow information of distribution payment. One statement corresponds to multiple flow information.
In this example, different modules only flow through the statement number (avoid: the previous module transfers a large number of business parameters to the next module). Each sub module can be decoupled by event, message and other middleware patterns, and different modules should be independent.
When the latter sub module processes the business, it obtains the required data according to the reconciliation number and verifies the correctness of the data.
Process reference illustration:
When the statement is written off, considering the large amount of order data, it can be realized based on the background task processing, using the mode of order write off one by one. The write off process is as follows:
An order is matched with a flow line: the payment flow line amount is greater than the amount of the order to be written off
One order is suitable for multiple flow lines: the amount of a single payment flow line is not enough to write off the amount to be paid of an order
- Untrusted verification thinking
- Generate statement module: verify the basic information of the order, whether the order has been written off, verify the payment flow information, and judge whether the payment flow information still has available amount and whether it is normal.
- Write off statement module: query the basic information of statement based on statement code. If the amount to be reconciled in the statement is the same as the total write off amount of the application order. Whether the flow information associated with the statement is available.
- Internal implementation module of write off: simulated write off is used to verify whether all orders have been written off and whether the amount deduction of simulated write off is successful.
- Order flow completion: analyze the write off log of the corresponding order to see if the write off amount is consistent with the original amount to be written off.
- After generating the statement module, the extended statement information is transferred to the write off statement module, which directly uses the received parameters for business processing.
- When the write off module uses the flow information, it obtains the statement and the associated flow information for direct use without verification.