Deep practice of Tencent Financial Core transaction solution tdmesh

Time:2021-4-29

Recently, at the archsummit global architects summit, Tencent’s billing team shared the in-depth practice of financial core transaction solution tdmesh. This is another big move of Tencent billing after its first offline display at Tencent global digital ecology conference in May this year.

It is understood that Tencent billing is an Internet trading platform incubated in supporting Tencent’s internal business revenue of 100 billion. It brings together the mainstream payment channels at home and abroad and provides multi-dimensional services such as account management, precision marketing, security risk control, audit and accounting analysis. The platform carries the revenue of Tencent, with 10 billion level accounts, 10 billion level daily transactions, covering 180 + countries (regions), serving tens of thousands of businesses and millions of merchants. It is a comprehensive one-stop billing platform.

Deep practice of Tencent Financial Core transaction solution tdmesh

Tdmesh is hatched from Tencent’s accounting technology system which has supported Tencent for 15 years. It includes multiple functional modules, such as global consistency, massive request support, 7 * 24 uninterrupted service management, second level intelligent monitoring and elastic scaling, etc., and forms a comprehensive financial level processing scheme.

Li Gang, general manager of Tencent’s billing platform department, said: “based on the challenges of high consistency, high performance, high availability and high reliability of billing, Tencent’s billing platform department, based on years of experience in serving internal business, has comprehensively optimized and integrated the core technology sector and launched a set of financial grade core transaction solution tdmesh. At the same time of continuous self evolution and continuous polishing, we also hope to participate in relevant technical discussions and work with partners to build a financial level high consistency technology paradigm to promote the development of the industry. “

To meet the challenge of “four high”, Tencent’s billing technology construction ideas

Tencent billing system infrastructure started in 2004   15 years later   In the course of development in, with the increase of access services, the rise of mobile payment and regional globalization of deployment, Tencent has faced a series of challenges in terms of billing

·High consistency of financial grade. Tencent billing carries 10 billion level account storage such as game account, integral rights and interests, pan entertainment subscription and advertiser, with hundreds of thousands of levels per second at the peak of transaction, more than 10 billion transactions per day, and more than 100 version or process changes per day on average. Under the huge scale of transactions, how to ensure zero error in revenue of 100 billion years and achieve financial consistency is the core problem of billing.

·High performance. The billing process, such as the consumption of game tickets, the deduction of per click advertising, etc., is characterized by high-frequency visits and hot spots. In addition, the sudden traffic of tens of times of the activity volume and the superimposed instantaneous peak value exceeding one million TPS require the billing to have high single machine performance and overall redundancy;

·High availability. Billing services have been deployed in 100 countries around the world+   Cluster, as a large-span payment system, is bound to face a variety of network anomalies, differences in payment channels in different countries, difficulties in configuration and data processing, which require billing services to have strong business disaster tolerance and data disaster tolerance capabilities, sufficient flexibility and fault tolerance design, and the unavailable time should be controlled at the annual minute level;

·High reliability. Mainly for the storage layer, as long as the response is successful, it means that the data will be dropped. For unexpected downtime such as operating system crash, it can ensure that the data will not be lost; At the same time, the failure of any distributed node, resulting in local data damage and brain crack and other issues, also requires strict data not to lose.

In order to meet the above “four high” challenges, Tencent billing adopts two major design ideas: Micro service and China Taiwan chemical

Microservice design, through reasonable function splitting, and fine service governance, such as service registration, discovery, fusing, self-healing, load balancing, link tracking and so on, realizes the rapid expansion of functions and efficient traffic scheduling, so as to achieve high scalability and high availability of the whole system.

The central platform design, through reasonable functional organization, clear module boundaries, component-based abstraction of internal similar logic, and unified service authentication, inter service configuration, intra service transactions, and unified storage of services, realizes the high reusability of the overall service and improves the research and development efficiency.

Deep practice of Tencent Financial Core transaction solution tdmesh

Based on the above ideas, Tencent billing has built a financial level billing solution tdmesh, which covers four high demand components in every link from access to storage, plus management and control modules in operation, such as service scheduling, monitoring, gray level change and data analysis. These component modules cooperate with each other to support the business development of Tencent billing and external partners.

Technical product system is perfect, tdmesh has been recognized by the industry

From the perspective of technical architecture, tdmesh has formed a complete and rich technical product system.

Deep practice of Tencent Financial Core transaction solution tdmesh

Chen Ningguo, chief billing architect and billing expert of Tencent, comprehensively analyzed the overall architecture of tdmesh at the conference. He said that Tencent’s financial core transaction solution, tdmesh, includes 1) TDEA unified entry scheduling, protocol management, and service authentication; 2) TDCC configuration engine, high consistency management of multi link complex association configuration;   3) Tdxa transaction engine, high consistency management of complex long link transaction logic in application layer;   4) TDF high performance basic service framework;   5) Tdmq is a distributed message queue with high consistency and high performance;   6) TD hold distributed cache with high consistency and high performance; 7) Tdsql financial high consistency and high reliability database; 8) Tdledger’s decentralized highly consistent accounting mechanism, blockchain; 9) Operation system to achieve unified scheduling and flexible management of services.

For developers, after using tdmesh, app can access all components of tdmesh in one step as long as it connects with tdxa, realizing a package of standardized processing of the whole transaction business chain from configuration, access, application layer transaction, to order storage, message communication, reconciliation, etc., so as to realize the four high requirements of high consistency of self service.

Deep practice of Tencent Financial Core transaction solution tdmesh

Typical application example of tdmesh in billing: a typical discount activity, through the cooperation of component technology capabilities in seven key links, realizes the smooth implementation of large amount of requests.

Deep practice of Tencent Financial Core transaction solution tdmesh

It is worth mentioning that the whole set of tdmesh distributed solution is not only suitable for billing, but also has high reference and application value for financial related industries, and has been recognized by the financial industry.

Up to now, tdsql, one of the components of tdmesh, a financial solution, has supported the core business system of Zhangjiagang rural commercial bank, the first traditional bank in China, to turn to a new generation of distributed core business system and realize self-control. At the same time, this distributed database product has also been opened to partners; At present, tdsql database has provided database services for more than 500 public cloud customers and nearly 30 private cloud customers such as Bank of China.

In this release, the tdmesh components mainly include tdxa, which is responsible for application layer transactions, and tdmq, which is responsible for system decoupling and transaction exception handling.

Tdxa (Tencent distributed extended architecture) is a distributed transaction scheduling engine which aims at long link consistency, supports mixed transaction modes and graphical development. It supports transaction patterns, such as TCC / saga / at, etc; Its choreographing ability is mainly to automatically confirm or roll back the process state through jump tracking, and zero manual intervention in case of exception; Its development mode is mainly a graphical one-stop integrated R & D platform; So as to achieve the ultimate goal

The complex distributed consistency problem is given to the platform, which makes the business development more focused, simplifies the complexity, and improves the fault tolerance and stability of the system.

Deep practice of Tencent Financial Core transaction solution tdmesh

Tdxa is mainly suitable for OLTP (online transaction processing) long transaction processing scenarios. Billing service is based on microservice architecture. It usually makes dozens of network calls for a complete transaction process, including RPC interface, native DB interface, kV interface, etc. Based on the requirement of high consistency, the complexity and uncontrollability of the code are obvious if the developer handles all kinds of exceptions to ensure high consistency. Therefore, the core value of tdxa lies in standardizing the handling of consistency problems through unified components, effectively reducing the workload of developers in dealing with errors or exceptions under the premise of ensuring high consistency and high quality of services, and meeting the requirements of high consistency, high performance and high reliability of billing system for diversified transaction scenarios and rapid growth of transaction volume in business development.

Deep practice of Tencent Financial Core transaction solution tdmesh

The normal operation of tdxa, especially the processing of timeout, the asynchronous decoupling between different systems and the data synchronization of database, depends on another core component, tdmq (tent distributed message queue)

Tdmq is a message queue designed by cloud native architecture, which has high consistency, high reliability and high performance across cities. It has been widely used in Tencent billing scenarios, supporting daily average 10 billion + request delivery, including payment notification, real-time reconciliation, real-time monitoring, real-time big data analysis, etc.

Deep practice of Tencent Financial Core transaction solution tdmesh

A typical feature of tdmq cloud is that it supports the separation of storage and computing. The upper computing layer is composed of independent brokers, which supports high-performance network access and topic management. The lower storage layer implements high consistent replication of log flow. In this way, it can be expanded according to business performance or capacity requirements to achieve the overall cost optimization. Typical features of tdmq include delay message, transaction message, message tracing, complete multi tenant support, arbitrary number of consumers, etc., which are friendly to high concurrent consumption scenarios. In addition, in terms of operation, it also supports fast data smooth migration, logical isolation and other features, so as to avoid the mutual influence of production and consumption among multiple topics.

Finally, as a technical product dedicated to the package solution of financial level core transaction, we hope to continuously polish and improve the usability, manageability, performance cost ratio, cloud native adaptation and other aspects through richer scenarios and customers, while constantly carrying out in-depth practice and summing up. These concepts are also being developed within Tencent, post investment companies, cloud customers, cloud service providers, cloud service providers, cloud service providers, cloud service providers, etc It is expected that through the continuous forging of technology and products, tdmesh can continuously add new ideas and practice cases for the development of the industry.