IT industry is going through a new cycle, and the discrete private IT infrastructure is turning to centralized, which is not very new in the history of industrial development. We have experienced different changes in the era of mainframe and PC. from the centralized mainframe to the decentralized personal computer to cloud computing, the development of it seems to follow the pendulum theorem of “negation of negation”.
Database, as the core underlying software of computer architecture, is also in the process of change. Its history can be traced back to 1970. In the past 50 years, database vendors have been in and out, but the status of traditional database vendors has never been shaken until the emergence of cloud computing.
Gartner predicts that by 2023, three quarters of the world’s databases will be running on the cloud. Enterprises are transferring new applications to the cloud, and the requirements for data storage and computing analysis capabilities are increasing. Cloud database naturally has the flexibility on cloud, which can provide strong innovation ability, rich and diverse product systems, cost-effective deployment mode and pay as you need payment mode. Lei Feng talked to Wang Yicheng, product director of Tencent cloud database, and learned about Tencent cloud’s database development ideas in the cloud’s original era.
Tencent cloud database layout
The figure above shows the overall layout of Tencent cloud data products and tools. The top layer is the SaaS layer of the database, including DTS data migration services, supporting mysql, redis, PG and other general open-source protocols. The DMS data management layer helps users realize graphical data management. In addition, it includes data audit, data subscription and so on. The audit supports real-time recording of all addition, deletion, modification and query operations. The subscription can It helps users get real-time incremental update data of cloud database, supports dynamic increase and decrease of subscription objects, online view of subscription data, modification of consumption time points and other features; disaster recovery is used for cross cloud disaster recovery and building hybrid cloud scenarios.
In addition, users only need to submit questions and requirements to obtain professional support from Tencent cloud database experts. Tencent cloud database expert service team has more than 10 years of DBA experience and database management experience of billion level user products. It is understood that the database intelligent management tool launched by Tencent cloud database is about to go online, supporting 7 × 24-hour database intelligent monitoring alarm.
Next is the PAAS layer of database, including relational database, non relational database, distributed database and data warehouse, new generation database. Relational databases include mysql, redis, PostgreSQL, MariaDB, sqlserver, Tdata, non relational databases include redis, Memcache, mongodb, ckv, etc.
Many databases are close to our daily life. Ckv, a non relational database developed by Tencent, is widely used in wechat red packet scenarios. Tcalusdb, which is charged according to the number of visits, is used in the game “glory of the king”. Many game applications or some services derived from Tencent system will use tcalusdb.
Tencent cloud database development history
According to Wang Yicheng, the development history of Tencent database started from QQ, and it was first stored in the form of kV. At that time, redis was not born, then gradually evolved to ckv, and then to caifutong online means Tencent has pure financial business, which is also the embryonic period of MySQL in the open source community. Therefore, Tencent did not choose the traditional commercial database, but built on the basis of MySQL Change.
Of course, it was painful at first, and Tencent was no exception. For example, the single database pressure is too large or the host latency is relatively large, which forces Mysql to evolve into a distributed database, which is today’s tdsql. “The earliest evolution started in Tencent, from single node to cluster version, and then gradually evolved from one master and one standby, and then directly evolved to three nodes. Based on the raft mode, this is the history of Tencent database self-development.”
In addition, there are many data analysis businesses in Tencent, such as advertising and other businesses, which need HTAP database system made of OLTP and OLAP. Based on the transaction data of OLTP and the analysis data of OLAP, we recommend and analyze the whole advertising business.
“Tencent’s overall strategy is also to go to the cloud within the whole group. Tdsql and ckv are originally some products of Tencent’s internal evolution. At present, they are gradually migrating to the cloud based computer room, which is one type. Another type is cloud database, which is completely based on cloud development, such as cynosdb and mongodb. Redis based on open source is also starting to be cloud based within Tencent, and some of them are new The business or the business that did not use the open source database before also began to migrate to the cloud gradually. For example, some game scenes and advertising departments may not have the overall underlying architecture of mongodb or redis, but they also start to use it in the internal cloud environment of Tencent. Wang Yicheng.
Cloud native database: cynosdb
To some extent, an AWS paper has opened the era of cloud native database. AWS believes that the core problem of high-throughput data processing has moved from computing and storage to network io. Aurora proposes a new relational database architecture, which sinks the processing of redo logs to a multi tenant scalable storage service specially customized for Aurora.
The database will enter the next era – cloud native.
In November 2018, Tencent cloud announced a new generation of self-developed cloud native database, cynosdb, which is currently in the public beta stage and will officially launch a commercial version in Q3 this year. The cynosdb database is 100% compatible with the open source database engine MySQL 5.7 and PostgreSQL 10, and can complete the query, application and tool smooth migration of the existing database without changing the code; the highest performance is eight times of MySQL database and four times of PostgreSQL database; the highest 128tb mass storage, no server serverless architecture, and automatic expansion.
Product architecture of Tencent cloud native database cynosdb
In the past, the cloud database is the ability to expand part of the cloud on top of the traditional database, which endows the database with the expansibility of part of the cloud. However, it has not made any changes to the database in essence, but only provides users with some basic operation and maintenance and primary DBA capabilities, while the database like cynosdb takes a pure cloud native route.
The real challenge of the database is at the kernel level. Take MySQL as an example, the traditional MySQL data is written in memory, and then it is swiped into redo. Through redo, the memory page is swiped, and then it is input to the disk. This brings about an obvious problem. To expand the disk, data must be imported, because the whole cache and data are on one machine. Cynosdb separates computing and storage. The redo layer is not swiped in the local computer, but transmitted to shared storage through the network, so that the storage layer can sense MySQL redo logs and play back the logs in the storage layer. Calculation is only responsible for calculation, no longer responsible for storing data.
“Why do you make cynosdb? The traditional database + cloud model has not brought many surprises to users. We believe that the future direction is “cloud + Database”, and the database will undergo fundamental changes. Cloud native, super fusion and autonomy are the three carriages leading the development of Tencent cloud database. Cynosdb is the product of the cloud’s primitive age. “
Wang Yicheng vividly compares that traditional databases, such as SQL server or cloud + database, are similar to buying broadband. In the field of cloud computing, they are instances. No matter how much and how they are used, they need to be rented, but the actual use level may be much smaller than the purchase.
“Cynosdb is a change in the business model, but the change in the business model must be a technological architecture or a technological transformation. In the past, it was based on the model of physical gene to do an example. When users upgrade or move, they need to move the data completely. Cynosdb realizes the separation of storage and calculation. Storage is a large shared disk, and enables disk storage to sense MySQL or database protocol, and then sink the log, which has a good neutralization on storage scalability and performance. “
Cloud native database market breaks out in two or three years
“Cloud native database market may break out in the next two or three years”, Wang Yicheng gives two reasons.
First, the incremental market of public cloud may not be as fast as before, reaching the previous 100% and 200%. In this market condition, cloud native database more caters to the market demand, and cloud manufacturers need to gradually educate the market.
On the other hand, the technology monopoly of foreign database manufacturers is gradually broken, and the rising momentum of domestic database is expected to further develop. The database market is about to start the second half of the competition, and cloud manufacturers are the new variables that affect this campaign. It’s a long process to compare with Oracle and achieve Oracle, which is not realistic at present.
How to use the understanding of database market, or the advantage of database technology, to impress the market customers worth 2-4 billion yuan, is a real problem for database manufacturers.
This article is reprinted from Zhang Shuai of Lei feng.com