“Not every cloud can support this traffic. There are two clouds in China, one is alicloud and the other is other cloud. ” On the evening of November 11, Zhang Jianfeng, CTO of Alibaba group, said, “Alibaba cloud is not the same. 10 years ago, we started from the first line of code to build the only cloud operating system in China, Feitian, which is developed by ourselves.”
In 2019, tmall double 11 set a new world record again, with the peak of order innovation reaching 544000 transactions / second and the daily data processing capacity reaching 970pb. This year, Alibaba’s core system went to cloud 100%, supporting the world-class traffic peak of double 11.
“Alibaba is a high-speed aircraft in flight, and we have successfully replaced it with a new engine in the process.” Zhang Jianfeng said that Alibaba cloud is the first one to use public cloud to host such a trillion scale core system. “A lot of cloud vendors’ own business systems are not on their own cloud, and all Alibaba systems will be on Alibaba cloud in the future.”
Alibaba is the first large Internet company in the world to run 100% of its core systems on the public cloud.
Appendix: core points of Zhang Jianfeng’s speech
Since last year, we have decided to put all the core systems of Alibaba on the cloud. To do so, we need to add an attribute, which is “Alibaba cloud”. Not every cloud can support this traffic. In China, we often say that there are two clouds in China, one is called “alicloud”, and the other is called “other clouds”.
Why is alicloud different from other clouds? Because in China, only Alibaba cloud is a cloud that has been completely developed from scratch, starting from every line of code, starting from ten years ago, and this year is the 10th year of Alibaba cloud, so our cloud has a special name, called “flying sky operating system”. When Dr. Wang Jian founded Alibaba cloud, he took the It’s called “flying operating system”.
Our cloud is completely a cloud developed by China. Many other clouds have been transformed from open source software, which is very different.
Second, this year’s double 11, Ali is still a high-speed flying plane. We have successfully replaced a new engine in this process. In the past, we put non core load on the cloud. Now Alibaba’s core system is on the cloud. Now our cloud turns the original dedicated technology into a public cloud, and everyone can enjoy inclusive services. We have the best system, and you can use it just like us.
Alibaba cloud carries 100% of Alibaba’s own core system, which is the first thing we have done in the world. Many cloud vendors have their own business systems, but their own systems are not on their own cloud. All of our future Alibaba systems are on Alibaba cloud’s Apsara operating system.
You will surely think, is this a simple replacement or a great progress and challenge in technology? Let me simply say that we have done a good job in several aspects. It is not a simple replacement. After going to the cloud, the performance has been greatly improved.
Today, you may have a feeling that in the first ten minutes, basically, the consumer didn’t feel any jitter, and the shopping was very smooth.
First, on the core virtual machine system, we research the Dragon architecture and use the self-developed server for virtualization. With the increase of the pressure, the load capacity of the general server will decrease gradually, which is not linear. The greater the pressure on our dragon server, the more linear the output is, which is very difficult to achieve.
Second, we have developed our own cloud native database. This year, there are two databases. One is oceanbase, a self-developed database. As you know, we won the first place in the world in TPC-C test. The second is polardb, which is also widely used in this double 11. There is no problem. Our peak value per second is far higher than the traditional database like oracle.
Third, our computing and storage are separated. Now Alibaba storage has a place to store data. Data is accessed from the far end, so storage can be easily expanded because it has a special pool. It couldn’t be done. Because the original remote storage, access network speed can not keep up.
Fourth, the reason why we are able to do remote storage faster than local read-write disks is that we have made RDMA network, which is the first company in the world to do RDMA network on a large scale.
We have made core breakthroughs in all four aspects.
Today, the flying sky operating system can schedule in 100000 or more servers. Today, there are many applications in double 11, but it doesn’t mean that every time point, all application loads are the same. We allocate most of our computer resources to applications such as trading at zero. After a little bit, we allocate data to data processing systems. Data processing reached a new high this year. In 2017, 300pb of data was processed, in 2018, 600pb of data was processed, and this year, about 970p of data will be processed.
You may not have a concept. How much data does a P have? Last time I talked with CCTV people, CCTV has made TV news programs for so many years, and he has saved about 80p data for decades. On the day of double 11, we have to deal with 970p data, which is very large. No advanced system can support it.
The second is very real-time. As you can see, today in seckill, the venue is all personalized and full of thousands of people. The data here is not only large, but also very real-time. So far, our rookie logistics system has generated more than 1 billion logistics orders, and this data is still rising rapidly, which all depends on the large amount of computing power behind Alibaba cloud.
Our data of this year, in addition to batch processing, also has stream processing, which is to process all data in real time. Like the transaction data you see, it changes every minute and every second. It is not calculated from the database. It is collected by the system automatically layer by layer for each order generated. Our system can process 2.5 billion records per second. This is a streaming system.
We have so many servers. To manage all of them, we need to manage all the message flows besides the flying sky system. So we have developed a message system called MQ, which is also the largest message system in the world.
Today, from APStar system, big data processing platform to intelligent application, these technologies add up to create a new distributed cloud based platform, which enables all core applications of Ali’s entire economy to run on it.
Finally, we released the self-developed chip containing light 800 this year. Next year’s double 11, a large number of Ali’s AI applications will run on self-developed chips.
Today, from the flying cloud operating system to the Dragon server, database, switch, switch operating system and RDMA network, all are developed by Alibaba. Today, we have accumulated very rich and strong capabilities, from hardware, database, cloud computing operating system to the core application platform above, which is the biggest difference between the double 11 and the previous year.
All the unthinkable will turn into the ordinary;
We believe in “believe” and everything is new.
Happy double 11!
Read the original text
This article is from alitech, a partner of yunqi community. If you need to reprint it, please contact the original author.