Introduction:Embracing “convergence” is the new direction of big data evolution. Data + ⼈⼯ intelligence ⼀ integration is the inevitable trend of data development, and artificial intelligence is the ultimate trend of data business development.
On December 13, Jia Yangqing, head of Alibaba cloud computing platform, pointed out at the Flink forward Asia 2020 summit that embracing “convergence” is a new direction of big data evolution, the integration of big data and artificial intelligence is an inevitable trend, and artificial intelligence is the ultimate export of big data business development. At the same time, he discussed the open source technology of Flink from different angles, as well as the development and future thinking of open source technology in the cloud era.
Flink open source community “getting better”
From the perspective of an ordinary open source technology enthusiast, Jia Yangqing defines the development of Flink in 2020 as “getting better”, which is mainly reflected in three aspects:
Technology. This year, Flink has made two new breakthroughs in technology: one is the successful application of Flink stream batch integration technology in the scale of Alibaba shuang11 core data business scenario; Second, the peak value of Flink real-time computing exceeded 4 billion pieces / second for the first time. Compared with last year, the computing performance is doubled without increasing the cluster size.
Application. Flink is no longer a high technology, but a de facto standard of open source streaming processing, flying into “ordinary people’s home”. For example, China Institute of information technology uses Flink as the technical blueprint to design streaming processing standards, and cloud computing companies, including Alibaba cloud, use Flink as the underlying engine of streaming processing.
Community. The Flink open source community is growing rapidly, and it is the most active mailing list of Apache; The Flink project is one of the top Apache projects submitted by GitHub the most times. Last year, the number of participants of Flink forward Asia reached 2000, and the Flink geek challenge attracted 4000 + developers; This year, the conference will be held online and offline at the same time. The awards of Flink geek challenge show us more and more developers’ enthusiasm for open source technology and the spirit of challenging.
Open source makes cloud more standardized
Jia Yangqing thinks: open source makes cloud more standardized. He mentioned that cloud computing is becoming more and more standardized, which is very beneficial to its own development. A large number of development technology systems on the cloud are becoming more and more popular, which means that a large number of cloud computing developers who are familiar with this system are more likely to turn the “application” into the “part” of the cloud.
Open source helps cloud break the existing technology system and form a standard, open and public technology system on the basis of fully respecting the original habits of cloud users. It is often difficult to set standards by one company alone.
What does alicloud mean for open source? Jia Yangqing mentioned the following three points:
fromorganizationLast year, Abba group became the open source technology committee to sort out and promote the implementation of Abba’s open source technology strategy from the group level.
fromParticipationFrom the beginning, Abba group has widely used open source technology to build its technology system. Take the data products of cloud computing platform business unit as an example, such as the technology that real-time computing Flink is in the state of Hadoop / k8s; Machine learning Pai also enables open source deep learning frameworks such as tensorflow.
fromContributionIn view of this, Ababa Group continues to contribute to the open source community. Take the Flink technology community as an example. Last year alone, cloud contributed millions of lines of code to the Apache Flink community. At present, the group has more than 1000 open source projects, covering data, AI, cloud native, database, middleware, hardware and other fields. More than 700000 developers around the world have lit GitHub star for Alibaba, and thousands of fans have participated in the project.
Embracing “convergence” is the new direction of big data evolution
Jia Yangqing said that the three development trends of the cloud are mainly the cloudization of IT infrastructure, the interconnection of core technology, and the digitalization and intellectualization of applications, which are also the core value of the cloud.
After the interconnection of cloud and business in it, enterprises should change from process driven to data intelligent driven to realize data and intelligence. Cloud computing and the underlying IAAs technology compete for the maturity, stability and price of infrastructure, while the upper PAAS technology requires data capability and intelligent capability.
Before, pan data, including database and IOT, are still in the most basic enabling mode of data, and need to integrate more data and AI capabilities.
Therefore, embracing “convergence” is the new direction of big data evolution. Data + ⼈⼯ intelligence ⼀ integration is the inevitable trend of data development, and artificial intelligence is the ultimate trend of data business development.
Only with AI, can data play its real business value. The reason why cloud data has core and competition is that data is combined with AI energy, which is the data of intelligent upgrading. For example, Flink + alink can not only provide the processing capability of streaming batch agent, but also realize the data + intelligence of Flink with the help of the open source alink machine learning algorithm library.
In the future, the social economy will be more integrated with the Internet, and the business of enterprises will also be transformed into the Internet business. It is not enough for an enterprise to only have it infrastructure. The enterprise should also change from process driven to data intelligent driven. The cloud computing platform will output full stack data technology capabilities, and provide the world’s top AI technology to realize data and intelligence together.
Author: alicloud real time computing Flink
This article is the original content of Alibaba cloud and cannot be reproduced without permission