On December 20, Beijing time, Linux foundation officially announced to LF AI foundation, the top foundation in AI field, that Tencent’s open source project Angel graduated from lf AI foundation, which is also the first open source project graduated from lf AI foundation in China. This means that angel has been recognized by global technical experts and has become one of the world’s top AI open source projects.
Ibrahim Haddad, executive director of LF AI foundation, said: “in the process from incubation to graduation, we can see angel’s amazing speed in improving new functions and landing application scenarios. With the release of version 3.0, angel has made outstanding achievements in function, applicability and contribution to the community. This is also a big step for the whole AI open source community to be mature and have industrial production capacity. “
Angel is Tencent’s first AI open source project, which was launched at the end of 2016 and opened in 2017. As the third generation of high performance computing platform for machine learning, angel is committed to solving the problems of sparse data large model training and large-scale graph data analysis. Tencent became one of the founding Platinum members of LF AI foundation in 2018, and contributed the open source project angel to the foundation in the same year.
Jiang Jie, vice president of Tencent cloud and general manager of Tencent’s data platform department, said: “Tencent is committed to building an open source ecosystem and sharing results with developers through open source code, and has become the most comprehensive open source company in the field of big data. We are very glad to see Angel graduated from lf AI foundation successfully. Under the guidance of foundation and the co construction of developers, we have completed the technological leap from a single model training platform to a full stack machine learning platform, which is also closely consistent with the future development direction of Tencent big data batch flow integration and ABC integration. Graduation is angel’s new start. In the future, we will further open angel’s core capabilities in the field of graph computing and federated learning. “
According to Xiao Han, director of LF AI foundation and Tencent AI expert, lf AI foundation has very strict regulations on the graduation process of open-source projects. Based on the technical content, open-source ecology, community interaction and other dimensions of the project, it strictly evaluates the maturity of the project, and finally the board of directors votes whether to grant graduation. Being able to graduate from lf AI means that the project has been recognized by global technical experts and become the top AI open source project.
Angel has achieved rapid development in the foundation’s incubation process, and completed the leap from version 2.0 to version 3.0. It has evolved from a simple model training system to a full stack machine learning platform including automatic feature engineering and model services.
LFAI & Angel
Angel’s feature engineering module is developed based on spark, which enhances the feature selection function of spark, and realizes automatic feature generation using feature cross and re index. These components can be seamlessly integrated into Spark’s pipeline. In order to make the whole system more intelligent, angel 3.0 added the function of super parameter adjustment.
In terms of model services, angel 3.0 provides a cross platform component, angel serving, which can not only meet angel’s own needs, but also provide model services for other platforms. In terms of ecology, angel also tries to share the parameter server (PS) capability with other computing platforms. At present, two platforms, spark on angel and pytorch on angel, have been completed.
Figure computing architecture
In version 3.0, angel focuses on the research and development of graph machine learning function, including graph representation and graph neural network learning model. It also provides end-to-end data processing capabilities such as preprocessing and graph mining. It has rich algorithm library, provides multiple graph computing paradigms and different kinds of graph algorithms for isomorphic graph and heterogeneous graph. Through the large-scale practice of Tencent’s internal business, angel also excels in the performance of graph algorithm, such as the graph structure with the scale of one billion nodes and one hundred billion sides. When running the algorithm, it can achieve ten times of processing performance with one third of the computing resources of spark graphx.
Figure calculation performance comparison
In addition to the improvement of technical functions, angel is becoming more and more mature in community ecology. It is understood that angel has more than 100 companies and institutional users in the whole industry, including microblog, Huawei, microbank, Xiaomi, didi and other large Internet enterprises. The intelligent titanium Ti, a one-stop machine learning application platform based on angel, supports many Tencent internal products including wechat payment, Tencent advertising, micro vision, etc., and also provides services for more industry enterprises through Tencent cloud opening up.
So far, angel has obtained 5500 star and 1400 fork in GitHub, and its outstanding performance in technology, application and ecology has been recognized by the open source community. Previously, at Tencent’s 2019 techno developer conference, Jiang Jie also announced the official open source of resource management platform core tke and distributed database tbase. Tencent is becoming the most comprehensive open source company in the field of big data.
Open source collaboration is one of Tencent’s most important technology strategies. At this year’s Chongqing smart Expo, Ma Huateng, chairman of the board of directors and chief executive officer of Tencent, specifically mentioned the importance of open source. He said that Tencent hopes to invest more efforts in the field of scientific research, and will actively participate in the construction of the “global science and technology community” through internal and external open source and other ways.
As of November 2019, Tencent has opened 91 projects on GitHub alone, with more than 1000 contributors participating and more than 260000 stars, ranking the top of GitHub’s global corporate contribution list. Tencent also joined nine open-source foundations, including Linux and Apache, and became the highest level member. LF AI foundation is a special open-source foundation in the field of artificial intelligence initiated by Linux foundation. It aims to advocate and organize AI open-source platforms and tools to participate in the construction, share IP intellectual property rights, and create a sustainable open-source AI ecosystem.
In 2018, Tencent was invited to join as the initial platinum member of LF AI foundation due to its outstanding technical influence in the field of AI open source, with seats on the board of directors and technical advisory committee, and donated high-performance distributed machine learning platform angel to the foundation in the same year.