Chen Dihao: large scale feature engineering of recommendation system and spark llvm based optimization

Time:2021-7-31

Recommended Systems InstituteThe fourth paradigm is a series of courses for recommendation system enthusiasts to share the research and application of recommendation system.

As the most popular big data processing framework, spark is widely used in machine learning scenarios and recommendation systems.The fourth paradigm optimizes the spark offline computing engine based on llvm, supports the newly released spark 3, and completely solves the problems of online failure and efficiency of spark applications in terms of function and performance.

Theme of this issue

This time, we will mainly share how the fourth paradigm optimizes spark offline computing engine based on llvm.

You can:

-Understand the implementation scheme of large-scale feature engineering of recommendation system

-Learn about spark execution plan optimization based on llvm

Practical knowledge points:

-Spark / LLVM

Introduction to the speaker

Chen Dihao

The fourth paradigm is the platform architect of prophet, who is responsible for the production of deep learning framework and the development of next-generation feature engine. Actively participated in the development of open source communities tensorflow, kubernetes, TVM and other projects, and had a certain understanding of distributed systems and deep learning platforms. At present, it focuses on the development of feature engine for offline and online consistency.

This course is best for you

Recommendation system college is for students interested in recommendation system technology. Students who meet the following conditions are most suitable to study:

  • If you want to improve your ability, get a promotion and a raise

If you want to improve your professional and technical ability, you must start from cultivating the ability to solve core problems. What we don’t rectify is dry goods. You must gain something in an hour.

  • If you want to achieve self-worth and self-improvement

“Code out the future” is not empty talk. The development of artificial intelligence technology has taken root in all walks of life and changed the future of mankind at a speed visible to the naked eye. If you want to study in the field of recommendation system and complete the realization of self-worth, you can study our course carefully and I believe it will accelerate the arrival of this day.

  • If you want to make contacts, improve the depth of technical discussion

Communicate more with excellent people, and you will become more and more excellent. There are many like-minded partners in the industry lurking in major Internet companies, and you don’t even know them? Come and study with them, make like-minded friends and technical experts, and establish your own high-quality academic circle.

It should be reminded that the theme of this issue is highly professional and technical, which is more suitable for Algorithm Engineers and senior machine learning enthusiasts with a certain foundation. If you want to know about this industry, you are also welcome to come to the live studio on time.

Chen Dihao: large scale feature engineering of recommendation system and spark llvm based optimization