CIKM 2020 | detailed explanation of six selected papers of meituan

Time:2021-4-29

CIKM is the top international academic conference in the fields of information retrieval, knowledge management and database. Since 1992, CIKM has successfully gathered first-class researchers and developers in the above three fields, providing an international forum for the exchange of the latest development of information and knowledge management research, data and knowledge base. The purpose of the conference is to identify the challenges and problems that will be faced by the future development of knowledge and information systems, and to determine the future research direction by collecting and evaluating the top research results with strong applicability and theory.

This year’s CIKM conference was originally planned to be held in Galway, Ireland, in October. Due to the epidemic situation, it was held online instead. Meituan AI platform / search and NLP Department / NLP Center / knowledge mapping group has six papers (including 4 long papers and 2 short papers) which have been approved by the international conferenceCIKM 2020receive.

These papers are the results of scientific research cooperation between meituan knowledge mapping group and Xi’an Jiaotong University, University of Chinese Academy of Sciences, University of Electronic Science and technology, Renmin University of China, Xi’an University of Electronic Science and technology, Nanyang University of technology, etc. they are the technical precipitation and application of multimodal knowledge mapping, mt-bert, graph embedding and map interpretability. I hope these papers can help more students learn and grow.

01 《Query-aware Tip Generation for Vertical Search》

|This paper is a cooperative paper between meituan knowledge mapping group and Hao Junmei of Xi’an Jiaotong University, Li canjia of University of Chinese Academy of Sciences and Wang Zili of Xi’an University of Electronic Science and technology.

Thesis download

Interpretable reason (also known as recommendation reason) is a natural language text displayed to users for highlight recommendation in search results page and discovery page (scene decision, must eat list, etc.), which can be regarded as a highly condensed real user comment, explaining recall results for users, mining merchant characteristics, attracting users to click, and providing scene guidance for users, It can help users to make decisions and optimize the user experience in the vertical search scenario.

CIKM 2020 | detailed explanation of six selected papers of meituan

Most of the existing text generation work does not consider the user’s intention information, which limits the implementation of generative recommendation reasons in scenario search. This paper proposes a query aware recommendation reason generation framework, which embeds user query information into the coding and decoding process of the generation model, and automatically generates personalized recommendation reasons suitable for different scenarios according to different user queries. In this paper, transformer and RNN are reformed respectively. Based on transformer structure, this paper introduces query information by improving self attention mechanism, including introducing query aware review encoder in encoder to make query related information be considered in the initial stage of comment coding, and introducing query aware tip decoder in decoder to make query related information be considered in the final stage of comment coding. Based on RNN structure, in the encoder side, the irrelevant information of query is filtered through the selective gate method, and the information related to query in the original comments is selected for encoding. In the decoder side, the query representation vector is added to the context vector calculation of attention mechanism to guide the decoding process, which solves the problem of uncontrollable decoding of the generation method to a certain extent, So as to generate personalized recommendation reasons.

CIKM 2020 | detailed explanation of six selected papers of meituan

Experiments are carried out on public data sets and meituan business data sets respectively, and the proposed method is superior to the existing methods. The algorithm proposed in this paper has been applied online, and has been implemented in many scenes such as search, recommendation, category selection and list of meituan.

CIKM 2020 | detailed explanation of six selected papers of meituan

02 《TABLE: A Task-Adaptive BERT-based ListwisE Ranking Model for Document Retrieval》

|This paper is a cooperative paper between meituan knowledge mapping group and Tang Hongyin, Professor Jin Beihong of Institute of software, Chinese Academy of Sciences.

Thesis download

In recent years, in order to improve the natural language understanding ability of models, more and more MRC and QA datasets have emerged. However, there are more or less some defects in these datasets, such as insufficient data, relying on manual construction of query and so on. To solve these problems, Microsoft proposes a reading comprehension dataset MS Marco (Microsoft machine reading comprehension) based on large-scale real scene data. The data set is generated based on real search queries in Bing search engine and Cortana intelligent assistant, including 1 million queries, 8 million documents and 180000 manually edited answers.

Based on MS Marco dataset, Microsoft proposes two different tasks: one is to retrieve and sort the documents in all datasets for a given problem, which belongs to the task of document retrieval and sorting; The other is to generate answers according to the questions and the given related documents, which belongs to QA task. In meituan business, document retrieval and sorting algorithms are widely used in search, advertising, recommendation and other scenarios. In addition, the time consumption of QA task directly on all candidate documents is unacceptable. QA task must rely on sorting task to select the top ranked documents, and the performance of sorting algorithm directly affects the performance of QA task. Based on the above reasons, we mainly focus on the task of document retrieval and sorting based on MS Marco.

Since the release of macro document sorting task in October 2018, it has attracted many enterprises and universities including Alibaba Damo college, Facebook, Microsoft, Carnegie Mellon University, Tsinghua University, etc. On the mt-bet platform of meituan, we propose a kind of bet algorithm for this text retrieval task, which is called table. It is worth noting that the table model proposed in this paper is the first model with more than 0.4% in the authoritative evaluation of Microsoft Marco in the field of information retrieval.

CIKM 2020 | detailed explanation of six selected papers of meituan

As shown in the figure above, this paper proposes a document retrieval model table based on Bert. In the pre training stage of table, a domain adaptive strategy is used. In the fine-tuning stage, this paper proposes a two-stage task adaptive training process, namely query type adaptive pointwise fine-tuning and list fine-tuning. Experiments show that this task adaptive process makes the model more robust. This work can explore richer matching features between queries and documents. Therefore, this paper significantly improves the effect of Bert in document retrieval task. Then on the basis of table, we propose two methods to solve OOV (out of vocabulary) mismatching: precise matching method and word reduction mechanism, which further improve the effect of the model. We call the improved model dr-bert. Details of dr-bert can be found in our technology blog:《Practice of mt-bert in text retrieval task》。

03 《Multi-Modal Knowledge Graphs for Recommender Systems》

|This paper is a cooperative paper between meituan knowledge mapping group and Tang Hongyin, Professor Jin Beihong of Institute of software, Chinese Academy of Sciences.

Thesis download

With the development of knowledge mapping technology, its structured data has been successfully applied in a series of downstream applications. In the direction of recommender system, structured graph data can use more comprehensive auxiliary information of target products to spread information through graph Association, so as to effectively represent and model the target products, and alleviate the problems of sparse user behavior and cold start in recommender system. In recent years, many researches have successfully combined the map data with the recommendation system by using the map path feature and the representation learning based on graph embedding, which improves the accuracy of the recommendation system.

In the existing work of the combination of atlas and recommender system, people often only focus on the graph nodes and node relationships, and do not use the data of each mode in the multi-modal knowledge atlas for modeling. Multimodal data includes image mode, such as stills of movies, text mode, such as comments of merchants, etc. These multimodal data can also be spread and generalized through the knowledge graph relationship, and bring high value information to the downstream recommendation system. However, due to the fact that multi-modal knowledge modeling is often the auxiliary information relationship of different modes, rather than the semantic relationship represented by the triples in the traditional graph, the traditional graph modeling method can not model the multi-modal knowledge graph well.

Therefore, aiming at the characteristics of multi-modal knowledge map, this paper proposes mkgat model, which is the first time to use the structured information of multi-modal knowledge map to improve the prediction accuracy of downstream recommendation system. The overall model framework of mkgat is shown in the following figure:

CIKM 2020 | detailed explanation of six selected papers of meituan

In the mkgat model, the embedded representation learning of multimodal atlas is mainly divided into three parts: 1) firstly, we use the mkg entity encoder to encode different types of input data (image, text, label, etc.) into high-order hidden vectors; 2) Next, we use the nodes around the entity node (including multimodal and entity nodes) to provide the corresponding information for the description of the node based on the attention layer of multimodal graph; 3) After the attention mechanism is used to synthesize the multimodal information, the traditional H + r = t training method is used for graph embedding representation learning.

When accessing to the downstream recommender system model, we also reuse the multi-modal entity coding and multi-modal graph attention mechanism module to represent the target entity and access to the recommender system model. Through the above methods, we conducted detailed experiments on meituan’s food search scenario and open dataset movielens. The results show that mkgat significantly improves the quality of the recommendation system in these two scenarios.

04 《S^3-Rec: Self-Supervised Learning for Sequential Recommendation with Mutual Information Maximization》

|This paper is a cooperative paper between meituan knowledge mapping group and Zhou Kun, Wang Hui, Zhu Yutao, Zhao Xin and Wen Jirong of Renmin University of China.

Thesis download

Sequence recommendation refers to the use of users’ long-term interaction history sequence to predict users’ future interaction products, which enhances the accuracy of recommendation to users by modeling sequence information. The existing sequential recommendation models use the task of commodity prediction to train the parameters of the model, but it is also limited by the only training task, which is easily affected by the problem of data sparsity; Although it optimizes the final recommendation target, it does not fully model the potential relationships in the context data, let alone use this part of information to help the sequence recommendation model.

CIKM 2020 | detailed explanation of six selected papers of meituan

In order to solve the above problems, this paper proposes a new model s ^ 3-rec, which is based on self attention network structure and adopts self supervised learning strategy for representation learning, so as to optimize the serialization recommendation task. The model is based on four special self-monitoring tasks, which learn the potential relationships among attributes, commodities, self sequences and original sequences. Because the above four kinds of information represent four different information granularity perspectives of input data, this paper uses mutual information maximization strategy to model the potential relationship of the four kinds of information, and then strengthens the representation of this kind of data. In this paper, a large number of experiments are carried out on six real datasets including meituan scenes to prove the superiority of the proposed method over the existing sequence recommendation advanced methods, and the model can still maintain good performance in limited training data scenes.

CIKM 2020 | detailed explanation of six selected papers of meituan

05《Leveraging Historical Interaction Data for Improving Conversational Recommender System》

|This paper is a cooperative paper between meituan knowledge mapping group and Zhou Kun, Wang Hui, Zhao Xin and Wen Jirong of Renmin University of China.

Thesis download

In recent years, session recommendation system has become an important research direction, and it has many applications in real life. A session recommendation system needs to be able to understand the user’s intention through the dialogue with the user, and then give appropriate recommendations, so it contains a session module and recommendation module. The existing session recommendation systems usually complete the recommendation based on the learned user representation, which needs to encode the conversation content. But in fact, it is difficult to accurately predict the user’s preference information only using conversation data. This paper hopes to help complete the recommendation by using the user’s historical interaction sequence.

CIKM 2020 | detailed explanation of six selected papers of meituan

Based on this assumption, the session recommendation system needs to consider the user’s historical interaction sequence and session data at the same time. This paper proposes a new pre training method, which combines the merchant preference sequence (from historical interaction data) and the merchant attribute preference sequence (from conversation data) through the pre training method, so as to improve the effect of the session recommendation system. In order to further improve the performance, this paper also designs a negative sample generator to generate high-quality negative samples to help training. Experiments on two real datasets show that the method is effective to improve the session recommendation system.

CIKM 2020 | detailed explanation of six selected papers of meituan

06 《Structural relationship representation learning with graph embedding for personalized product search》

|This paper is a cooperative paper by meituan knowledge mapping group, Liu Shang and Cong Gao of Nanyang University of technology.

[Paper Download](https://dl.acm.org/doi/abs/10…
)

Personalization is very important in product search, and the user’s preference affects the user’s purchase decision to a great extent. For example, when a young user searches for a “loose T-shirt” on the e-commerce platform, he is more likely to buy a fashion style or shirt that he is interested in and has a brand. The purpose of personalized product search (PPS) is to generate user specific product suggestions for a given query, which plays an important role in many e-commerce platforms.

In this work, we use the logical structure representation learned from user query commodity to naturally preserve the cooperation signals and interaction information between users / queries / commodities on the logical path, so as to improve the personalized commodity search. We call these logical structures “constructive graph patterns”. For example, as shown in Figure 1, there are three key patterns. Note that when a branch has three or more branches, we can randomly sample two of them and get the following pattern:

CIKM 2020 | detailed explanation of six selected papers of meituan

Specifically, we propose a new method: graph embedding model based on logical structure representation learning (graphlsr). The conceptual advantage of graphlsr is that it is an embedded framework, which can effectively learn the representation of logical structure and the approximate relationship of user (query or product) in geometric operation, and integrate it into personalized product search. The key idea behind it is that we have learned how to embed three types of connection graph patterns into low dimensional space to enhance personalized product search. The framework is shown in Figure 2. It consists of two main components: graph embedding module and personalized search module. The graph embedding module at the bottom of Figure 2 uses the designed three connection graph patterns to learn the embedded nodes for logical representation learning, which is also convenient for learning the similarity between users (queries or commodities). Then the presentation information is introduced into the personalized search module.

The personalized search module takes the user, query, commodity and representation learned from graph embedding as input, and uses MLP to integrate the corresponding information. The extracted short and dense features of users, queries and commodities are input into MLP network to learn user specific query representation and user specific commodity representation. Then we input them into another MLP to calculate the probability score of prediction.

CIKM 2020 | detailed explanation of six selected papers of meituan

Table 3 compares graphlsr with four personalized search methods in the MRR, MRR and MRR of personalized product search task [email protected] and [email protected] Performance in the following aspects:

CIKM 2020 | detailed explanation of six selected papers of meituan

summary

The above is some research work done by the knowledge mapping group of search and NLP Department on multimodal knowledge mapping, mt-bert, graph embedding and map interpretability. The results of this paper are also the specific problems we encounter and solve in the actual work scenario. Most of the work has been carried out in the actual business scenarios, such as content search, product search, recommendation reasons and so on, And achieved good business income. Meituan AI platform / search and NLP center has been committed to transforming academic achievements into technological productivity through the combination of production and research. At the same time, more people with lofty ideals are welcome to join our team.

If you want to read more technical articles, please pay attention to the official WeChat official account of meituantech. Special purchases for the Spring Festival [official account] special purchases for the Spring Festival [special purchases for the Spring Festival], [2018], [2017 goods], and algorithm [2019], can be seen in the technical team of the US team.

CIKM 2020 | detailed explanation of six selected papers of meituan