[paper reading]

Which Is Plagiarism: Fashion Image Retrieval Based on Regional Representation for Design Protection

By Ying Lang, yuan he, Fan Yang, Jianfeng Dong, Hui Xue

Unit: Ali; Zhejiang Business University; azft

Meeting cvpr2020

Paper address

summary

In the field of clothing, although the crackdown on counterfeit goods has been continuous, the problem of piracy and plagiarism is still widespread, and from the online and offline, the means of plagiarism are becoming more and more sophisticated. Look at the following three areas of clothing plagiarism

•Picture embezzlement

The cost of plagiarism is very low, and it is easy to be locked by the image retrieval system of the platform

•Creative piracy

The cost of plagiarism is slightly higher, but the algorithm based on similarity measurement can recall and govern them

•Modify some areas of clothing

The cost of plagiarism is high, which needs to be checked manually, and the cost of cracking down on counterfeiting is also high

There are two groups of piracy examples, in which the picture on the left of each group is authentic clothing, and the picture on the right is pirated clothing

Difficulties in retrieving pirated clothing

The form of pirated clothing emerges in endlessly. Some pirated clothing is similar to the original, but some are not

And some pirated clothing and original clothing belong to different types, which improves the requirements of network training

Definition of pirated clothing

As the first work in the field of pirated clothing retrieval, the author defines pirated clothing as plagiarizing the original clothing design and style as a whole, and the number of local areas of clothing modification is less than or equal to 2

The clothing in the image is divided into five areas, including collar, chest, waist and two sleeve areas

method

Loss function based on triples (for similarity retrieval)

$$
\begin{array}{c}
\mathcal{L}_{t r i}\left(I, I^{+}, I^{-}\right)=\sum_{r=1}^{R} \max \left(D_{r}^{I, I^{+}}-D_{r}^{I, I^{-}}+m, 0\right) \\
\mathcal{L}_{t r a}=\sum_{n=1}^{N} \mathcal{L}_{t r i}\left(I, I^{+}, I^{-}\right)
\end{array}
$$

Loss function based on triplet

$$
\begin{array}{c}
\mathcal{L}_{t r i}^{\prime}\left(I, I^{+}, I^{-}\right)=\sum_{r=1}^{R} \max \left(D_{r}^{I, I^{+}}-D_{r}^{I, I^{-}}+m, 0\right) \cdot \lambda_{r} \\
\alpha_{t r i}=\frac{\operatorname{avg}\left\{\left\|f_{r}(I)-f_{r}\left(I^{+}\right)\right\|_{2} ; r=1,2, \ldots R\right\}}{\max \left\{\left\|f_{r}(I)-f_{r}\left(I^{+}\right)\right\|_{2} ; r=1,2, \ldots R\right\}} \\
\mathcal{L}_{p l a}=\sum_{n=1}^{N}\left[\mathcal{L}_{t r i}^{\prime}\left(I, I^{+}, I^{-}\right) \cdot \alpha_{t r i}\right]
\end{array}
$$

Network framework

Overall framework of PS net

Network Backbone

Feature extraction of images by HR net

The parallel connection of multi-resolution subnets in HR net makes every representation from high-resolution to low-resolution receive information repeatedly from other parallel representations, thus obtaining rich high-resolution representations

However, HR net is not necessary and can be replaced by RESNET and VGg net

Landmark Branch

Then, the key points are divided into two parts

Retrieval Branch

Aggregate the features of local regions for retrieval

According to the prediction of key points and the output thermal map of landmark branch, the position of specific local area on the characteristic map is obtained
Then, according to the location of a specific region on the feature map, the corresponding local feature map of the region in the feature map of retrieval branch is obtained by ROI pooling

Playarized fashion dataset

· a total of 60000 images, 40000 for training and 20000 for testing

· includes 4 types of clothing: short sleeve T-shirts, long sleeve tops, coats and dresses

The pictures were crawled from Taobao and labeled by experts

summary

A new retrieval problem of plagiarized clothing is proposed

A new dataset, plagiarism fashion, has been established for the retrieval of plagiarized clothing

This paper proposes a multi task network PS net based on region representation and achieves SOTA

PS net can also be used for traditional clothing retrieval and key point estimation tasks