Fsaf: embedding anchor free branch to guide acnor based algorithm training | cvpr2019

Time:2021-7-6

Fsaf deeply analyzes the selection problem of FPN layer in training, and embeds it into the original network in the form of super simple anchor free branch, which almost has no effect on the speed. It can select the optimal FPN layer more accurately and bring good accuracy improvement

Source: Xiaofei’s algorithm Engineering Notes official account

Paper: feature selective anchor free module for single shot object detection

Fsaf: embedding anchor free branch to guide acnor based algorithm training | cvpr2019

Introduction


The first problem of target detection is size change. Many algorithms use FPN and anchor box to solve this problem. In the positive sample judgment, the FPN layer for prediction is generally determined according to the size of the target, and the larger the target is, the higher the FPN layer is used, and then the further judgment is made according to the IOU of the target and anchor box. However, such a design will bring two limitations: the feature selection based on head beating and the anchor sampling based on IOU.

Fsaf: embedding anchor free branch to guide acnor based algorithm training | cvpr2019

As shown in Figure 2, 60×60 chooses the middle anchor, while 50×50 and 40×40 choose the smallest anchor. The choice of anchor is based on people’s experience, which may not be the optimal choice in some scenarios.

Fsaf: embedding anchor free branch to guide acnor based algorithm training | cvpr2019

In order to solve the above problems, this paper proposes a simple and efficient feature selection method fsaf (feature selective anchor free), which can select the optimal layer for optimization in each round of training. As shown in Figure 3, fsaf adds anchor free branches to each layer of FPN, including classification and regression. During training, the most suitable FPN layer is selected for training according to the prediction results of anchor free branches. The final network output can synthesize the results of fsaf’s anchor free branches and the prediction results of the original network at the same time.

Network Architecture


Fsaf: embedding anchor free branch to guide acnor based algorithm training | cvpr2019

The network result of fsaf is very simple, as shown in Figure 4. In the original network structure, fsaf introduces two additional convolution layers for each layer of FPN to predict the classification and regression results of anchor free respectively. In this way, in the case of sharing features, anchor free and anchor based methods can make joint prediction.

Ground-truth and Loss


Fsaf: embedding anchor free branch to guide acnor based algorithm training | cvpr2019

For the target $B = [x, y, W, H] $, it can be mapped to any FPN layer $P during training_ The mapping area is $B ^ L_ p=[x^l_ p, y^l_ p, w^l_ p, h^l_ p]$。 In general, $B ^ L_ p=b/2^l$。 Define valid boundary $B ^ L_ e=[x^l_ e, y^l_ e, w^l_ e, h^l_ e] $and ignore boundary $B ^ L_ i=[x^l_ i, y^l_ i, w^l_ i, h^l_ i] , which can be used to define the positive sample region, the ignored region and the negative sample region in the feature graph. The effective boundary and the ignored boundary are proportional to the mapping results, and the proportions are $/ epsilon, respectively_ E = 0.2 $and $- epsilon_ I = 0.5 $, the final classification loss is the sum of the loss values of all positive and negative samples divided by the number of positive samples.

Classification Output

The classification result includes $k $dimension, and the target mainly sets the corresponding dimension

  • The region within the effective boundary is a positive sample point.
  • The regions from the ignored boundary to the effective boundary do not participate in the training.
  • Ignoring the boundary mapping to the adjacent feature pyramid layer, the region within the mapped boundary does not participate in the training
  • The rest areas are negative sample points.

The training of classification adopts focal loss, $- alpha = 0.25 $, $- gamma = 2.0 $. The total loss of classification is the sum of the loss values of all positive and negative regions divided by the number of effective regions.

Box Regression Output

The output of regression results is four dimensions of offset values which are independent of classification, and only the points in the effective region are regressed. For the valid area location $(I, J) $, the mapping target is expressed as $d ^ L_{ i,j}=[d^l_{ t_{ i,j}}, d^l_{ l_{ i,j}}, d^l_{ b_{ i,j}}, d^l_{ r_{ i. J}] $, respectively from the current position to $B ^ L_ The distance between the boundary of P $and the corresponding 4-dimensional vector at this position is $d ^ L_{ i. J} / S $, $s = 4.0 $are normalized constants. The training of regression adopts IOU loss, and the loss of complete anchor free branch takes the mean value of all effective regions.

Online Feature Selection


Fsaf: embedding anchor free branch to guide acnor based algorithm training | cvpr2019

The anchor free design allows us to use any FPN layer $P_ In order to find the optimal FPN layer, fsaf module needs to calculate the target prediction effect of each FPN layer. For classification and regression, the loss of focal loss and IOU loss of each effective region are calculated respectively

Fsaf: embedding anchor free branch to guide acnor based algorithm training | cvpr2019

After obtaining the results of each layer, the layer with the minimum loss value is selected as the FPN layer of the current round of training

Fsaf: embedding anchor free branch to guide acnor based algorithm training | cvpr2019

Joint Inference and Training


Inference

Because fsaf has few changes to the original network, the results of anchor free and anchor based branches are filtered slightly during reasoning, and then combined for NMS.

Optimization

The complete loss function synthesizes anchor based and anchor free bifurcations, $l = L ^ {AB} + – lambda (L ^ {AF})_{ cls}}+L^{af_{ reg}})$

Experiments


Fsaf: embedding anchor free branch to guide acnor based algorithm training | cvpr2019

The comparison experiment of various structures and FPN layer selection methods.

Fsaf: embedding anchor free branch to guide acnor based algorithm training | cvpr2019

The accuracy is compared with the reasoning speed.

Fsaf: embedding anchor free branch to guide acnor based algorithm training | cvpr2019

Compared with SOTA method.

Conclusion


Fsaf deeply analyzes the selection problem of FPN layer in training, and embeds it into the original network in the form of super simple anchor free branch, which almost has no effect on the speed. It can select the optimal FPN layer more accurately and bring good accuracy improvement. It should be noted that although the previous rigid selection method is abandoned, there are still some artificial settings, such as the definition of effective area, so this method is not the most perfect.



If this article is helpful to you, please like it or read it
More content, please pay attention to WeChat official account.

Fsaf: embedding anchor free branch to guide acnor based algorithm training | cvpr2019