多层特征融合和并行自注意力的孪生网络目标跟踪算法

目标跟踪是计算机视觉方向上的一项重要课题，其中尺度变化、形变和旋转是目前跟踪领域较难解决的问题。针对以上跟踪中所面临的具有挑战性的问题，基于已有的孪生网络算法提出多层特征融合和并行自注意力的孪生网络目标跟踪算法(MPSiamRPN)。首先，用修改后的ResNet50对模板图片和搜索图片进行特征提取，为处理网络过深而导致目标部分特征丢失，提出多层特征融合模块(multi-layer feature fusion module, MLFF)将ResNet后三层特征进行融合；其次，引入并行自注意力模块(parallel self-attention module, PSA),该模块由通道自注意力和空间自注意力组成，通道自注意力可以选择性地强调对跟踪有益的通道特征，空间自注意力能学习目标丰富的空间信息；最后，采用区域提议网络(regional proposal network, RPN)来完成分类和回归操作，从而确定目标的位置和形状。实验显示，提出的MPSiamRPN在OTB100、VOT2018两个测试数据集上取得了具有可竞争性的结果。

Multi-layer feature fusion and parallel self-attention Siamese networks for visual tracking

Object tracking is an important topic in computer visual directions, where scale changes, deformation and rotation are difficult to resolve in the field. For the challenging problems faced in the above track, based on existing Siamese network algorithms, this paper proposed multi-layer feature fusion and parallel self-attention Siamese networks(MPSiamRPN) for visual tracking. Firstly, MPSiamRPN used the modified ResNet50 to extract features from the template image and the search image. In order to deal with the loss of some features caused by the deep network, it proposed a multi-layer feature fusion module to fuse the features of the last three layers of ResNet. Secondly, it introduced the parallel self-attention module, which was composed of channel self-attention and spatial self-attention. The channel self-attention could selectively emphasize the beneficial channel features for tracking, and the spatial self-attention could learn the rich spatial information of the target. Finally, it proposed the region proposal network(RPN) to perform classification and regression operations to determine the location and shape of the target. Experiments show that the MPSiamRPN can achieve competitive results on OTB100 and VOT2018 test datasets.