Non-maximum suppression for object detection based on the chaotic whale optimization algorithm

Non-maximum suppression (NMS) as a post-processing step for object detection is mainly used to remove redundant bounding boxes in the object and plays a vital role in many detectors. Its positioning accuracy mainly depends on the bounding box with the highest score, and this strategy is difficult to eliminate the false positive. In order to solve the problem, this paper regards the post-processing step as a combinatorial optimization problem and combines the chaotic whale optimization algorithm and non-maximum suppression. The chaotic search method is used to generate an initial combinatorial solution, and the whale optimization algorithm is discretized to create an updated combinatorial strategy. Under the guidance of the fitness function, the optimal combination is searched. In addition, the method of difference set area (DSA) is proposed to optimize the final detection result. The experiment uses the current mainstream framework Faster R-CNN as the detector on PASCAL VOC2012, COCO2017 and the Warships datasets. The experimental results show that the proposed method can significantly improve the average precision (AP) of detectors compared with the most advanced methods.