基于多模态的车辆行人检测技术研究

针对单一传感器在复杂路况以及恶劣天气情况下车辆行人检测效果不佳，搭建了一套可见光、可见光偏振、短波红外和长波红外多模态数据采集系统，构建了一个多模态数据集，并提出了一种多模态车辆行人检测算法。首先，提出了一种基于改进型SIFT特征点的多尺度部分强度不变特征的异源图像配准算法；然后，提出基于YOLOv5多模态数据目标检测网络。最终实现了平均精度在日间数据集1.0%的提升，日间夜间混合数据集10.9%的提升。

Research on Vehicle and Pedestrian Detection Technology Based on Multimodal Image

Aiming at the problem of poor detection of vehicles and pedestrians by a single sensor in complex road conditions and bad weather conditions, a set of visible light, visible polarization, short-wave infrared and long-wave infrared multimodal data acquisition system was built to construct a multimodal dataset, and a multimodal pedestrian detection algorithm for vehicle pedestrians was proposesd. Firstly, a heterologous image registration algorithm based on the multi-scale partial intensity invariant features of improved SIFT feature points was proposed. Then, for target detection, a multimodal data object detection network based on YOLOv5 was proposed. Finally, the average accuracy is improved by 1.0% in the daytime dataset and 10.9% in the daytime and nighttime mixed dataset.