DiLO: Direct light detection and ranging odometry based on spherical range images for autonomous driving

Over the last few years, autonomous vehicles have progressed very rapidly. The odometry technique that estimates displacement from consecutive sensor inputs is an essential technique for autonomous driving. In this article, we propose a fast, robust, and accurate odometry technique. The proposed technique is light detection and ranging (LiDAR)-based direct odometry, which uses a spherical range image (SRI) that projects a three-dimensional point cloud onto a two-dimensional spherical image plane. Direct odometry is developed in a vision-based method, and a fast execution speed can be expected. However, applying LiDAR data is difficult because of the sparsity. To solve this problem, we propose an SRI generation method and mathematical analysis, two key point sampling methods using SRI to increase precision and robustness, and a fast optimization method. The proposed technique was tested with the KITTI dataset and real environments. Evaluation results yielded a translation error of 0.69%, a rotation error of 0.0031°/m in the KITTI training dataset, and an execution time of 17 ms. The results demonstrated high precision comparable with state-of-the-art and remarkably higher speed than conventional techniques.     

Efficient graph attentional network for 3D object detection from Frustum-based LiDAR point clouds

LiDAR-based 3D object detection is important for autonomous driving scene perception, but point clouds produced by LiDAR are irregular and unstructured in nature, and cannot be adopted by the conventional Convolutional Neural Networks (CNN). Recently, Graph Convolutional Networks (GCN) has been proved as an ideal way to handle non-Euclidean structure data, as well as for point cloud processing. However, GCN involves massive computation for searching adjacent nodes, and the heavy computational cost limits its applications in processing large-scale LiDAR point cloud in autonomous driving. In this work, we adopt a frustum-based point cloud-image fusion scheme to reduce the amount of LiDAR point clouds, thus making the GCN-based large-scale LiDAR point clouds feature learning feasible. On this basis, we propose an efficient graph attentional network to accomplish the goal of 3D object detection in autonomous driving, which can learn features from raw LiDAR point cloud directly without any conversions. We evaluate the model on the public KITTI benchmark dataset, the 3D detection mAP is 63.72% on KITTI Cars, Pedestrian and Cyclists, and the inference speed achieves 7.9 fps on a single GPU, which is faster than other methods of the same type.     

基于交叉自注意力机制的LiDAR点云三维目标检测

针对基于深度学习的激光雷达(light detection and ranging, LiDAR)点云三维(3D)目标检测对小目标的检测精度较低和噪声干扰问题,提出一种基于交叉自注意力机制的3D点云目标检测方法CSA-RCNN (cross self-attention region convolutional neural network)。利用交叉自注意力(cross self-attention, CSA)同时学习点云的坐标和特征,并设计多尺度融合(multi-scale fusion, MF)模块自适应捕捉各层级多尺度特征。此外,还设计重叠采样策略对感兴趣目标区域选择性地重采样以获得更多前景点,有效降低了噪声采样。在广泛使用的KITTI数据集上进行算法性能测试,结果表明,本文方法对行人等小目标的检测精度有较大提升,平均精度均值相比PointRCNN等4种经典算法均获得提升,显著提高3D点云目标的检测性能。     

Real-time 3D multi-pedestrian detection and tracking using 3D LiDAR point cloud for mobile robot

Mobile robots are used in modern life; however, object recognition is still insufficient to realize robot navigation in crowded environments. Mobile robots must rapidly and accurately recognize the movements and shapes of pedestrians to navigate safely in pedestrian-rich spaces. This study proposes real-time, accurate, three-dimensional (3D) multi-pedestrian detection and tracking using a 3D light detection and ranging (LiDAR) point cloud in crowded environments. The pedestrian detection quickly segments a sparse 3D point cloud into individual pedestrians using a lightweight convolutional autoencoder and connected-component algorithm. The multi-pedestrian tracking identifies the same pedestrians considering motion and appearance cues in continuing frames. In addition, it estimates pedestrians' dynamic movements with various patterns by adaptively mixing heterogeneous motion models. We evaluate the computational speed and accuracy of each module using the KITTI dataset. We demonstrate that our integrated system, which rapidly and accurately recognizes pedestrian movement and appearance using a sparse 3D LiDAR, is applicable for robot navigation in crowded spaces.     

Collision-free local planner for unknown subterranean navigation

When operating in confined spaces or near obstacles, collision-free path planning is an essential requirement for autonomous exploration in unknown environments. This study presents an autonomous exploration technique using a carefully designed collision-free local planner. Using LiDAR range measurements, a local end-point selection method is designed, and the path is generated from the current position to the selected end-point. The generated path showed the consistent collision-free path in real-time by adopting the Euclidean signed distance field-based grid-search method. The results consistently demonstrated the safety and reliability of the proposed path-planning method. Real-world experiments are conducted in three different mines, demonstrating successful autonomous exploration flights in environment with various structural conditions. The results showed the high capability of the proposed flight autonomy framework for lightweight aerial robot systems. In addition, our drone performed an autonomous mission in the tunnel circuit competition (Phase 1) of the DARPA Subterranean Challenge.     

机载平台下基于深度检测网络的目标跟踪重捕算法

目标检测与跟踪是机载光电设备至关重要的功能模块,其检测跟踪的性能直接关系到目标感知的精度.近年来基于Siamese网络的改进跟踪算法在各种挑战性的数据集上取得了优异的效果,但大多数改进算法采用局部搜索策略,无法更新模板,且模板会引入背景干扰,最终因跟踪点漂移导致跟踪失败.为了解决这些问题,本文提出了一种结合目标边缘检测的改进全连接Siamese跟踪算法,该算法利用目标的轮廓模板代替边界框模板,减少了背景杂波的干扰;同时,在Siamese网络的基础上增加了一路改进tiny-YOLOv3目标检测网络,利用K均值聚类找到最合适的锚框(anchor box),引入了扩张模块层来扩展感受野,增加了系统的抗遮挡能力,提高机载光电设备的目标捕获概率.在基准测试数据集以及挂飞数据集基础上的仿真测试性能表明本文提出的改进模型特别适合机载光电设备在跟踪与重捕复杂环境下的运动目标,在长期跟踪中能够更好地适应目标的变形和遮挡,提升系统响应时间与适应性.     

基于路侧激光雷达的多目标检测与跟踪算法

为了检测与跟踪城市交叉口复杂环境下的道路目标,提出一种基于路侧激光雷达的多目标检测与跟踪算法。首先利用背景减除法滤除背景点云,随后融合5帧点云并利用曲率体素聚类算法检测目标得到3 D包围盒信息,之后通过自适应阈值的双门控和生存周期管理策略,有效提升关联精度并减少了目标丢失和误检,最后利用交互式多模型无迹卡尔曼滤波(IMM-UKF)和联合概率数据互联(JPDA)的融合算法完成道路目标的跟踪。试验结果表明,该算法在保证检测和跟踪性能基础上满足实时性要求,具有工程实用价值。     

基于点云极化表征与孪生网络的智能车定位

基于3维激光雷达 (LiDAR) 的智能车定位在地图存储空间与匹配效率、准确率等方面仍存在诸多问题。该文提出一种轻量级点云极化地图构建方法：采用多通道图像模型对3维点云进行编码生成点云极化图,利用孪生网络结构提取并训练点云极化指纹,结合轨迹位姿信息构建点云极化地图。还提出一种基于点云极化地图匹配的智能车定位方法：采用孪生网络对查询指纹与地图指纹进行相似度建模实现快速的地图粗匹配,采用基于2阶隐马尔可夫模型 (HMM2) 的地图序列精确匹配方法获取最近的地图节点,通过点云配准计算车辆位姿。使用实地数据集和公开的KITTI数据集进行测试。实验结果表明,地图匹配准确率高于96%,定位平均误差约为30 cm,并对不同类型的LiDAR传感器与不同的场景具有较好的鲁棒性。     

机载LiDAR点云分块插值滤波

现有机载激光雷达（LiDAR）点云滤波算法在简单地形下取得了较好的滤波效果,但普遍对陡坡地形适应性较差。为提高在不同地形下的滤波性能,提出了基于分块的多尺度表面插值滤波算法。该算法首先通过改进的区域增长分块算法将原始点云分为点云块集和散点集,然后通过构建的多尺度表面插值算法同时对点云块和散点进行分类。利用国际摄影测量与遥感学会(ISPRS)提供的基准数据验证表明,该方法在15个样本中有11个样本滤波效果优于现有滤波方法,对各类地形均有较强适应性,且该方法平均总误差最小。对三种不同地形特征的高密度数据滤波实验,也验证了该方法的良好性能。     

基于Faster R-CNN模型的低空平台偏振高光谱目标检测

随着无人机等低空平台在侦察领域的不断扩展以及对性能要求的不断提高,各应用场景对目标检测精度和速度也提出了越来越高的要求.传统的目标成像方法难以满足图像质量需求,人工识别目标的方法也无法应对战场环境的快速变化.结合深度学习和偏振高光谱成像技术的发展,通过模拟偏振高光谱低空目标检测平台,提出基于Faster R-CNN的地面军事目标检测方法.采用区域建议网络模块进行模型训练,而在目标检测阶段通过对特征图进行兴趣区域池化操作得到建议特征图,最后利用建议特征图完成目标类别判定.实验选取3种典型的军事车辆缩比模型,通过偏振高光谱相机在室内外模拟环境中获取目标在不同场景条件的图像数据,以及某型无人机在低空条件下的地面车辆目标数据进行实验验证.实验表明,该方法在有效完成地面目标的检测时,能够达到理想的检测精度和速度.     

LiDAR-only 3D object detection based on spatial context

LiDAR-based 3D Object detection is one of the popular topics in recent years, and it is widely used in the fields of autonomous driving and robot controlling. However, due to the scanning pattern of LiDAR, the point clouds of objects at far distance are sparse and more difficult to be detected. To solve this problem, we propose a two-stage network based on spatial context information, named SC-RCNN (Spatial Context RCNN), for object detection in 3D point cloud scenes. SC-RCNN first uses a backbone with sparse convolutions and submanifold sparse convolutions to extract the voxel features of point scenes and generate a series of candidate boxes. For the sparsity of far-distance point clouds, we design the local grid point pooling (LGP Pooling) to extract features and spatial context information around candidate regions for subsequent box refinement. In addition, we propose the pyramid candidate box augmentation (PCB Augmentation) to expand the candidate boxes with a multi-scale style, enriching the feature encoding. The experimental results show that SC-RCNN significantly outperforms previous methods on KITTI dataset and Waymo dataset, and is particularly robust to the sparsity of point clouds.     

Exploiting Reactive Mobility for Collaborative Target Detection in Wireless Sensor Networks

Recent years have witnessed the deployments of wireless sensor networks in a class of mission-critical applications such as object detection and tracking. These applications often impose stringent Quality-of-Service requirements including high detection probability, low false alarm rate, and bounded detection delay. Although a dense all-static network may initially meet these Quality-of-Service requirements, it does not adapt to unpredictable dynamics in network conditions (e.g., coverage holes caused by death of nodes) or physical environments (e.g., changed spatial distribution of events). This paper exploits reactive mobility to improve the target detection performance of wireless sensor networks. In our approach, mobile sensors collaborate with static sensors and move reactively to achieve the required detection performance. Specifically, mobile sensors initially remain stationary and are directed to move toward a possible target only when a detection consensus is reached by a group of sensors. The accuracy of final detection result is then improved as the measurements of mobile sensors have higher Signal-to-Noise Ratios after the movement. We develop a sensor movement scheduling algorithm that achieves near-optimal system detection performance under a given detection delay bound. The effectiveness of our approach is validated by extensive simulations using the real data traces collected by 23 sensor nodes.     

Precise and efficient Chinese license plate recognition in the real monitoring scene of intelligent transportation system

In this paper, the performance of you only look once ( YOLO) series detectors on Chinese license platerecognition (LPR) in the real intelligent transportation system (ITS) monitoring scene is investigated. Specially, aprecise and efficient automatic license plate recognition ( ALPR ) system based on the YOLOv4 detector isproposed. The proposed ALPR system contains three stages including vehicle detection, license plate detection(LPD) and LPR. In vehicle detection stage, YOLOv4 detector is directly applied. In LPD stage, YOLOv4-tinydetector is exploited. In the last stage, the YOLOv4-tiny detector with attention mechanism for LPR is proposed touse. In addition, a large Chinese license plate dataset containing 10 500 images collected from all 31 provinces inthe Chinese mainland is created. This Chinese license plate dataset is named Hefei University of Technology licenseplate version 1 (HFUT-LP v1). Particularly, HFUT-LP v1 dataset is collected in the real ITS monitoring scene. Inorder to compare the performance of different object detection algorithms for ALPR, a variety of object detectionalgorithms are used to make a comprehensive performance evaluation. Experimental results show that theproposedALPR system achieves very high accuracy and has very fast processing speed, which is suitable for real-time LPR.     

Exploring the usefulness of light field super-resolution for object detection

In order to solve the impact of image degradation on object detection, an object detection method based on light field super-resolution ( LFSR) is proposed. This method takes LFSR as an image enhancement step to provide high- quality images for object detection without using expensive imaging equipment. To evaluate this method, three types of objects: person, bicycle, and car, are chosen and the results are compared from 5 parts: detected object quantity, mean confidence score, detection results in different scenes, error detection, and detection results from different images sizes and detection speed. Experimental results based on the common object in context ( COCO) dataset show that the method incorporated LFSR improves performance of object detection models.     

频域内面向目标检测的领域自适应

近年来,基于深度学习的目标检测技术在机器人、自动驾驶和交通监控等领域有着广泛的应用。然而,由于训练集和测试集样本分布偏差的原因,将现成的预训练检测器应用到实际开放场景时通常会出现明显性能下降。针对该问题提出了一种频域内的领域自适应方法,利用离散余弦变换的频域能量集中特性,通过在频域内对少数重要频率系数进行处理,实现了面向目标检测的领域自适应,降低了对存储和计算资源的要求并减少了领域差异。该方法可以分为两个阶段:第一阶段使用无监督图像转换方式,将源域已标注的训练数据向目标域作转换;第二阶段采用基于对抗的领域自适应方法训练目标检测模型,对转换后的训练数据与目标域内的数据作特征适配。针对不同天气场景的目标识别实验表明:所提的频域内领域自适应方法在4种领域自适应对比算法中排名第一,与仅用源域数据训练的模型相比,mAP值提升了33.9%。     

基于长时信息的自适应话音激活检测

语音信号的长时信息应用于话音激活检测中表现优越.利用三种听觉滤波器组,对语音信号进行非线性的谱分解,本文提出了六种基于听觉滤波器组的长时信息,并提出了基于长时信息的自适应话音激活检测算法.该算法无需训练数据,根据多种长时信息,直接在待测信号中挑选出类别明确的信号,然后利用这些信号训练分类模型,对待测信号按帧进行语音-非语音分类.在TIMIT语音库和NOISEX-92噪声库上的实验表明,该算法在极低信噪比环境下,仍表现出更高的准确性和更强的稳健性.同时,在线实验表明,算法在实时处理中仍能取得优异的性能.     

Finding suits in images of people in unconstrained environments

Clothing style analysis is a critical step for understanding images of people. To automatically identify the style of clothing that people wear is a challenging task due to various poses of person and large variations for even the same clothing category. Suit as one of the clothing style is a key element in many important activities. In this paper, we propose a novel suits detection method for images of people in unconstrained environments. In order to cope with various human poses, human pose estimation is incorporated. By analyzing the style of clothing, we propose the color features, shape features and statistical features for suits detection. Experiments with four popular classifiers have been conducted to demonstrate that the proposed features are effective and robust. Comparative experiments with Bag of Words (BoW) method demonstrate that the proposed features are superior to BoW which is a popular method for object detection. The proposed method has achieved promising performance over our dataset, which is a challenging web image set with various human poses and diverse styles of clothing.     

基于深度学习物体检测的视觉跟踪方法

提出了一种基于深度学习物体检测的视觉跟踪方法。该方法利用深度学习在特征表达上的优势,采用基于回归的深度检测模型SSD （Single Shot Multibox Detector）提取候选目标,并结合颜色直方图特征和HOG （Histogram of Oriented Gradient）特征进行目标筛选,实现目标跟踪。为了提升深度检测模型的物体检测性能,文中构建了多尺度目标搜索图,可在一张图上实现不同尺度的目标检测。在标准跟踪测试库上选取八个具有代表性的跟踪视频序列,并选取六种具有代表性的跟踪方法进行了对比测试。结果表明,文中所提方法在跟踪效果上,整体优于参与对比的其他算法,且对于物体姿态变化、尺寸变化、旋转变化、光照变化、复杂背景杂波等影响因素具有较好的鲁棒性。     

基于双模态特征增强的目标检测算法研究与应用北大核心CSCD

为提升目标检测算法在复杂环境下的精确性和实用性,将多源信息和深度学习技术相结合,提出了一种基于双模态特征增强的目标检测方法。该方法以红外和可见光图像作为输入,利用颜色空间转换、边缘提取、直方图均衡化等传统图像处理方法丰富图像信息,达到数据增强效果;特征提取部分采用卷积神经网络结构分别提取目标红外及可见光信息,并设计混合注意力机制分别从通道和空间位置角度提升有效特征权重;同时,针对目标双模态信息,引入了自适应交叉融合结构,提高特征多样性;最后,利用交替上下采样将目标全局和局部特征充分融合,并以自主选择方式提取目标相关特征实现检测。通过在标准数据集以及实际场景数据集上的实验结果表明,所提方法有效融合并增强了目标多模态特征,提升了目标检测效果,并能较好的应用于电网场景中,辅助机器人完成目标设备检测。     

面向多源遥感图像的自适应目标检测

近年来,目标检测已经在含有大量标注的数据上展现出了良好的效果,但当真实测试数据与标注数据存在域间差异时,往往会导致训练好的目标检测模型性能降低。由于相比于自然图像,多源遥感图像在成像方式和分辨率等方面存在特有的差异,而传统的方法需要将多源图像数据重新标注,这将消耗大量人力和时间,因此在遥感图像上实现自适应目标检测面临特有的挑战。针对以上问题,本文提出了一种面向多源遥感图像的自适应目标检测算法,在图像级别和语义级别上对网络进行对抗训练。此外,通过结合超分辨网络,进一步缩小了图像级别的差异,实现了自适应目标检测。本文在两个多源遥感数据集上进行实验,结果表明本文方法有效提升了目标域上的检测效果。