RAFNet: RGB-D attention feature fusion network for indoor semantic segmentation

Semantic segmentation based on the complementary information from RGB and depth images has recently gained great popularity, but due to the difference between RGB and depth maps, how to effectively use RGB-D information is still a problem. In this paper, we propose a novel RGB-D semantic segmentation network named RAFNet, which can selectively gather features from the RGB and depth information. Specifically, we construct an architecture with three parallel branches and propose several complementary attention modules. This structure enables a fusion branch and we add the Bi-directional Multi-step Propagation (BMP) strategy to it, which can not only retain the feature streams of the original RGB and depth branches but also fully utilize the feature flow of the fusion branch. There are three kinds of complementary attention modules that we have constructed. The RGB-D fusion module can effectively extract important features from the RGB and depth branch streams. The refinement module can reduce the loss of semantic information and the context aggregation module can help propagate and integrate information better. We train and evaluate our model on NYUDv2 and SUN-RGBD datasets, and prove that our model achieves state-of-the-art performances.     

基于相位谱和调谐幅度谱的显著性检测方法

针对目前视觉注意选择领域中的谱残余方法(SR)显著图对比度较差、细节显著性检测效果不理想的问题,通过分析图像频谱特性与显著性的关系,提出了一种基于频谱分析的显著性区域检测方法。该方法通过保留傅里叶相位谱并对幅度谱进行分段非线性调谐,达到抑制图像冗余信息、增强图像显著性信息的效果。实验结果表明,本文基于相位谱和幅度谱调谐(PTA)的显著性检测方法得到的显著图较SR方法对比度更高,对显著细节的检测效果也更明显。     

应用改进频率调谐的海上小目标检测方法

提出了一种新的基于可见光图像的海上小目标检测方法。由于频率调谐方法将图像空间域整体均值与高斯滤波后差分结果,作为显著性度量的标准,因此当图像背景中存在较多杂波干扰时,显著目标检测效果不理想。提出的方法对基于频率调谐的显著性检测方法进行了改进,首先对图像LAB空间中3个特征分量进行分块,在每个分块区域中应用频率调谐显著性检测方法,进而将其结果合并为总显著图,以检测海上小目标。该方法克服了频率调谐方法,当海面背景中存在大量海杂波,无法有效提取小目标的缺陷。实验结果表明了该方法的有效性。     

基于小波变换和Retinex算法的显著性检测

近些年来,关于图像显著性检测的研究越来越热门。基于之前提出的很多算法产生的显著图都存在背景信息杂乱、干扰噪声多、细节丢失等问题,本文提出了一种基于小波变换和Retinex算法的显著性检测算法来解决以上问题。首先,利用Retinex算法对图像进行前期处理;然后,对前期处理过的图像进行SLIC超像素分割,对超像素进行小波变换,分别生成原始图像低频部分和高频部分的特征图,并进行适当的双边滤波降噪,生成对应的显著图;最后,通过加权组合这两种显著图,得到最终的显著性图。实验结果表明,本文提出的算法生成的显著图具有受背景影响小、噪声少以及细节突出等优势。     

Online learning and fusion of orientation appearance models for robust rigid object tracking

We introduce a robust framework for learning and fusing of orientation appearance models based on both texture and depth information for rigid object tracking. Our framework fuses data obtained from a standard visual camera and dense depth maps obtained by low-cost consumer depth cameras such as the Kinect. To combine these two completely different modalities, we propose to use features that do not depend on the data representation: angles. More specifically, our framework combines image gradient orientations as extracted from intensity images with the directions of surface normals computed from dense depth fields. We propose to capture the correlations between the obtained orientation appearance models using a fusion approach motivated by the original Active Appearance Models (AAMs). To incorporate these features in a learning framework, we use a robust kernel based on the Euler representation of angles which does not require off-line training, and can be efficiently implemented online. The robustness of learning from orientation appearance models is presented both theoretically and experimentally in this work. This kernel enables us to cope with gross measurement errors, missing data as well as other typical problems such as illumination changes and occlusions. By combining the proposed models with a particle filter, the proposed framework was used for performing 2D plus 3D rigid object tracking, achieving robust performance in very difficult tracking scenarios including extreme pose variations.     

结合显著度图和自适应能量线的快速图像缩放*

针对Seam Carving图像缩放方法中能量函数的局限性和动态规划算法的复杂性,提出了一种基于图像内容的快速图像缩放方法。结合基于视觉注意力模型的显著度图,完善图像内容的能量描述;同时,提出一种基于图像内容的粗、细能量线自适应机制,与层次加速动态规划一起,加速缩放效率。仿真实验结果表明,该算法在有效保护图像内容的同时,降低了算法的计算复杂度,减少了图像缩放时间。     

显著性检测指导的高光区域修复

目的为解决传统的基于光照模型的高光修复算法无法很好地对高光区域存在饱和现象的单幅图像进行处理这一问题,提出一种显著性检测指导的高光区域修复算法。方法首先在亮度空间应用显著性模型,实现高光区域的自动检测和标记,之后运用改进的Exemplar-Based算法,综合利用图像的邻域和边缘信息,对标记的高光区域进行自适应修复,去除图像中的高光。结果分别对仿真及自然场景下的高光图像进行测试,实验结果表明,与原修复算法和传统高光去除算法相比,所提算法的修复效果更符合人眼视觉、修复后的图像质量更好。结论本文算法与Exemplar-Based算法及Tan方法相比,对高光区域存在饱和现象的单幅图像有较好的修复效果,并且有效地克服了传统高光去除算法受光照模型限制的缺点。     

基于显著性多尺度特征协作融合的行人重识别方法

由于背景信息复杂、遮挡等因素的影响,现有基于局部特征的行人重识别方法所提取的特征不具有辨别力和鲁棒性,从而导致重识别精度较低,针对该问题,提出一种基于显著性检测与多尺度特征协作融合的SMC-ReID方法。利用显著性检测提取行人中具有判别力的特征区域,融合显著性特征与全局特征并完成不同尺度的切块,将上述不同尺度的特征进行协作融合以保证特征切块后的连续性,根据全局特征和局部特征的差异性联合3种损失函数进行学习。在推理阶段,将各个尺度的特征降低到同一维度并融合成新的特征向量,以实现相似性度量。在行人重识别公开数据集Market1501、DukeMTMC-reID和CUHK03上进行实验,结果表明,SMC-ReID方法所提取的特征具有较强的可区分性和鲁棒性,识别准确率优于SVDNet和PSE+ECN等方法。     

融合视觉显著性再检测的孪生网络无人机目标跟踪算法

针对旋翼飞行器在跟踪过程中目标尺度变化、快速运动、视角变化等问题,本文提出了一种基于MobileNetV2的孪生网络目标跟踪算法,可在无人机机载处理器上实时运行。该算法主要包含目标得分估计模块与目标尺度估计模块两个部分。结合多特征融合的策略,可准确预测出目标位置与目标框IoU,同时以目标框IoU为指导,利用梯度上升法对目标框进行迭代修正,进一步提升预测精度。针对完全遮挡而导致的目标跟丢问题,本文设计了一个基于视觉显著性的目标再检测算法,该算法可实时高效地预测出图像的显著性区域,以指导对目标的再检测,进而恢复跟踪。最后,通过标准无人机跟踪数据集测试与实际无人机跟踪实验,验证了算法的可行性。