RAFNet: RGB-D attention feature fusion network for indoor semantic segmentation

Semantic segmentation based on the complementary information from RGB and depth images has recently gained great popularity, but due to the difference between RGB and depth maps, how to effectively use RGB-D information is still a problem. In this paper, we propose a novel RGB-D semantic segmentation network named RAFNet, which can selectively gather features from the RGB and depth information. Specifically, we construct an architecture with three parallel branches and propose several complementary attention modules. This structure enables a fusion branch and we add the Bi-directional Multi-step Propagation (BMP) strategy to it, which can not only retain the feature streams of the original RGB and depth branches but also fully utilize the feature flow of the fusion branch. There are three kinds of complementary attention modules that we have constructed. The RGB-D fusion module can effectively extract important features from the RGB and depth branch streams. The refinement module can reduce the loss of semantic information and the context aggregation module can help propagate and integrate information better. We train and evaluate our model on NYUDv2 and SUN-RGBD datasets, and prove that our model achieves state-of-the-art performances.     

Online learning and fusion of orientation appearance models for robust rigid object tracking

We introduce a robust framework for learning and fusing of orientation appearance models based on both texture and depth information for rigid object tracking. Our framework fuses data obtained from a standard visual camera and dense depth maps obtained by low-cost consumer depth cameras such as the Kinect. To combine these two completely different modalities, we propose to use features that do not depend on the data representation: angles. More specifically, our framework combines image gradient orientations as extracted from intensity images with the directions of surface normals computed from dense depth fields. We propose to capture the correlations between the obtained orientation appearance models using a fusion approach motivated by the original Active Appearance Models (AAMs). To incorporate these features in a learning framework, we use a robust kernel based on the Euler representation of angles which does not require off-line training, and can be efficiently implemented online. The robustness of learning from orientation appearance models is presented both theoretically and experimentally in this work. This kernel enables us to cope with gross measurement errors, missing data as well as other typical problems such as illumination changes and occlusions. By combining the proposed models with a particle filter, the proposed framework was used for performing 2D plus 3D rigid object tracking, achieving robust performance in very difficult tracking scenarios including extreme pose variations.     

结合显著度图和自适应能量线的快速图像缩放*

针对Seam Carving图像缩放方法中能量函数的局限性和动态规划算法的复杂性,提出了一种基于图像内容的快速图像缩放方法。结合基于视觉注意力模型的显著度图,完善图像内容的能量描述;同时,提出一种基于图像内容的粗、细能量线自适应机制,与层次加速动态规划一起,加速缩放效率。仿真实验结果表明,该算法在有效保护图像内容的同时,降低了算法的计算复杂度,减少了图像缩放时间。     

RGB-D Hand-Held Ob ject Recognition Based on Heterogeneous Feature Fusion

Ob ject recognition has many applications in human-machine interaction and multimedia retrieval. However, due to large intra-class variability and inter-class similarity, accurate recognition relying o...     

基于指尖触控的互动投影系统

随着人机交互技术的不断发展,互动投影技术的应用逐渐广泛.该技术通过摄像机等传感器技术感知周围场景动态并结合投影的显示功能实现交互的方式,如增强现实、结构光编码、三维重建等.基于触控的互动投影技术在未来的嵌入式人机交互系统中具有广阔的前景.文章基于单目摄像机与投影仪系统实现能够在任意平面徒手触摸的交互.系统的功能主要通过前景区域分割、指尖点的定位以及触摸检测三个步骤来实现.与目前相关领域的研究文献进行的对比实验证明,所提算法在稳定性、准确性以及实时性等方面具有一定优势.     

基于近似高斯金字塔的视觉注意模型快速算法

利用输入图像的近似高斯金字塔,将经典的基于显著性的视觉注意模型改造为时空开销更小的版本,从而使其更加适合在嵌入式实时系统中实现.首先采用矩形窗口近似圆形窗口,矩形平均算子近似高斯卷积核;然后采用“先做行累加,再做列累加”的方法来实现矩形平均算子,并直接采样计算出各个特征通道的显著性分布图,该算法关于输入图像像素点个数具有线性时间复杂度;最后,还给出了在显著性分布图中抑制已提取区域显著性的快速算法.在Berkeley分割图像库上的实验结果表明,该方法极大地减小了系统实现的时空开销,且输出结果的误差在可接受范围内.提出的用矩形窗口近似圆形窗口,用矩形平均算子近似高斯卷积核的方法,还适用于其他需要在嵌入式实时系统中实现的图像处理问题.     

孪生网络框架下融合显著性和干扰在线学习的航拍目标跟踪算法

针对一般跟踪算法不能很好地解决航拍视频下目标分辨率低、视场大、视角变化多等特殊难点,该文提出一种融合目标显著性和在线学习干扰因子的无人机(UAV)跟踪算法.通用模型预训练的深层特征无法有效地识别航拍目标,该文跟踪算法能根据反向传播梯度识别每个卷积滤波器的重要性来更好地选择目标显著性特征,以此凸显航拍目标特性.另外充分利用连续视频丰富的上下文信息,通过引导目标外观模型与当前帧尽可能相似地来在线学习动态目标的干扰因子,从而实现可靠的自适应匹配跟踪.实验证明:该算法在跟踪难点更多的UAV123数据集上跟踪成功率和准确率分别比孪生网络基准算法高5.3%和3.6%,同时速度达到平均28.7帧/s,基本满足航拍目标跟踪准确性和实时性需求.     

基于多流网络一致性的视频显著性检测

现有的视频显著性检测算法通常采用双流结构提取视频的时空线索,其中运动信息作为双流结构的一个分支,在显著物体发生剧烈或慢速移动时存在运动估计准确率低的问题,并且不合理的训练数据或方案使得权重偏向单个分支结构。提出一种基于多流网络一致性的视频显著性检测算法MSNC。设计并使用一种新的三重网络结构提取预选目标区域的颜色信息、时序信息和先验特征,通过先验特征补偿运动流的缺陷,并提高运动线索的利用率。采用多流一致性融合模型优化三流分支,得到不同特征的最佳融合方案。同时通过循环训练策略平衡三重网络的权重,以避免网络过度拟合单流分支,从而有效地提高运动估计和定位的准确率。在Davis数据集上的实验结果表明,相比PCSA、SSAV、MGA等算法,该算法的鲁棒性更优,其maxF和S-Measure值分别达到0.893和0.912,MAE仅为0.021。     

结合l1/2范数与显著性约束的背景减除

传统背景减除模型在背景静止和前景对象移动较快时提取到的前景效果较好,但当背景变化或前景对象移动缓慢时容易将动态背景误判为前景或检测出的前景有较多空洞。针对传统背景减除模型在动态背景和前景对象移动缓慢条件下存在前景检测精度下降的问题,提出一种基于l_1/2范数与显著性约束的背景减除模型。将观测数据分为低秩背景、运动前景和动态干扰3类,利用l_1/2范数约束运动前景加强前景稀疏性,有效抑制动态背景对前景提取造成的干扰,提高运动前景在动态背景中的检测精度。引入视频每一帧的显著性约束,通过对每一帧图像进行低秩稀疏分解来检测移动缓慢的目标。实验结果表明,该模型对于复杂场景具有较强的适应能力,可有效去除动态背景对前景的干扰,快速检测出移动缓慢的前景对象,相比于l_1/1/2-RPCA背景减除模型的平均查全率、查准率和调和平均值分别提升了9、14和10个百分点。