视频显著性检测研究进展EI北大核心CSCD

视频显著性检测是计算机视觉领域的一个热点研究方向,其目的在于通过联合空间和时间信息实现视频序列中与运动相关的显著性目标的连续提取.由于视频序列中目标运动模式多样、场景复杂以及存在相机运动等,使得视频显著性检测极具挑战性.对现有的视频显著性检测方法进行梳理,介绍相关实验数据集,并通过实验比较分析现有方法的性能.首先,介绍了基于底层线索的视频显著性检测方法,主要包括5类:基于变换分析的方法、基于稀疏表示的方法、基于信息论的方法、基于视觉先验的方法和其他方法.然后,对基于学习的视频显著性检测方法进行了总结,主要包括传统学习方法和深度学习方法,并着重对后一类方法进行了介绍.随后,介绍了常用的视频显著性检测数据集,给出了4种算法性能评价指标,并在不同数据集上对最新的几种算法进行了定性和定量的比较分析.最后,对视频显著性检测的关键问题进行了总结,并对未来的发展趋势进行展望.     

基于深度学习的显著性目标检测综述

随着深度学习的不断发展,基于深度学习的显著性目标检测已经成为计算机视觉领域的一个研究热点。首先对现有的基于深度学习的显著性目标检测算法分别从边界/语义增强、全局/局部结合和辅助网络三个角度进行了分类介绍并给出了显著性图,同时对三种类型方法进行了定性分析比较;然后简单介绍了基于深度学习的显著性目标检测常用的数据集和评估准则;接着对所提基于深度学习的显著性目标检测方法在多个数据集上进行了性能比较,包括定量比较、P-R曲线和视觉比较;最后指出现有基于深度学习的显著性目标检测方法在复杂背景、小目标、实时性检测等方面的不足,并对基于深度学习的显著性目标检测的未来发展方向,如复杂背景、实时、小目标、弱监督等显著性目标检测进行了探讨。     

基于深度学习的视频目标检测综述

视频目标检测是为了解决每一个视频帧中出现的目标如何进行定位和识别的问题.相比于图像目标检测,视频具有高冗余度的特性,其中包含了大量的时空局部信息.随着深度卷积神经网络在静态图像目标检测领域的迅速普及,在性能上相较于传统方法显示出了非常大的优越性,并逐步在基于视频的目标检测任务上也发挥了应有的作用.但现有的视频目标检测算法仍然面临改进与优化主流目标检测算法的性能、保持视频序列的时空一致性、检测模型轻量化等关键技术的挑战.针对上述问题和挑战,在调研大量文献的基础上系统地对基于深度学习的视频目标检测算法进行了总结.从基于光流、检测等基础方法对这些算法进行了分类,从骨干网络、算法结构、数据集等角度细致探究了这些方法.结合在ImageNet VID等数据集上的实验结果,分析了该领域具有代表性算法的性能优势和劣势,以及算法之间存在的联系.对视频目标检测中待解决的问题与未来研究方向进行了阐述和展望.视频目标检测已成为众多的计算机视觉领域学者追逐的热点,将来会有更加高效、精度更高的算法被相继提出,其发展方向也会越来越好.     

时空渐进式学习的视频显著性目标检测

视频显著性目标检测需要同时结合空间信息和时间信息,连续地定位视频序列中与运动相关的显著性目标,其核心问题在于如何高效地刻画运动目标的时空特征.现有的视频显著性目标检测算法大多使用光流,ConvLSTM以及3D卷积等提取时域特征,缺乏对时间信息的连续学习能力.为此,设计了一种鲁棒的时空渐进式学习网络(spatial-temporal progressive learning network, STPLNet),以完成对视频序列中显著性目标的高效定位.在空间域中使用一种U型结构对各视频帧进行编码解码,在时间域中通过学习视频序列中帧间运动目标的主体部分和形变区域特征,渐进地对运动目标特征进行编码,能够捕捉到目标的时间相关性特征和运动趋向性.在4个公开数据集上与13个主流的视频显著性目标检测算法进行一系列对比实验,所提出的模型在多个指标(max F, S-measure (S), MAE)上达到了最优结果,同时在运行速度上具有较好的实时性.     

光学遥感图像目标检测算法综述

目标检测技术是光学遥感图像理解的基础问题, 具有重要的应用价值. 本文对遥感图像目标检测算法发展进行了梳理和分析. 首先阐述了遥感图像目标检测的特点和挑战; 之后系统总结了典型的检测方法, 包括早期的基于手工设计特征的算法和现阶段基于深度学习的方法, 对于深度学习方法首先介绍了典型的目标检测模型, 进而针对遥感图像本身的难点详细梳理了优化改进方案; 接着介绍了常用的检测数据集, 并对现有方法的性能进行比较; 最后对现阶段问题进行总结并对未来发展趋势进行展望.     

显著性物体检测研究综述

人类的视觉系统能够迅速地、有选择地从视觉场景中检测出感兴趣的目标或者具有显著特征的物体,并根据更高层次的视觉任务目的对它们进行处理和理解,从而实现相应的行为或决策.将人类这种选择性视觉注意机制引入到计算机视觉的信息处理中,可以有效地减少视觉计算所需处理的数据量、加速整个处理过程,并进一步方便更高层次视觉任务的处理,因而该方面的研究受到学术界的广泛关注并应用到计算机视觉的各个领域.首先简单介绍了视觉注意力研究的发展历程,然后综述了显著性物体检测的各种方法,包括传统的方法和基于深度学习的方法,并对这两大类的方法作了进一步的分类和小结.接着,介绍了现有的显著性物体检测的数据集,并详细描述了用于评价检测算法效果的多种评测方法和指标.此外,还探讨了显著性物体检测在不同领域的应用.最后,对显著性物体检测研究的发展趋势和方向进行了分析和总结.     

视觉单目标跟踪算法综述

目标跟踪技术根据视频上下文信息，建立一个跟踪模型对目标的运动状态进行预测，被广泛用于智能视频监控、自动驾驶、机器人导航、人机交互等多个计算机视觉领域。随着深度学习在语音识别，图像分类以及目标检测等领域的巨大成功，越来越多的研究将深度学习框架应用于目标跟踪任务中。介绍了当前单目标跟踪任务的难点和传统的方法,重点分析了当前基于深度学习的单目标跟踪算法的发展现状，从预训练网络＋相关滤波算法、基于孪生网络的方法、基于卷积神经网络的方法、基于生成对抗网络的方法以及其他深度学习方法几个方面，分别对当前流行的深度学习目标跟踪算法进行了概述。此外，总结了用于评测单目标跟踪算法性能的代表性数据集，列举了最新的研究成果在不同数据集上的实验结果并分析了当前单目标跟踪领域的问题和趋势。     

基于深度学习的视频异常检测方法综述

近年来,随着视频监控技术的广泛应用,对海量视频进行智能分析并及时发现其中的异常状态或事件的视频异常检测任务受到了广泛关注。对基于深度学习的视频异常检测方法进行了综述。首先,对视频异常检测问题进行概述,包括基本概念、基本类型、建模流程、学习范式及评价方式。其次,提出将现有基于深度学习的视频异常检测方法分为基于重构的方法、基于预测的方法、基于分类的方法及基于回归的方法4类并详细阐述了各类方法的建模思想、代表性工作及其优缺点。然后,在此基础上介绍了常用的单场景视频异常检测公开数据集和评估指标,并对比分析了代表性异常检测方法的性能。最后,总结全文并从数据集、方法及评估指标3方面对视频异常检测研究的未来发展方向进行了展望。     

基于视觉显著性的均值漂移跟踪算法

为解决突变运动下的目标跟踪问题,提出了一种基于视觉显著性的均值漂移跟踪算法,将视觉注意机制运用到均值漂移跟踪框架中,利用时空显著性算法对视频序列进行检测,生成视觉显著图,从视觉显著图对应的显著性区域中建立目标的颜色特征表示模型来实现运动目标跟踪.实验结果表明:该算法在摄像机摇晃等动态场景下可以较准确检测出时空均显著的目标,有效克服了在运动目标发生丢失和遮挡等情况下跟踪不稳定的问题,具有较强的鲁棒性,从而实现复杂场景下目标较准确的跟踪.     

基于关键点的Anchor Free目标检测模型综述

目标检测是计算机视觉应用的基础, 基于锚框的一些目标检测算法已难以满足目标检测中对目标处理的效率、性能等诸多方面的要求, 而anchor free方法逐渐广泛应用于目标检测. 本文首先重点介绍了CornerNet、CenterNet、FCOS模型的一系列基于关键点的anchor free目标检测方法, 综述了算法思路及其优缺点; 然后分别对基于锚框和基于关键点的目标检测算法在同一个数据集上作了性能比较和分析; 最后对基于关键点的目标检测进行了总结, 并展望了目标检测的未来发展方向.     

How important is location information in saliency detection of natural images

Location information, i.e., the position of content in image plane, is considered as an important supplement in saliency detection. The effect of location information is usually evaluated by integrating it with the selected saliency detection methods and measuring the improvement, which is highly influenced by the selection of saliency methods. In this paper, we provide direct and quantitative analysis of the importance of location information for saliency detection in natural images. We firstly analyze the relationship between content location and saliency distribution on four public image datasets, and validate the distribution by simply treating location based Gaussian distribution as saliency map. To further validate the effectiveness of location information, we propose a location based saliency detection approach, which completely initializes saliency maps with location information and propagate saliency among patches based on color similarity, and discuss the robustness of location information’s effect. The experimental results show that location information plays a positive role in saliency detection, and the proposed method can outperform most state-of-the-art saliency detection methods and handle natural images with different object positions and multiple salient objects.     

光场显著性检测研究综述

显著性检测一直是计算机视觉领域的关键问题,在视觉跟踪、图像压缩和目标识别等方面有着非常重要的应用。基于传统RGB图像和RGB-D （RGB depth）图像的显著性检测易受复杂背景、光照、遮挡等因素影响,在复杂场景的检测精度较低,鲁棒的显著性检测仍存在很大挑战。随着光场成像技术的发展,人们开始从新的途径解决显著性检测问题。光场数据记录着空间光线位置信息和方向信息,隐含场景的几何结构,能为显著性检测提供可靠的背景、深度等先验信息。因此,利用光场数据进行显著性检测得到了广泛关注,成为研究热点。尽管基于光场数据的显著性检测算法陆续出现,但是缺少对该问题的深刻理解以及研究进展的全面综述。本文系统地综述了基于光场数据的显著性检测研究现状,并进行深入探讨和展望。对光场理论以及用于光场显著性检测的公共数据集进行介绍;系统地介绍了光场显著性检测领域的算法模型和最新进展,从人工设计光场特征、稀疏编码特征和深度学习特征等方面进行全面阐述及分析;通过4个公共光场显著性数据集上的实验数据对不同方法的优缺点进行比较和分析,并结合实际应用指出当前研究的局限性与发展趋势。     

Video attention prediction using gaze saliency

In recent years, the significant progress has been achieved in the field of visual saliency modeling. Our research key is in video saliency, which differs substantially from image saliency and could be better detected by adding the gaze information from the movement of eyes while people are looking at the video. In this paper we purposed a novel gaze saliency method to predict video attention, which is inspired by the widespread usage of mobile smart devices with camera. It is a non-contacted method to predict visual attention, and it does not bring the burden on the hardware. Our method first extracts the bottom-up saliency maps from the video frames, and then constructs the mapping from eye images obtained by the camera in synchronization with the video frames to the screen region. Finally the combination between top-down gaze information and bottom-up saliency maps is conducted by point-wise multiplication to predict the video attention. Furthermore, the proposed approach is validated on the two datasets: one is the public dataset MIT, the other is the dataset we collected, versus other four usual methods, and the experiment results show that our method achieves the state-of-the-art.

     

Exploiting contrast cues for salient region detection

Visual saliency detection is an important cue used in human visual system, which can offer efficient solutions for both biological and artificial vision systems. Although there are many saliency detection models that can achieve good results on public datasets, the accuracy and reliability of salient object detection models still remains a challenge. For this reason, a novel effective salient region detection model is presented in this paper. Based on the principle that a combination of global statistics and surrounding contrast saliency operators can yield even better results than just using either alone, we use a histogram-based contrast method to calculate the global saliency values in an opponent color space. At the same time, we partition the input image into a set of regions, and the regional saliency is detected by considering the color isolation with spatial information and textural distinctness simultaneously. The final saliency is obtained based on a weighted fusion of the two saliency results. The experimental results from three widely used databases validate the efficacy of the proposed method in comparison with fourteen state-of-the-art existing methods.     

A New Framework for Multiscale Saliency Detection Based on Image Patches

In this paper, we propose a new multiscale saliency detection algorithm based on image patches. To measure saliency of pixels in a given image, we segment the image into patches by a fixed scale and then use principal component analysis to reduce the dimensions which are noises with respect to the saliency calculation. The dissimilarities between a patch and other patches, which indicate the patch’s saliency, are computed based on the dissimilarity of colors and the spatial distance. Finally, we implement our algorithm through multiple scales that further decrease the saliency of background. Our method is compared with other saliency detection approaches on two public image datasets. Experimental results show that our method outperforms the state-of-the-art methods on predicting human fixations and salient object segmentation.     

多尺度构图先验的显著目标检测

目的 针对基于对比度的显著检测方法,因忽略了特征的空间分布而导致准确性不高的问题,启发于边界先验关于图像空间布局的思想,提出构图先验的显著检测方法。方法 假定目标分布于三分构图线周围,根据相关性比较计算显著值。首先,对图像进行多尺度超像素分割并构造闭环图;其次,提取构图线区域超像素特征并使用Manifold Ranking算法计算显著目标与背景的分布;然后,从目标和背景两个角度对显著值进行细化并利用像素区别性对像素点的显著值进行矫正;最后,融合多尺度显著值得到最终显著图。结果 在公开的MSRA-1000、CSSD、ECSSD数据集上验证本文方法并与其他算法进行对比。本文方法在各数据集上准确率最高,分别为92.6%,89.2%,76.6%。且处理单幅图像平均时间为0.692 s,和其他算法相比也有一定优势。结论 人眼视觉倾向于在构图线周围寻找显著目标,构图先验是根据人眼注意机制研究显著性,具有合理性,且构图先验的方法提高了显著目标检测的准确性。     

Integrating bottom-up and top-down visual stimulus for saliency detection in news video

This paper presents a new attention model for detecting visual saliency in news video. In the proposed model, bottom-up (low level) features and top-down (high level) factors are used to compute bottom-up saliency and top-down saliency respectively. Then, the two saliency maps are fused after a normalization operation. In the bottom-up attention model, we use quaternion discrete cosine transform in multi-scale and multiple color spaces to detect static saliency. Meanwhile, multi-scale local motion and global motion conspicuity maps are computed and integrated into motion saliency map. To effectively suppress the background motion noise, a simple histogram of average optical flow is adopted to calculate motion contrast. Then, the bottom-up saliency map is obtained by combining the static and motion saliency maps. In the top-down attention model, we utilize high level stimulus in news video, such as face, person, car, speaker, and flash, to generate the top-down saliency map. The proposed method has been extensively tested by using three popular evaluation metrics over two widely used eye-tracking datasets. Experimental results demonstrate the effectiveness of our method in saliency detection of news videos compared to several state-of-the-art methods.     

Color boosted visual saliency detection and its application to image classification

For many applications in graphics, design and human computer interaction, it is essential to reliably estimate the visual saliency of images. In this paper, we propose a visual saliency detection method that combines the respective merits of color saliency boosting and global region based contrast schemes to achieve more accurate saliency maps. Our method is compared with existing saliency detection methods when evaluated using four public available datasets. Experimental results show that our method consistently outperformed current state-of-the-art methods on predicting human fixations. We also demonstrate how the extracted saliency map can be used for image classification.     

Adaptive saliency fusion based on quality assessment

A variety of saliency models based on different schemes and methods have been proposed in the recent years, and the performance of these models often vary with images and complement each other. Therefore it is a natural idea whether we can elevate saliency detection performance by fusing different saliency models. This paper proposes a novel and general framework to adaptively fuse saliency maps generated using various saliency models based on quality assessment of these saliency maps. Given an input image and its multiple saliency maps, the quality features based on the input image and saliency maps are extracted. Then, a quality assessment model, which is learned using the boosting algorithm with multiple kernels, indicates the quality score of each saliency map. Next, a linear summation method with power-law transformation is exploited to fuse these saliency maps adaptively according to their quality scores. Finally, a graph cut based refinement method is exploited to enhance the spatial coherence of saliency and generate the high-quality final saliency map. Experimental results on three public benchmark datasets with state-of-the-art saliency models demonstrate that our saliency fusion framework consistently outperforms all saliency models and other fusion methods, and effectively elevates saliency detection performance.     

融合双层信息的显著性检测

目的 针对已有工作在颜色及结构显著性描述方面的缺陷,提出一种新的图像显著性检测方法。方法 本文方法在不同的图像区域表达上从颜色与空间结构角度计算图像的显著性,充分考虑图像的特征与像素聚类方式之间的适应性。首先,根据颜色复杂度、边缘与连通性等信息,将图像从像素空间映射到双层区域表示空间。然后,根据两个层次空间的特性,与每个图像区域的边界特性,计算图像的结构和颜色显著度。最后,由于不同图像表示中的显著性信息存在互补性,将所有这些信息进行融合得到最终的显著性图。结果 在公认的MSRA-1000数据集上验证本文方法并与目前国际上流行的方法进行对比。实验结果表明,本文方法在精确率、召回率以及绝对误差(分别为75.03%、89.39%、85.61%)等方面要优于当前前沿的方法。结论 提出了一种融合双层信息的显著性检测算法。根据图像本身信息控制区域数目构建图像双层表示,提高了方法的普适性;利用图像不同层次的特性从不同角度计算显著性,增强了方法鲁棒性。