Video-based salient object detection via spatio-temporal difference and coherence

Salient object detection aims to extract the attractive objects in images and videos. It can support various robotics tasks and multimedia applications, such as object detection, action recognition and scene analysis. However, efficient detection of salient objects in videos still faces many challenges as compared to that in still images. In this paper, we propose a novel video-based salient object detection method by exploring spatio-temporal characteristics of video content, i.e., spatial-temporal difference and spatial-temporal coherence. First, we initialize the saliency map for each keyframe by deriving spatial-temporal difference from color cue and motion cue. Next, we generate the saliency maps of other frames by propagating the saliency intra and inter frames with the constraint of spatio-temporal coherence. Finally, the saliency maps of both keyframes and non-keyframes are refined in the saliency propagation. In this way, we can detect salient objects in videos efficiently by exploring their spatio-temporal characteristics. We evaluate the proposed method on two public datasets, named SegTrackV2 and UVSD. The experimental results show that our method outperforms the state-of-the-art methods when taking account of both effectiveness and efficiency.     

Salient region detection using efficient wavelet-based textural feature maps

A salient region is part of an image that captures the highest level of attention by the human visual system. In this paper, a new salient region detection method is proposed by linearly combining the feature maps for the L, a and b color channels. Since, the wavelet transform is capable of providing a multi-scale spatial-frequency decomposition of the image, the color feature maps are obtained using this transform. A scheme is proposed whereby the channel feature maps are linearly combined. The weights for the linear combination are determined by making use of the entropy of the channel feature maps and a Gaussian kernel, utilizing the fact that the salient objects are generally clustered and scene-centric. The salient region is further refined by making use of the proximity of the pixels to the centers of gravity in the image feature map. Extensive simulations are conducted in order to evaluate the performance of the proposed saliency detection scheme by applying it to the natural images from several datasets. Experimental results show that the proposed method provides values of precision, recall and F-measure larger than and that of the mean absolute error smaller than those provided by other existing methods. The performance of the proposed salient region detection method is also evaluated on noisy images and it is shown to be robust against noise.     

Continuous monitoring of range spatial keyword query over moving objects

In this paper, we propose an efficient solution for processing continuous range spatial keyword queries over moving spatio-textual objects (namely, CRSK-mo queries). Major challenges in efficient processing of CRSK-mo queries are as follows: (i) the query range is determined based on both spatial proximity and textual similarity; thus a straightforward spatial proximity based pruning of the search space is not applicable as any object far from a query location with a high textual similarity score can still be the answer (and vice versa), (ii) frequent location updates may invalidate a query result, and thus require frequent re-computing of the result set for any object updates. To address these challenges, the key idea of our approach is to exploit the spatial and textual upper bounds between queries and objects to form safe zones (at the client-side) and buffer regions (at the server-side), and then use these bounds to quickly prune objects and queries through smart in-memory data structures. We conduct extensive experiments with a synthetic dataset that verify the effectiveness and efficiency of our proposed algorithm.     

Monochromatic and bichromatic ranked reverse boolean spatial keyword nearest neighbors search

Recently, Reverse k Nearest Neighbors (RkNN) queries, returning every answer for which the query is one of its k nearest neighbors, have been extensively studied on the database research community. But the RkNN query cannot retrieve spatio-textual objects which are described by their spatial location and a set of keywords. Therefore, researchers proposed a RSTkNN query to find these objects, taking both spatial and textual similarity into consideration. However, the RSTkNN query cannot control the size of answer set and to be sorted according to the degree of influence on the query. In this paper, we propose a new problem Ranked Reverse Boolean Spatial Keyword Nearest Neighbors query called Ranked-RBSKNN query, which considers both spatial similarity and textual relevance, and returns t answers with most degree of influence. We propose a separate index and a hybrid index to process such queries efficiently. Experimental results on different real-world and synthetic datasets show that our approaches achieve better performance.     

Exact and approximate flexible aggregate similarity search

Aggregate similarity search, also known as aggregate nearest-neighbor (Ann) query, finds many useful applications in spatial and multimedia databases. Given a group Q of M query objects, it retrieves from a database the objects most similar to Q, where the similarity is an aggregation (e.g., \({{\mathrm{sum}}}\), \(\max \)) of the distances between each retrieved object p and all the objects in Q. In this paper, we propose an added flexibility to the query definition, where the similarity is an aggregation over the distances between p and any subset of \(\phi M\) objects in Q for some support \(0< \phi \le 1\). We call this new definition flexible aggregate similarity search and accordingly refer to a query as a flexible aggregate nearest-neighbor ( Fann ) query. We present algorithms for answering Fann queries exactly and approximately. Our approximation algorithms are especially appealing, which are simple, highly efficient, and work well in both low and high dimensions. They also return near-optimal answers with guaranteed constant-factor approximations in any dimensions. Extensive experiments on large real and synthetic datasets from 2 to 74 dimensions have demonstrated their superior efficiency and high quality.     

Categorical top-<Emphasis Type="Italic">k</Emphasis> spatial influence query

The influence of a spatial facility object depicts the importance of the object in the whole data space. In this paper, we present a novel definition of object influence in applications where objects are of different categories. We study the problem of Spatial Influence Query which considers the contribution of an object in forming functional units consisting of a given set of objects with different categories designated by users. We first show that the problem of spatial influence query is NP-hard with respect to the number of object categories in the functional unit. To tackle the computational hardness, we develop an efficient framework following two main steps, possible participants finding and optimal functional unit computation. Based on this framework, for the first step, novel and efficient pruning techniques are developed based on the nearest neighbor set (NNS) approach. To find the optimal functional unit efficiently, we propose two algorithms, an exact algorithm and an efficient approximate algorithm with performance guarantee. Comprehensive experiments on both real and synthetic datasets demonstrate the effectiveness and efficiency of our techniques.     

Mathematical models and routing algorithms for CAD technological preparation of cutting processes

Resource-conscious technologies for cutting sheet material include the ICP and ECP technologies that allow for aligning fragments of the contours of cutouts. In this work, we show the mathematical model for the problem of cutting out parts with these technologies and algorithms for finding cutting tool routes that satisfy technological constraints. We give a solution for the problem of representing a cutting plan as a plane graph G = (V,F,E), which is a homeomorphic image of the cutting plan. This has let us formalize technological constraints on the trajectory of cutting the parts according to the cutting plan and propose a series of algorithms for constructing a route in the graph G = (V,F,E), which is an image of an admissible trajectory. Using known coordinates of the preimages of vertices of graph G = (V,F,E) and the locations of fragments of the cutting plan that are preimages of edges of graph G = (V,F,E), the resulting route in the graph G = (V,E) can be interpreted as the cutting tool’s trajectory.The proposed algorithms for finding routes in a connected graph G have polynomial computational complexity. To find the optimal route in an unconnected graph G, we need to solve, for every dividing face f of graph G, a travelling salesman problem on the set of faces incident to f.     

Casting an Object with a Core

This paper addresses geometric problems in manufacturing objects by casting. In casting, molten material is poured into the cavity of the cast and allowed to solidify, after which the cast is removed. The cast has two cast parts to be removed in opposite directions. To manufacture more complicated objects, the cast may also have a side core to be removed in a direction skewed to the removal directions for the cast parts. We address the following problem: Given an object and the removal directions for the cast parts and the side core, can a cast be constructed such that the cast parts and the side core can be removed in the directions specified without colliding with the object or each other? We give necessary and sufficient conditions for the problem, as well as a discrete algorithm to perform the test in O(n ³log?n) time for polyhedral objects, where n is the number of vertices, edges, and facets. If the test result is positive, a cast with complexity O(n ³) can be constructed within the same time bound. We also present an example to show that a cast may have Ω(n ³) complexity in the worst case.     

Video parsing via spatiotemporally analysis with images

Effective parsing of video through the spatial and temporal domains is vital to many computer vision problems because it is helpful to automatically label objects in video instead of manual fashion, which is tedious. Some literatures propose to parse the semantic information on individual 2D images or individual video frames, however, these approaches only take use of the spatial information, ignore the temporal continuity information and fail to consider the relevance of frames. On the other hand, some approaches which only consider the spatial information attempt to propagate labels in the temporal domain for parsing the semantic information of the whole video, yet the non-injective and non-surjective natures can cause the black hole effect. In this paper, inspirited by some annotated image datasets (e.g., Stanford Background Dataset, LabelMe, and SIFT-FLOW), we propose to transfer or propagate such labels from images to videos. The proposed approach consists of three main stages: I) the posterior category probability density function (PDF) is learned by an algorithm which combines frame relevance and label propagation from images. II) the prior contextual constraint PDF on the map of pixel categories through whole video is learned by the Markov Random Fields (MRF). III) finally, based on both learned PDFs, the final parsing results are yielded up to the maximum a posterior (MAP) process which is computed via a very efficient graph-cut based integer optimization algorithm. The experiments show that the black hole effect can be effectively handled by the proposed approach.     

动态背景下空时特性均显著的运动目标检测

从序列图像中提取变化区域是运动检测的主要作用,动态背景的干扰严重影响检测结果,使得有效性运动检测成为一项困难工作。受静态图像显著性检测启发,提出了一种新的运动目标检测方法,采用自底向上与自顶向下的视觉计算模型相结合的方式获取图像的空时显著性：先检测出视频序列中的空间显著性,在其基础上加入时间维度,利用改进的三帧差分算法获取具有运动目标的时间显著性,将显著性目标的检测视角由静态图像转换为空时性均显著的运动目标。实验和分析结果表明：新方法在摄像机晃动等动态背景中能较准确检测出空时均显著的运动目标,具有较高的鲁棒性。     

Sparse regularization method for the detection and removal of random-valued impulse noise

In this paper, we propose a novel two-stage algorithm for the detection and removal of random-valued impulse noise using sparse representations. The main aim of the paper is to demonstrate the strength of image inpainting technique for the reconstruction of images corrupted by random-valued impulse noise at high noise densities. First, impulse locations are detected by applying the combination of sparse denoising and thresholding, based on sparse and overcomplete representations of the corrupted image. This stage optimally selects threshold values so that the sum of the number of false alarms and missed detections obtained at a particular noise level is the minimum. In the second stage, impulses, detected in the first stage, are considered as the missing pixels or holes and subsequently these holes are filled-up using an image inpainting method. Extensive simulation results on standard gray scale images show that the proposed method successfully removes random-valued impulse noise with better preservation of edges and other details compared to the existing techniques at high noise ratios.     

A model of best choice under incomplete information

This paper suggests two approaches to the construction of a two-player game of best choice under incomplete information with the choice priority of one player and the equal weights of both players. We consider a sequence of independent identically distributed random variables (x _i , y _i ), i = 1..., n, which represent the quality of incoming objects. The first component is announced to the players and the second component is hidden. Each player chooses an object based on the information available. The winner is the player whose object has a greater sum of the quality components than the opponent’s object. We derive the optimal threshold strategies and compare them for both approaches.     

Flexible integration of multimedia sub-queries with qualitative preferences

Complex multimedia queries, aiming to retrieve from large databases those objects that best match the query specification, are usually processed by splitting them into a set of m simpler sub-queries, each dealing with only some of the query features. To determine which are the overall best-matching objects, a rule is then needed to integrate the results of such sub-queries, i.e., how to globally rank the m-dimensional vectors of matching degrees, or partial scores, that objects obtain on the m sub-queries. It is a fact that state-of-the-art approaches all adopt as integration rule a scoring function, such as weighted average, that aggregates the m partial scores into an overall (numerical) similarity score, so that objects can be linearly ordered and only the highest scored ones returned to the user. This choice however forces the system to compromise between the different sub-queries and can easily lead to miss relevant results. In this paper we explore the potentialities of a more general approach, based on the use of qualitative preferences, able to define arbitrary partial (rather than only linear) orders on database objects, so that a larger flexibility is gained in shaping what the user is looking for. For the purpose of efficient evaluation, we propose two integration algorithms able to work with any (monotone) partial order (thus also with scoring functions): MPO, which delivers objects one layer of the partial order at a time, and iMPO, which can incrementally return one object at a time, thus also suitable for processing top k queries. Our analysis demonstrates that using qualitative preferences pays off. In particular, using Skyline and Region-prioritized Skyline preferences for queries on a real image database, we show that the results we get have a precision comparable to that obtainable using scoring functions, yet they are obtained much faster, saving up to about 70% database accesses.     

Scalable and efficient processing of top-<Emphasis Type="Italic">k</Emphasis> multiple-type integrated queries

In this paper, we define a new class of queries, the top-k multiple-type integrated query (simply, top-k MULTI query). It deals with multiple data types and finds the information in the order of relevance between the query and the object. Various data types such as spatial, textual, and relational data types can be used for the top-k MULTI query. The top-k MULTI query distinguishes itself from the traditional top-k query in that the component scores to calculate final scores are determined dependent of the query. Hence, each component score is calculated only when the query is given for each data type rather than being calculated apriori as in the top-k query. As a representative instance, the traditional top-k spatial keyword query is an instance of the top-k MULTI query. It deals with the spatial data type and text data type and finds the information based on spatial proximity and textual relevance between the query and the object, which is determined only when the query is given. In this paper, we first define the top-k MULTI query formally and define a new specific instance for the top-k MULTI query, the top-k spatial-keyword-relational(SKR) query, by integrating the relational data type into the traditional top-k spatial keyword query. Then, we investigate the processing approaches for the top-k MULTI query. We discuss the scalability of those approaches as new data types are integrated. We also devise the processing methods for the top-k SKR query. Finally, through extensive experiments on the top-k SKR query using real and synthetic data sets, we compare efficiency of the methods in terms of the query performance and storage.     

基于自适应背景模板与空间先验的显著性物体检测方法

目前,显著性检测已成为国内外计算机视觉领域研究的一个热点,但现有的显著性检测算法大多无法有效检测出位于图像边缘的显著性物体.针对这一问题,本文提出了基于自适应背景模板与空间先验的显著性物体检测方法,共包含三个步骤:第一,根据显著性物体在颜色空间上具有稀有性,获取基于自适应背景模板的显著图.将图像分割为超像素块,提取原图的四周边界作为原始背景区域.利用设计的自适应背景选择策略移除原始背景区域中显著的超像素块,获取自适应背景模板.通过计算每个超像素块与自适应背景模板的相异度获取基于自适应背景模板的显著图.并采用基于K-means的传播机制对获取的显著图进行一致性优化;第二,根据显著性物体在空间分布上具有聚集性,利用基于目标中心优先与背景模板抑制的空间先验方法获得空间先验显著图.第三,将获得的两种显著图进行融合得到最终的显著图.在公开数据集MSRA-1000、SOD、ECSSD和新建复杂数据集CBD上进行实验验证,结果证明本文方法能够准确有效地检测出图像中的显著性物体.     

基于背景学习的显著物体检测

目的 显著物体检测的目标是提取给定图像中最能吸引人注意的物体或区域,在物体识别、图像显示、物体分割、目标检测等诸多计算机视觉领域中都有广泛应用。已有的基于局部或者全局对比度的显著物体检测方法在处理内容复杂的图像时,容易造成检测失败,其主要原因可以总结为对比度参考区域设置的不合理。为提高显著物体检测的完整性,提出背景驱动的显著物体检测算法,在显著值估计和优化中充分利用背景先验。方法 首先采用卷积神经网络学习图像的背景分布,然后从得到的背景图中分割出背景区域作为对比度计算参考区域来估计区域显著值。最后,为提高区域显著值的一致性,采用基于增强图模型的优化实现区域显著值的扩散,即在传统k-正则图局部连接的基础上,添加与虚拟节点之间的先验连接和背景区域节点之间的非局部连接,实现背景先验信息的嵌入。结果 在公开的ASD、SED、SOD和THUS-10000数据库上进行实验验证,并与9种流行的算法进行对比。本文算法在4个数据库上的平均准确率、查全率、F-measure和MAE指标分别为0.873 6、0.795 2、0.844 1和0.112 2,均优于当前流行的算法。结论 以背景区域作为对比度计算参考区域可以明显提高前景区域的显著值。卷积神经网络可以有效学习图像的背景分布并分割出背景区域。基于增强图模型的优化可以进一步实现显著值在前景和背景区域的扩散,提高区域显著值的一致性,并抑制背景区域的显著性响应。实验结果表明,本文算法能够准确、完整地检测图像的显著区域,适用于复杂图像的显著物体检测或物体分割应用。     

The direction-constrained <Emphasis Type="Italic">k</Emphasis> nearest neighbor query

Finding k nearest neighbor objects in spatial databases is a fundamental problem in many geospatial systems and the direction is one of the key features of a spatial object. Moreover, the recent tremendous growth of sensor technologies in mobile devices produces an enormous amount of spatio-directional (i.e., spatially and directionally encoded) objects such as photos. Therefore, an efficient and proper utilization of the direction feature is a new challenge. Inspired by this issue and the traditional k nearest neighbor search problem, we devise a new type of query, called the direction-constrained k nearest neighbor (DCkNN) query. The DCkNN query finds k nearest neighbors from the location of the query such that the direction of each neighbor is in a certain range from the direction of the query. We develop a new index structure called MULTI, to efficiently answer the DCkNN query with two novel index access algorithms based on the cost analysis. Furthermore, our problem and solution can be generalized to deal with spatio-circulant dimensional (such as a direction and circulant periods of time such as an hour, a day, and a week) objects. Experimental results show that our proposed index structure and access algorithms outperform two adapted algorithms from existing kNN algorithms.     

Hierarchical salient object detection model using contrast-based saliency and color spatial distribution

Visual saliency is an important cue in human visual system to detect salient objects in natural scenes. It has attracted a lot of research focus in computer vision, and has been widely used in many applications including image retrieval, object recognition, image segmentation, and etc. However, the accuracy of salient object detection model remains a challenge. Accordingly, a hierarchical salient object detection model is presented in this paper. In order to accurately interpret object saliency in image, we propose to investigate distinctive features from a global perspective. Image contrast and color distribution are calculated to generate saliency maps respectively, which are then fused using the principal component analysis. Compared with state-of-the-art models, the proposed model can accurately detect the salient object which conform with the human visual principle. The experimental results from the MSRA database validate the effectiveness of our proposed model.     

中心矩形构图先验的显著目标检测

目的 许多显著目标检测算法侧重从背景角度进行显著性检测,而从前景角度和空间角度进行显著性检测的算法较少,为了解决这个问题,提出了一种基于中心矩形构图先验的显著目标检测算法。方法 假定目标分布在中心矩形构图线附近。首先,对图像进行超像素分割并构造闭环图;其次,提取中心矩形构图线上的超像素特征,并进行流形排序,获取初始显著值;然后,通过基于中心矩形构图线获取的初始显著值确定中心矩形构图交点显著值和紧凑性关系显著值;最后,融合三者获得最终的中心矩形构图先验显著图。结果 通过MSRA-1000,CSSD,ECSSD,THUS-10000数据集对比验证了中心矩形构图先验算法有较高的准确度和最高的F-measure值,整体效果上优于目前先进的几种算法。且处理单幅图像的平均时间为0.673 s,相比与其他算法也有较大优势。结论 从前景角度和空间角度考虑的中心矩形构图先验的显著目标检测算法相比于传统的算法更加具有鲁棒性,无论图像是复杂的还是简单的,都取得很好的检测效果,充分说明算法的有效性。