Attention-driven image interpretation with application to image retrieval

Visual attention, a selective procedure of human's early vision, plays a very important role for humans to understand a scene by intuitively emphasizing some focused regions/objects. Being aware of this, we propose an attention-driven image interpretation method that pops out visual attentive objects from an image iteratively by maximizing a global attention function. In this method, an image can be interpreted as containing several perceptually attended objects as well as a background, where each object has an attention value. The attention values of attentive objectives are then mapped to importance factors so as to facilitate the subsequent image retrieval. An attention-driven matching algorithm is proposed in this paper based on a retrieval strategy emphasizing attended objects. Experiments on 7376 Hemera color images annotated by keywords show that the retrieval results from our attention-driven approach compare favorably with conventional methods, especially when the important objects are seriously concealed by the irrelevant background.     

Contextual Part Analogies in 3D Objects

In this paper we address the problem of finding analogies between parts of 3D objects. By partitioning an object into meaningful parts and finding analogous parts in other objects, not necessarily of the same type, many analysis and modeling tasks could be enhanced. For instance, partial match queries can be formulated, annotation of parts in objects can be utilized, and modeling-by-parts applications could be supported. We define a similarity measure between two parts based not only on their local signatures and geometry, but also on their context within the shape to which they belong. In our approach, all objects are hierarchically segmented (e.g. using the shape diameter function), and each part is given a local signature. However, to find corresponding parts in other objects we use a context enhanced part-in-whole matching. Our matching function is based on bi-partite graph matching and is computed using a flow algorithm which takes into account both local geometrical features and the partitioning hierarchy. We present results on finding part analogies among numerous objects from shape repositories, and demonstrate sub-part queries using an implementation of a simple search and retrieval application. We also demonstrate a simple annotation tool that carries textual tags of object parts from one model to many others using analogies, laying a basis for semantic text based search.     

3D shape metamorphosis based on T-spline level sets

We propose a new method for 3D shape metamorphosis, where the in-between objects are constructed by using T-spline scalar functions. The use of T-spline level sets offers several advantages: First, it is convenient to handle complex topology changes without the need of model parameterization. Second, the constructed objects are smooth (C² in our case). Third, high quality meshes can be easily obtained by using the marching triangulation method. Fourth, the distribution of the degrees of freedom can be adapted to the geometry of the object. Given one source object and one target object, we firstly find a global coordinate transformation to approximately align the two objects. The T-spline control grid is adaptively generated according to the geometry of the aligned objects, and the initial T-spline level set is found by approximating the signed distance function of the source object. Then we use an evolution process, which is governed by a combination of the signed distance function of the target object and a curvature-dependent speed function, to deform the T-spline level set until it converges to the target shape. Additional intermediate objects are inserted at the beginning/end of the sequence of generated T-spline level sets, by gradually projecting the source/target object to the initial/final T-spline level set. A fully automatic algorithm is developed for the above procedures. Experimental results are presented to demonstrate the effectiveness of our method.     

Optimal Image and Video Closure by Superpixel Grouping

Detecting independent objects in images and videos is an important perceptual grouping problem. One common perceptual grouping cue that can facilitate this objective is the cue of contour closure, reflecting the spatial coherence of objects in the world and their projections as closed boundaries separating figure from background. Detecting contour closure in images consists of finding a cycle of disconnected contour fragments that separates an object from its background. Searching the entire space of possible groupings is intractable, and previous approaches have adopted powerful perceptual grouping heuristics, such as proximity and co-curvilinearity, to constrain the search. We introduce a new formulation of the problem, by transforming the problem of finding cycles of contour fragments to finding subsets of superpixels whose collective boundary has strong edge support (few gaps) in the image. Our cost function, a ratio of a boundary gap measure to area, promotes spatially coherent sets of superpixels. Moreover, its properties support a global optimization procedure based on parametric maxflow. Extending closure detection to videos, we introduce the concept of spatiotemporal closure. Analogous to image closure, we formulate our spatiotemporal closure cost over a graph of spatiotemporal superpixels. Our cost function is a ratio of motion and appearance discontinuity measures on the boundary of the selection to an internal homogeneity measure of the selected spatiotemporal volume. The resulting approach automatically recovers coherent components in images and videos, corresponding to objects, object parts, and objects with surrounding context, providing a good set of multiscale hypotheses for high-level scene analysis. We evaluate both our image and video closure frameworks by comparing them to other closure detection approaches, and find that they yield improved performance.     

Learning Style Compatibility Between Objects in a Real‐World 3D Asset Database

Large 3D asset databases are critical for designing virtual worlds, and using them effectively requires techniques for efficient querying and navigation. One important form of query is search by style compatibility: given a query object, find others that would be visually compatible if used in the same scene. In this paper, we present a scalable, learning‐based approach for solving this problem which is designed for use with real‐world 3D asset databases; we conduct experiments on 121 3D asset packages containing around 4000 3D objects from the Unity Asset Store. By leveraging the structure of the object packages, we introduce a technique to synthesize training labels for metric learning that work as well as human labels. These labels can grow exponentially with the number of objects, allowing our approach to scale to large real‐world 3D asset databases without the need for expensive human training labels. We use these synthetic training labels in a metric learning model that analyzes the in‐engine rendered appearance of an object—combining geometry, material, and texture—whereas prior work considers only object geometry, or disjoint geometry and texture features. Through an ablation experiment, we find that using this representation yields better results than using renders which lack texture, materiality, or both.     

Caching on the World Wide Web

With the recent explosion in usage of the World Wide Web, the problem of caching Web objects has gained considerable importance. Caching on the Web differs from traditional caching in several ways. The nonhomogeneity of the object sizes is probably the most important such difference. In this paper, we give an overview of caching policies designed specifically for Web objects and provide a new algorithm of our own. This new algorithm can be regarded as a generalization of the standard LRU algorithm. We examine the performance of this and other Web caching algorithms via event- and trace-driven simulation     

Categorical top-<Emphasis Type="Italic">k</Emphasis> spatial influence query

The influence of a spatial facility object depicts the importance of the object in the whole data space. In this paper, we present a novel definition of object influence in applications where objects are of different categories. We study the problem of Spatial Influence Query which considers the contribution of an object in forming functional units consisting of a given set of objects with different categories designated by users. We first show that the problem of spatial influence query is NP-hard with respect to the number of object categories in the functional unit. To tackle the computational hardness, we develop an efficient framework following two main steps, possible participants finding and optimal functional unit computation. Based on this framework, for the first step, novel and efficient pruning techniques are developed based on the nearest neighbor set (NNS) approach. To find the optimal functional unit efficiently, we propose two algorithms, an exact algorithm and an efficient approximate algorithm with performance guarantee. Comprehensive experiments on both real and synthetic datasets demonstrate the effectiveness and efficiency of our techniques.     

Plane-based optimization for 3D object reconstruction from single line drawings

In previous optimization-based methods of 3D planar-faced object reconstruction from single 2D line drawings, the missing depths of the vertices of a line drawing (and other parameters in some methods) are used as the variables of the objective functions. A 3D object with planar faces is derived by finding values for these variables that minimize the objective functions. These methods work well for simple objects with a small number N of variables. As N grows, however, it is very difficult for them to find expected objects. This is because with the nonlinear objective functions in a space of large dimension N, the search for optimal solutions can easily get trapped into local minima. In this paper, we use the parameters of the planes that pass through the planar faces of an object as the variables of the objective function. This leads to a set of linear constraints on the planes of the object, resulting in a much lower dimensional nullspace where optimization is easier to achieve. We prove that the dimension of this nullspace is exactly equal to the minimum number of vertex depths which define the 3D object. Since a practical line drawing is usually not an exact projection of a 3D object, we expand the nullspace to a larger space based on the singular value decomposition of the projection matrix of the line drawing. In this space, robust 3D reconstruction can be achieved. Compared with two most related methods, our method not only can reconstruct more complex 3D objects from 2D line drawings, but also is computationally more efficient.     

Real-time monocular object SLAM

We present a real-time object-based SLAM system that leverages the largest object database to date. Our approach comprises two main components: (1) a monocular SLAM algorithm that exploits object rigidity constraints to improve the map and find its real scale, and (2) a novel object recognition algorithm based on bags of binary words, which provides live detections with a database of 500 3D objects. The two components work together and benefit each other: the SLAM algorithm accumulates information from the observations of the objects, anchors object features to especial map landmarks and sets constrains on the optimization. At the same time, objects partially or fully located within the map are used as a prior to guide the recognition algorithm, achieving higher recall. We evaluate our proposal on five real environments showing improvements on the accuracy of the map and efficiency with respect to other state-of-the-art techniques.     

Approximate Bayesian methods for kernel-based object tracking

A framework for real-time tracking of complex non-rigid objects is presented. The object shape is approximated by an ellipse and its appearance by histogram based features derived from local image properties. An efficient search procedure is used to find the image region with a histogram most similar to the histogram of the tracked object. The procedure is a natural extension of the mean-shift procedure with Gaussian kernel which allows handling the scale and orientation changes of the object. The presented procedure is integrated into a set of Bayesian filtering schemes. We compare the regular and mixture Kalman filter and other sequential importance sampling (particle filtering) techniques.     

Oscillation analysis for salient object detection

Salient object detection from an image is important for many multimedia applications. Existing methods provide good solutions to saliency detection; however, their results often emphasize the high-contrast edges, instead of regions/objects. In this paper, we present a method for salient object detection based on oscillation analysis. Our study shows that salient objects and their backgrounds have different amplitudes of oscillation between the local minima and maxima. Based on this observation, our method analyzes the oscillation in an image by estimating its local minima and maxima and computes the saliency map according to the oscillation magnitude contrast. Our method detects the local minima and maxima and performs extreme interpolation to smoothly propagate these information to the whole image. In this way, the oscillation information is smoothly assigned to regions, retaining well-defined salient boundaries as there are large variations near the salient boundaries (edges between objects and their backgrounds). As a result, our saliency map highlights salient regions/objects instead of high-contrast boundaries. We experiment with our method on two large public data set. Our results demonstrate the effectiveness of our method. We further apply our salient object detection method to automatic salient object segmentation, which again shows the success.     

利用非高速摄像机测量转速的方法研究

对有些测量对象根本无法安装测量设备或者设备的主人不允许安装测量设备,比如测量乒乓球的旋转速度等情况;因此需要引进一种新的转速测量方法,即基于视频的角速度测量方法;本研究的目标就是利用两台价格低廉的普通摄像机取代价格昂贵的高速专业摄像机;通过两台低速摄像机同时对高速旋转对象进行采样,两台低速摄影机分别设置不同的采样频率;根据获得的采样图像,发现两组不同采样数据之间关系,建立相应的数学方程,计算出高速旋转对象的旋转角速度;通过实验,利用两台最高30帧/秒速度的摄像机,测量转速为100转/秒左右旋转对象,取得相当准确的精度的,因此,证明了该方法具有很好的实际应用价值.     

一种基于模糊融合的遥感图像目标识别方法

数据融合利用多传感器的信息,克服了单一传感器信息不完整、不精确、不确定的缺点,因此广泛应用于目标识别中,该文提出了一种基于模糊融合的遥感图像目标识别的新方法。首先在单源图像上提取可疑目标,然后根据目标在不同类型图像上的成像特点,选择合适的目标特征,充分考虑到各特征的重要程度,把模糊隶属度函数和模糊密度结合起来,最后利用特征层模糊融合对目标的身份进行判定。此方法应用在实际目标的识别中,取得了很好的效果。     

Preference-based content replacement using recency-latency tradeoff

This work introduces and establishes a new model for cache management, where clients suggest preferences regarding their expectations for the time they are willing to wait, and the level of obsolescence they are willing to tolerate. The cache uses these preferences to decide upon entrance and exit of objects to and from its storage, and select the best copy of requested object among all available copies (fresh, cached, remote). We introduce three replacement policies, each evicts objects based on ongoing scores, considering users’ preferences combined with other objects’ properties such as size, obsolescence rate and popularity. Each replacement algorithm follows a different strategy: (a) an optimal solution that use dynamic programming approach to find the best objects to be kept (b) another optimal solution that use branch and bound approach to find the worst objects to be thrown out (c) an algorithm that use heuristic approach to efficiently select the objects to be evicted. Using these replacement algorithms the cache is able to keep the objects that are best suited for users preferences and dump the other objects. We compare our proposed algorithms to the Least-Recently-Used algorithm, and provide evidence to the advantages of our algorithms providing better service to cache’s users with less burden on network resources and reduced workloads on origin servers.     

Calculus of nonrigid surfaces for geometry and texture manipulation

We present a geometric framework for automatically finding intrinsic correspondence between three-dimensional nonrigid objects. We model object deformation as near isometries and find the correspondence as the minimum-distortion mapping. A generalization of multidimensional scaling is used as the numerical core of our approach. As a result, we obtain the possibility to manipulate the extrinsic geometry and the texture of the objects as vectors in a linear space. We demonstrate our method on the problems of expression-invariant texture mapping onto an animated three-dimensional face, expression exaggeration, morphing between faces, and virtual body painting.     

Shape matching of two-dimensional objects

In this paper we present results in the areas of shape matching of nonoccluded and occluded two-dimensional objects. Shape matching is viewed as a ``segment matching' problem. Unlike the previous work, the technique is based on a stochastic labeling procedure which explicitly maximizes a criterion function based on the ambiguity and inconsistency of classification. To reduce the computation time, the technique is hierarchical and uses results obtained at low levels to speed up and improve the accuracy of results at higher levels. This basic technique has been extended to the situation where various objects partially occlude each other to form an apparent object and our interest is to find all the objects participating in the occlusion. In such a case several hierarchical processes are executed in parallel for every object participating in the occlusion and are coordinated in such a way that the same segment of the apparent object is not matched to the segments of different actual objects. These techniques have been applied to two-dimensional simple closed curves represented by polygons and the power of the techniques is demonstrated by the examples taken from synthetic, aerial, industrial and biological images where the matching is done after using the actual segmentation methods.     

Dynamic Template Tracking and Recognition

In this paper we address the problem of tracking non-rigid objects whose local appearance and motion changes as a function of time. This class of objects includes dynamic textures such as steam, fire, smoke, water, etc., as well as articulated objects such as humans performing various actions. We model the temporal evolution of the object’s appearance/motion using a linear dynamical system. We learn such models from sample videos and use them as dynamic templates for tracking objects in novel videos. We pose the problem of tracking a dynamic non-rigid object in the current frame as a maximum a-posteriori estimate of the location of the object and the latent state of the dynamical system, given the current image features and the best estimate of the state in the previous frame. The advantage of our approach is that we can specify a-priori the type of texture to be tracked in the scene by using previously trained models for the dynamics of these textures. Our framework naturally generalizes common tracking methods such as SSD and kernel-based tracking from static templates to dynamic templates. We test our algorithm on synthetic as well as real examples of dynamic textures and show that our simple dynamics-based trackers perform at par if not better than the state-of-the-art. Since our approach is general and applicable to any image feature, we also apply it to the problem of human action tracking and build action-specific optical flow trackers that perform better than the state-of-the-art when tracking a human performing a particular action. Finally, since our approach is generative, we can use a-priori trained trackers for different texture or action classes to simultaneously track and recognize the texture or action in the video.     

Learning of grasp selection based on shape-templates

The ability to grasp unknown objects still remains an unsolved problem in the robotics community. One of the challenges is to choose an appropriate grasp configuration, i.e., the 6D pose of the hand relative to the object and its finger configuration. In this paper, we introduce an algorithm that is based on the assumption that similarly shaped objects can be grasped in a similar way. It is able to synthesize good grasp poses for unknown objects by finding the best matching object shape templates associated with previously demonstrated grasps. The grasp selection algorithm is able to improve over time by using the information of previous grasp attempts to adapt the ranking of the templates to new situations. We tested our approach on two different platforms, the Willow Garage PR2 and the Barrett WAM robot, which have very different hand kinematics. Furthermore, we compared our algorithm with other grasp planners and demonstrated its superior performance. The results presented in this paper show that the algorithm is able to find good grasp configurations for a large set of unknown objects from a relatively small set of demonstrations, and does improve its performance over time.     

Function from motion

In order for a robot to operate autonomously in its environment, it must be able to perceive its environment and take actions based on these perceptions. Recognizing the functionalities of objects is an important component of this ability. In this paper, we look into a new area of functionality recognition: determining the function of an object from its motion. Given a sequence of images of a known object performing some function, we attempt to determine what that function is. We show that the motion of an object, when combined with information about the object and its normal uses, provides us with strong constraints on possible functions that the object might be performing