Hierarchical Multiview Rigid Registration

Registration is a key step in the 3D reconstruction of real‐world objects. In this paper, we propose a hierarchical method for the rigid registration of multiple views. The multiview registration problem is solved via hierarchical optimization defined on an undirected graph. Each node or edge in this graph represents a single view or a connection between two overlapped views, respectively. The optimizations are performed hierarchically on the edges, the loops, and the entire graph. First, each overlapped pair of views is locally aligned. Then, a loop‐based incremental registration algorithm is introduced to refine the initial pairwise alignments. After a loop is registered, the views in the loop are merged into a metaview in the graph. Finally, global error diffusion is applied to the entire graph to evenly distribute the accumulated errors to all views. In addition, a new objective function is defined to describe the loop closure problem; it improves the accuracy and robustness of registration by simultaneously considering transformation and registration errors. The experimental results show that the proposed hierarchical approach is accurate, efficient and robust for initial view states that are not well posed.     

Encouraging second-order consistency for multiple graph matching

The problems in computer vision of finding the global correspondences across a set of images can be formulated as a multiple graph matching problem consisting of pairwise matching problems. In the multiple graph matching problem, matching consistency is as important as matching accuracy for preventing the contrariety among matched results. Unfortunately, since the majority of conventional pairwise matching methods only approximate the original graph matching problem owing to its computational complexity, a framework that separately matches each graph pair could generate inconsistent results in practical environments. In this paper, we propose a novel multiple graph matching method based on the second-order consistency concept, which simultaneously considers the matching information of all possible graph pairs. We reformulate the multiple graph matching problem to encourage second-order consistency and design an iterative optimization framework. In our experiments, the proposed method outperforms the state-of-the-art methods in terms of both consistency and accuracy.     

Using Bounded Diameter Minimum Spanning Trees to Build Dense Active Appearance Models

We present a method for producing dense active appearance models (AAMs), suitable for video-realistic synthesis. To this end we estimate a joint alignment of all training images using a set of pairwise registrations and ensure that these pairwise registrations are only calculated between similar images. This is achieved by defining a graph on the image set whose edge weights correspond to registration errors and computing a bounded diameter minimum spanning tree. Dense optical flow is used to compute pairwise registration and a flow refinement method to align small scale texture is introduced. Further, given the registration of training images, vertices are added to the AAM to minimise the error between the observed flow fields and the flow fields interpolated between the AAM mesh points. We demonstrate a significant improvement in model compactness.     

Selection of views to materialize in a data warehouse

A data warehouse stores materialized views of data from one or more sources, with the purpose of efficiently implementing decision-support or OLAP queries. One of the most important decisions in designing a data warehouse is the selection of materialized views to be maintained at the warehouse. The goal is to select an appropriate set of views that minimizes total query response time and the cost of maintaining the selected views, given a limited amount of resource, e.g., materialization time, storage space, etc. In This work, we have developed a theoretical framework for the general problem of selection of views in a data warehouse. We present polynomial-time heuristics for a selection of views to optimize total query response time under a disk-space constraint, for some important special cases of the general data warehouse scenario, viz.: 1) an AND view graph, where each query/view has a unique evaluation, e.g., when a multiple-query optimizer can be used to general a global evaluation plan for the queries, and 2) an OR view graph, in which any view can be computed from any one of its related views, e.g., data cubes. We present proofs showing that the algorithms are guaranteed to provide a solution that is fairly close to (within a constant factor ratio of) the optimal solution. We extend our heuristic to the general AND-OR view graphs. Finally, we address in detail the view-selection problem under the maintenance cost constraint and present provably competitive heuristics.     

A Path Following Algorithm for the Graph Matching Problem

We propose a convex-concave programming approach for the labeled weighted graph matching problem. The convex-concave programming formulation is obtained by rewriting the weighted graph matching problem as a least-square problem on the set of permutation matrices and relaxing it to two different optimization problems: a quadratic convex and a quadratic concave optimization problem on the set of doubly stochastic matrices. The concave relaxation has the same global minimum as the initial graph matching problem, but the search for its global minimum is also a hard combinatorial problem. We, therefore, construct an approximation of the concave problem solution by following a solution path of a convex-concave problem obtained by linear interpolation of the convex and concave formulations, starting from the convex relaxation. This method allows to easily integrate the information on graph label similarities into the optimization problem, and therefore, perform labeled weighted graph matching. The algorithm is compared with some of the best performing graph matching methods on four data sets: simulated graphs, QAPLib, retina vessel images, and handwritten Chinese characters. In all cases, the results are competitive with the state of the art.     

Robust point matching for nonrigid shapes by preserving local neighborhood structures

In previous work on point matching, a set of points is often treated as an instance of a joint distribution to exploit global relationships in the point set. For nonrigid shapes, however, the local relationship among neighboring points is stronger and more stable than the global one. In this paper, we introduce the notion of a neighborhood structure for the general point matching problem. We formulate point matching as an optimization problem to preserve local neighborhood structures during matching. Our approach has a simple graph matching interpretation, where each point is a node in the graph, and two nodes are connected by an edge if they are neighbors. The optimal match between two graphs is the one that maximizes the number of matched edges. Existing techniques are leveraged to search for an optimal solution with the shape context distance used to initialize the graph matching, followed by relaxation labeling updates for refinement. Extensive experiments show the robustness of our approach under deformation, noise in point locations, outliers, occlusion, and rotation. It outperforms the shape context and TPS-RPM algorithms on most scenarios.     

Registering multiview range data to create 3D computer objects

Concerns the problem of range image registration for the purpose of building surface models of 3D objects. The registration task involves finding the translation and rotation parameters which properly align overlapping views of the object so as to reconstruct from these partial surfaces, an integrated surface representation of the object. The registration task is expressed as an optimization problem. We define a function which measures the quality of the alignment between the partial surfaces contained in two range images as produced by a set of motion parameters. This function computes a sum of Euclidean distances from control points on one surfaces to corresponding points on the other. The strength of this approach is in the method used to determine point correspondences. It reverses the rangefinder calibration process, resulting in equations which can be used to directly compute the location of a point in a range image corresponding to an arbitrary point in 3D space. A stochastic optimization technique, very fast simulated reannealing (VFSR), is used to minimize the cost function. Dual-view registration experiments yielded excellent results in very reasonable time. A multiview registration experiment took a long time. A complete surface model was then constructed from the integration of multiple partial views. The effectiveness with which registration of range images can be accomplished makes this method attractive for many practical applications where surface models of 3D objects must be constructed     

View selection for designing the global data warehouse

A global data warehouse (DW) integrates data from multiple distributed heterogeneous databases and other information sources. A global DW can be abstractly seen as a set of materialized views. The selection of views for materialization in a DW is an important decision in the design of a DW. Current commercial products do not provide tools for automatic DW design. We provide a general method that, given a set of select-project-join queries to be satisfied by the DW, generates sets of materialized views that satisfy all the input queries. This process is complex since ‘common subexpressions' between the queries need to be detected and exploited. Our method is then applied to solve the problem of selecting such a materialized view set that fits in the space allocated to the DW for materialization and minimizes the combined overall query evaluation and view maintenance cost. We design algorithms which are implemented and we report on their experimental evaluation.     

基于多agent系统的分布式问题求解中的重尾现象

通过随机给变量赋初值,统计采用基于多agent系统的分布式逃逸算法求解约束满足问题所需的求解步数,不同类型问题的求解性能具有很大的差别。已发现一类着色问题的求解,显现出重尾分布的现象。该文提出了一个新的全局调度策略,通过权重的重置,保持权重的相对平衡性,有效地消除了重尾现象。     

Inference Methods for CRFs with Co-occurrence Statistics

The Markov and Conditional random fields (CRFs) used in computer vision typically model only local interactions between variables, as this is generally thought to be the only case that is computationally tractable. In this paper we consider a class of global potentials defined over all variables in the CRF. We show how they can be readily optimised using standard graph cut algorithms at little extra expense compared to a standard pairwise field. This result can be directly used for the problem of class based image segmentation which has seen increasing recent interest within computer vision. Here the aim is to assign a label to each pixel of a given image from a set of possible object classes. Typically these methods use random fields to model local interactions between pixels or super-pixels. One of the cues that helps recognition is global object co-occurrence statistics, a measure of which classes (such as chair or motorbike) are likely to occur in the same image together. There have been several approaches proposed to exploit this property, but all of them suffer from different limitations and typically carry a high computational cost, preventing their application on large images. We find that the new model we propose produces a significant improvement in the labelling compared to just using a pairwise model and that this improvement increases as the number of labels increases.     

优化的多幅眼底图像拼接方法

针对多幅眼底图像拼接的问题,提出一种优化的多幅眼底图像拼接方法。该方法在基于特征的层次鲁棒估计方法完成两两图像初始配准的基础上,提出一种基于重叠区域的配准修正,得到两幅图像之间的良好配准;然后建立图像之间配准关系的无向图,并将配准图像中匹配特征的几何误差和相似度作为图中边的权值,通过计算关系图的最短路径,确定基准图像;再构造基于基准图像的直接约束和基于非基准图像的间接约束的多幅图像配准整体模型,用来生成多幅拼接图像。最后在图像融合部分,提出基于距离变换的多频带融合方法,消除拼接图像中的接缝,达到平滑融合效果。实验结果表明,该方法可有效消除多幅配准的累积误差,实现多幅眼底图像输入顺序无关性的精确配准和无缝拼接。     

Recognition of occluded objects by reducing feature interactions

The main difficulty for the recognition of occluded objects lies in the fact that the original feature set is corrupted and no longer reliable to represent the object of interest. This corruption is caused by the interactions between features from different objects, denoted as feature interactions, which is a key issue addressed in our algorithm. In this paper, a local to global strategy is represented for the occlusion recognition problem, which combines the pairwise grouping and graph matching algorithms. Local appearance similarity serves as priors to reduce feature interactions, by which the performance of graph matching algorithms is improved in order to deal with the contaminated data set. With our formulation, a global decision on object recognition can be made based on locally gathered information. Experimental results show that the proposed framework can dramatically reduce incorrect matches and objects under severe occlusions can still be recognized.     

Simultaneous Registration of Multiple Corresponding Point Sets

We present a new technique for the simultaneous registration of multiple corresponding point sets with rigid 3D transformations. This class of problems is a generalization of the classic pairwise point set registration task, involving multiple views with multiple correspondences existing between them. The proposed technique requires the computation of a constant matrix which encodes the point correspondence information, followed by an efficient iterative algorithm to compute the optimal rotations. The optimal translations are then recovered directly through the solution of a linear equation system. The algorithm supports weighting of data according to confidence, and we show how it may be incorporated into two robust estimation frameworks to detect and reject outlier data. We have integrated our method into a generalized multiview ICP surface matching system and tested it with synthetic and real data. These tests indicate that the technique is accurate and efficient. The algorithm also compares favorably to another multiview technique from the literature.     

基于凸优化改进的相机全局位置估计方法

相机全局位置估计作为运动恢复结构算法（Structure from motion,SfM）中的核心内容一直以来都是计算机视觉领域的研究热点.现有相机全局位置估计方法大多对外点敏感,在处理大规模、无序图像集时表现的尤为明显.增量式SfM中的迭代优化步骤可以剔除大部分的误匹配从而降低外点对估计结果的影响,而全局式SfM中没有有效地剔除误匹配的策略,估计结果受外点影响较大.针对这种情况,本文提出一种改进的相机全局位置估计方法:首先,结合极线约束提出一种新的对误匹配鲁棒的相对平移方向估计算法,减少相对平移方向估计结果中存在的外点;然后,引入平行刚体理论提出一种新的预处理方法将相机全局位置估计转化为一个适定性问题;最后,在此基础上构造了一个对外点鲁棒的凸优化线性估计模型,对模型解算获取相机位置估计全局最优解.本文方法可以很好地融合到当下的全局式SfM流程中.与现有典型方法的对照实验结果表明:在处理大规模、无序图像时,本文方法能显著提高相机全局位置估计的鲁棒性,并保证估计过程的高效性和估计结果的普遍精度.     

Multi-view kernel construction

In many problem domains data may come from multiple sources (or views), such as video and audio from a camera or text on and links to a web page. These multiple views of the data are often not directly comparable to one another, and thus a principled method for their integration is warranted. In this paper we develop a new algorithm to leverage information from multiple views for unsupervised clustering by constructing a custom kernel. We generate a multipartite graph (with the number of parts given by the number of views) that induces a kernel we then use for spectral clustering. Our algorithm can be seen as a generalization of co-clustering and spectral clustering and a relative of Kernel Canonical Correlation Analysis. We demonstrate the algorithm on four data sets: an illustrative artificial data set, synthetic fMRI data, voxels from an fMRI study, and a collection of web pages. Finally, we compare its performance to common alternatives.     

Latent ranking analysis using pairwise comparisons in crowdsourcing platforms

Ranking items is an essential problem in recommendation systems. Since comparing two items is the simplest type of queries in order to measure the relevance of items, the problem of aggregating pairwise comparisons to obtain a global ranking has been widely studied. Furthermore, ranking with pairwise comparisons has recently received a lot of attention in crowdsourcing systems where binary comparative queries can be used effectively to make assessments faster for precise rankings. In order to learn a ranking based on a training set of queries and their labels obtained from annotators, machine learning algorithms are generally used to find the appropriate ranking model which describes the data set the best.In this paper, we propose a probabilistic model for learning multiple latent rankings by using pairwise comparisons. Our novel model can capture multiple hidden rankings underlying the pairwise comparisons. Based on the model, we develop an efficient inference algorithm to learn multiple latent rankings as well as an effective inference algorithm for active learning to update the model parameters in crowdsourcing systems whenever new pairwise comparisons are supplied. The performance study with synthetic and real-life data sets confirms the effectiveness of our model and inference algorithms.     

Globally Consistent Range Scan Alignment for Environment Mapping

A robot exploring an unknown environment may need to build a worldmodel from sensor measurements. In order to integrate all the framesof sensor data, it is essential to align the data properly. Anincremental approach has been typically used in the past, in whicheach local frame of data is aligned to a cumulative global model, andthen merged to the model. Because different parts of the model areupdated independently while there are errors in the registration,such an approach may result in an inconsistent model.In this paper, we study the problem of consistent registration ofmultiple frames of measurements (range scans), together with therelated issues of representation and manipulation of spatialuncertainties. Our approach is to maintain all the local frames ofdata as well as the relative spatial relationships between localframes. These spatial relationships are modeled as random variablesand are derived from matching pairwise scans or from odometry. Thenwe formulate a procedure based on the maximum likelihood criterion tooptimally combine all the spatial relations. Consistency is achievedby using all the spatial relations as constraints to solve for thedata frame poses simultaneously. Experiments with both simulated andreal data will be presented.     

Graph embedding and extensions: a general framework for dimensionality reduction

A large family of algorithms - supervised or unsupervised; stemming from statistics or geometry theory - has been designed to provide different solutions to the problem of dimensionality reduction. Despite the different motivations of these algorithms, we present in this paper a general formulation known as graph embedding to unify them within a common framework. In graph embedding, each algorithm can be considered as the direct graph embedding or its linear/kernel/tensor extension of a specific intrinsic graph that describes certain desired statistical or geometric properties of a data set, with constraints from scale normalization or a penalty graph that characterizes a statistical or geometric property that should be avoided. Furthermore, the graph embedding framework can be used as a general platform for developing new dimensionality reduction algorithms. By utilizing this framework as a tool, we propose a new supervised dimensionality reduction algorithm called marginal Fisher analysis in which the intrinsic graph characterizes the intraclass compactness and connects each data point with its neighboring points of the same class, while the penalty graph connects the marginal points and characterizes the interclass separability. We show that MFA effectively overcomes the limitations of the traditional linear discriminant analysis algorithm due to data distribution assumptions and available projection directions. Real face recognition experiments show the superiority of our proposed MFA in comparison to LDA, also for corresponding kernel and tensor extensions     

Robust self-tuning semi-supervised learning

We investigate the issue of graph-based semi-supervised learning (SSL). The labeled and unlabeled data points are represented as vertices in an undirected weighted neighborhood graph, with the edge weights encoding the pairwise similarities between data objects in the same neighborhood. The SSL problem can be then formulated as a regularization problem on this graph. In this paper we propose a robust self-tuning graph-based SSL method, which (1) can determine the similarities between pairwise data points automatically; (2) is not sensitive to outliers. Promising experimental results are given for both synthetic and real data sets.     

Interactive, graph-based visual analysis of high-dimensional, multi-parameter fluorescence microscopy data in toponomics

In Toponomics, the function protein pattern in cells or tissue (the toponome) is imaged and analyzed for applications in toxicology, new drug development and patient-drug-interaction. The most advanced imaging technique is robot-driven multi-parameter fluorescence microscopy. This technique is capable of co-mapping hundreds of proteins and their distribution and assembly in protein clusters across a cell or tissue sample by running cycles of fluorescence tagging with monoclonal antibodies or other affinity reagents, imaging, and bleaching in situ. The imaging results in complex multi-parameter data composed of one slice or a 3D volume per affinity reagent. Biologists are particularly interested in the localization of co-occurring proteins, the frequency of co-occurrence and the distribution of co-occurring proteins across the cell. We present an interactive visual analysis approach for the evaluation of multi-parameter fluorescence microscopy data in toponomics. Multiple, linked views facilitate the definition of features by brushing multiple dimensions. The feature specification result is linked to all views establishing a focus+context visualization in 3D. In a new attribute view, we integrate techniques from graph visualization. Each node in the graph represents an affinity reagent while each edge represents two co-occurring affinity reagent bindings. The graph visualization is enhanced by glyphs which encode specific properties of the binding. The graph view is equipped with brushing facilities. By brushing in the spatial and attribute domain, the biologist achieves a better understanding of the function protein patterns of a cell. Furthermore, an interactive table view is integrated which summarizes unique fluorescence patterns. We discuss our approach with respect to a cell probe containing lymphocytes and a prostate tissue section.