Occluded Video Instance Segmentation: A Benchmark

International Journal of Computer Vision - Can our video understanding systems perceive objects when a heavy occlusion exists in a scene? To answer this question, we collect a large-scale dataset...     

Dynamic graph cuts for efficient inference in Markov Random Fields

Abstract-In this paper we present a fast new fully dynamic algorithm for the st-mincut/max-flow problem. We show how this algorithm can be used to efficiently compute MAP solutions for certain dynamically changing MRF models in computer vision such as image segmentation. Specifically, given the solution of the max-flow problem on a graph, the dynamic algorithm efficiently computes the maximum flow in a modified version of the graph. The time taken by it is roughly proportional to the total amount of change in the edge weights of the graph. Our experiments show that, when the number of changes in the graph is small, the dynamic algorithm is significantly faster than the best known static graph cut algorithm. We test the performance of our algorithm on one particular problem: the object-background segmentation problem for video. It should be noted that the application of our algorithm is not limited to the above problem, the algorithm is generic and can be used to yield similar improvements in many other cases that involve dynamic change.     

Robust Higher Order Potentials for Enforcing Label Consistency

This paper proposes a novel framework for labelling problems which is able to combine multiple segmentations in a principled manner. Our method is based on higher order conditional random fields and uses potentials defined on sets of pixels (image segments) generated using unsupervised segmentation algorithms. These potentials enforce label consistency in image regions and can be seen as a generalization of the commonly used pairwise contrast sensitive smoothness potentials. The higher order potential functions used in our framework take the form of the Robust P ⁿ model and are more general than the P ⁿ Potts model recently proposed by Kohli et al. We prove that the optimal swap and expansion moves for energy functions composed of these potentials can be computed by solving a st-mincut problem. This enables the use of powerful graph cut based move making algorithms for performing inference in the framework. We test our method on the problem of multi-class object segmentation by augmenting the conventional crf used for object segmentation with higher order potentials defined on image regions. Experiments on challenging data sets show that integration of higher order potentials quantitatively and qualitatively improves results leading to much better definition of object boundaries. We believe that this method can be used to yield similar improvements for many other labelling problems.     

Global Stereo Reconstruction under Second-Order Smoothness Priors

Second-order priors on the smoothness of 3D surfaces are a better model of typical scenes than first-order priors. However, stereo reconstruction using global inference algorithms, such as graph cuts, has not been able to incorporate second-order priors because the triple cliques needed to express them yield intractable (nonsubmodular) optimization problems. This paper shows that inference with triple cliques can be effectively performed. Our optimization strategy is a development of recent extensions to alpha-expansion, based on the ldquo QPBOrdquo algorithm. The strategy is to repeatedly merge proposal depth maps using a novel extension of QPBO. Proposal depth maps can come from any source, for example, frontoparallel planes as in alpha-expansion, or indeed any existing stereo algorithm, with arbitrary parameter settings.     

IMPSAC: synthesis of importance sampling and random sample consensus

This paper proposes a new method for recovery of epipolar geometry and feature correspondence between images which have undergone a significant deformation, either due to large rotation or wide baseline of the cameras. The method also encodes the uncertainty by providing an arbitrarily close approximation to the posterior distribution of the two view relation. The method operates on a pyramid from coarse to fine resolution, thus raising the problem of how to propagate information from one level to another in a statistically consistent way. The distribution of the parameters at each resolution is encoded nonparametrically as a set of particles. At the coarsest level, a random sample consensus Monte Carlo Markov chain (RANSAC-MCMC) estimator is used to initialize this set of particles, the posterior can then be approximated as a mixture of Gaussians fitted to these particles. The distribution at a coarser level influences the distribution at a finer level using the technique of sampling-importance-resampling (SIR) and MCMC, which allows for asymptotically correct approximations of the posterior distribution. The estimate of the posterior distribution at the level above is being used as the importance sampling function to generate a new set of particles, which can be further improved by MCMC. It is shown that the method is superior to previous single resolution RANSAC-style feature matchers.     

Sparsity and persistence: mixed norms provide simple signal models with dependent coefficients

Sparse regression often uses ℓ _p norm priors (with p < 2). This paper demonstrates that the introduction of mixed-norms in such contexts allows one to go one step beyond in signal models, and promote some different, structured, forms of sparsity. It is shown that the particular case of the ℓ_1,2 and ℓ_2,1 norms leads to new group shrinkage operators. Mixed norm priors are shown to be particularly efficient in a generalized basis pursuit denoising approach, and are also used in a context of morphological component analysis. A suitable version of the Block Coordinate Relaxation algorithm is derived for the latter. The group-shrinkage operators are then modified to overcome some limitations of the mixed-norms. The proposed group shrinkage operators are tested on simulated signals in specific situations, to illustrate and compare their different behaviors. Results on real data are also used to illustrate the relevance of the approach.     

The Problem of Degeneracy in Structure and Motion Recovery from Uncalibrated Image Sequences

The aim of this work is the recovery of 3D structure and camera projection matrices for each frame of an uncalibrated image sequence. In order to achieve this, correspondences are required throughout the sequence. A significant and successful mechanism for automatically establishing these correspondences is by the use of geometric constraints arising from scene rigidity. However, problems arise with such geometry guided matching if general viewpoint and general structure are assumed whilst frames in the sequence and/or scene structure do not conform to these assumptions. Such cases are termed degenerate.In this paper we describe two important cases of degeneracy and their effects on geometry guided matching. The cases are a motion degeneracy where the camera does not translate between frames, and a structure degeneracy where the viewed scene structure is planar. The effects include the loss of correspondences due to under or over fitting of geometric models estimated from image data, leading to the failure of the tracking method. These degeneracies are not a theoretical curiosity, but commonly occur in real sequences where models are statistically estimated from image points with measurement error.We investigate two strategies for tackling such degeneracies: the first uses a statistical model selection test to identify when degeneracies occur: the second uses multiple motion models to overcome the degeneracies. The strategies are evaluated on real sequences varying in motion, scene type, and length from 13 to 120 frames.     

Carbonation of ternary building cementing materials

The carbonation processes of ettringite and calcium aluminate hydrates phases developed by hydration of calcium aluminate cement, fly ash and calcium sulphate ternary mixtures have been studied. The hydrated samples were submitted to 4% of CO₂ in a carbonation chamber, and were analysed, previous carbonation and after 14 and 90 days of carbonation time, by infrared spectroscopy and X-ray diffraction; the developed morphology was performed with the 14 days carbonated samples. The results evidenced that ettringite reacts with CO₂ after 14 days of exposition time and evolves totally at 90 days; the developed hydrated phases C₃AH₆ in samples with major CAC content, also reacts with CO₂. Due to carbonation, calcium carbonate – mainly vaterite but also aragonite-, depending on the initial formulation, aluminium hydroxide and gypsum were detected.