Motion from occlusions

In computer vision, occlusions are almost always seen as undesirable singularities that pose difficult challenges to image motion analysis problems, such as optic flow computation, motion segmentation, disparity estimation, or egomotion estimation. However, it is well known that occlusions are extremely powerful cues for depth or motion perception, and could be used to improve those methods.

In this paper, we propose to recover camera motion information based uniquely on occlusions, by observing two specially useful properties: occlusions are independent of the camera rotation, and reveal direct information about the camera translation.

We assume a monocular observer, undergoing general rotational and translational motion in a static environment. We present a formal model for occlusion points and develop a method suitable for occlusion detection. Through the classification and analysis of the detected occlusion points, we show how to retrieve information about the camera translation (FOE). Experiments with real images are presented and discussed in the paper.     

Motion segmentation and depth ordering using an occlusion detector

We present a novel method for motion segmentation and depth ordering from a video sequence in general motion. We first compute motion segmentation based on differential properties of the spatio-temporal domain, and scale-space integration. Given a motion boundary, we describe two algorithms to determine depth ordering from two- and three- frame sequences. An remarkable characteristic of our method is its ability compute depth ordering from only two frames. The segmentation and depth ordering algorithms are shown to give good results on 6 real sequences taken in general motion. We use synthetic data to show robustness to high levels of noise and illumination changes; we also include cases where no intensity edge exists at the location of the motion boundary, or when no parametric motion model can describe the data. Finally, we describe human experiments showing that people, like our algorithm, can compute depth ordering from only two frames, even when the boundary between the layers is not visible in a single frame.     

Motion estimation and segmentation

In the general structure-from-motion (SFM) problem involving several moving objects in a scene, the essential first step is to segment moving objects independently. We attempt to deal with the problem of optical flow estimation and motion segmentation over a pair of images. We apply a mean field technique to determine optical flow and motion boundaries and present a deterministic algorithm. Since motion discontinuities represented by line process are embedded in the estimation of the optical flow, our algorithm provides accurate estimates of optical flow especially along motion boundaries and handles occlusion and multiple motions. We show that the proposed algorithm outperforms other well-known algorithms in terms of estimation accuracy and timing.     

分区域去运动模糊

去运动模糊一直是计算机视觉领域中面向画质增强的一个热点研究方向。模糊核的估算是去运动模糊中的关键问题。提出一种新的思路,即首先将模糊图像按照模糊核的相似度进行图像分割,再对分割后的图像应用空间不变去模糊算法。本文方法主要包含以下几个步骤:分离输入图像中的光照、颜色和纹理信息;分割图像;分区域估算模糊核,计算重叠区域模糊核,并根据计算出的模糊核进行分区域单核去模糊;利用重叠区域整合拼接去模糊结果并还原光照和颜色信息。实验结果表明本文方法比基于单核的去运动模糊算法效果要好。     

多模型鲁棒估计的运动分割

提出了一种基于多模型结构鲁棒估计的运动分割算法。首先对视频处理对象进行基于 四叉树的分裂合并,获取鲁棒估计的初始运动数目以及相应的运动模型的初始参数,然后通过参数估 计,不断更新模型参数,之后通过把每个运动区域和几个运动模型相关联,来同时估计多个运动的区 域,最后通过小物体的运动检测方法检测出小的运动物体,最终达到分割的目的。试验证明该算法取 得了比较明显的结果。     

融合多特征的运动一致性图像分割

目的:在彩色图像分割中,光流法能够得到运动区域,但难以获得运动目标准确的分割边界,而常用的算法往往会产生过分割。为了克服光流法的不足,在保留显著性区域的同时抑制过分割,从而获得具有运动一致性区域的分割结果,提出融合多特征的运动一致性图像分割算法。方法:首先通过Mean Shift算法获取图像的初始分割,然后利用空域信息（包括颜色、边缘和区域面积）对视觉感知上具有相似性的区域进行合并,再利用时域信息进行运动一致性区域合并,最终得到分割结果。结果:实验结果表明通过结合时空信息,该方法能够有效抑制过分割,不仅弥补了光流场不能准确提取目标边缘的不足,而且提高了分割目标的完整性。结论:与两种流行的彩色图像分割算法相比,所提方法获得了更加理想的结果。     

Motion cues and saliency based unconstrained video segmentation

Multimedia Tools and Applications - The segmentation of moving objects become challenging when the object motion is small, the shape of object changes, and there is global background motion in...     

一种基于结构张量的运动物体分割方法

视频中运动物体的分割是众多视频应用的前提。视频可以看作是三维时空中的对象,通过邻域的灰度值的结构方向来分析视频的运动。提出了时空结构张量进行运动物体分割的方法。首先分析了视频中物体的运动与灰度的变化的关系,然后删除了三维结构张量中与时间无关的信息,并将此时空结构张量用于视频中运动物体的分割。实验表明时空结构张量运动物体分割方法相对于传统的结构张量,分割准确,计算简单。     

基于运动估计的交互式医学图像序列分割

针对医学图像难以自动分割,而医学图像序列采用手工分割时工作量巨大、效率低的问题,提出了一种新的交互式图像序列分割方法.在计算机的辅助下,用手工精确地描画出第一幅图像中对象的边界轮廓.后续图像的分割曲线用运动估计的方法自动得到.每完成一幅图像的分割用户都可以检查分割效果,如果不满意则可用手工修正.这个过程重复进行,直到整个图像序列分割完毕.实验结果表明,该方法能精确、快速地实现医学图像序列的分割.     

Motion segmentation of multiple translating objects from line correspondences

This paper presents an analytic approach to motion segmentation of multiple translating objects from line correspondences in three perspective views. The basic idea of our algorithm is to view the estimation of multiple translational motions as the estimation of a single, though more complex, multibody motion model that is then factored into the original models by polynomial differentiation. Experimental results on synthetic and real scenes are presented.     

Motion segmentation and pose recognition with motion history gradients

This paper presents a fast and simple method using a timed motion history image (tMHI) for representing motion from the gradients in successively layered silhouettes. This representation can be used to (a) determine the current pose of the object and (b) segment and measure the motions induced by the object in a video scene. These segmented regions are not “motion blobs”, but instead are motion regions that are naturally connected to parts of the moving object. This method may be used as a very general gesture recognition “toolbox”. We demonstrate the approach with recognition of waving and overhead clapping motions to control a music synthesis program. Accepted: 13 August 2001     

Motion estimation,segmentation and separation,using hypercomplex phase correlation,clustering techniques and graph-based optimization

The problem of estimating the motions in image sequences has been extensively studied. However, most of the proposed motion estimation approaches deal only with the luminance component of the images, although the color information is also important. The usual way to apply these techniques in color image sequences is to process each color channel separately. As proposed by Ell, Sangwine and Moxey, a more sophisticated approach is to handle the color channels in a “holistic” manner, using quaternions. We present a motion estimation approach which is based on the Hypercomplex (quaternionic) Phase Correlation of Moxey et al. We show that the quaternionic approach is computationally more efficient and more robust to sensor noise, compared with the corresponding three-channels-separately approach. With the assumption of smoothly time-varying velocities we propose the application of a weighted fuzzy c-means clustering procedure to the obtained velocity estimates. This renders the estimation more robust. A methodology in the hypercomplex Fourier transform domain for separating the moving objects/layers is also presented. An energy-minimization-based approach for the spatial assignment of the velocities and the creation of segmentation maps, is given. We furthermore show how to apply the motion estimation approach locally in space for the extraction of dense motion fields. Our experimental results and comparisons with state-of-the-art methodologies verify the effectiveness of the proposed approach.     

Shape segmentation using relaxation

Relaxation is applied to the segmentation of closed boundary curves of shapes. The ambiguous segmentation of the boundary is represented by a directed graph structure whose nodes represent segments, where two nodes are joined by an arc if the segments are consecutive along the boundary. A probability vector is associated with each node; each component of this vector provides an estimate of the probability that the corresponding segment is a particular part of the object. Relaxation is used to eliminate impossible sequences of parts, or reduce the probabilities of unlikely ones. In experiments involving airplane shapes, this almost always results in a drastic simplification of the graph with only good interpretations surviving. The approach is also extended to include curve linking and gap filling. A chain coded input image is broken into segments based on a measure of local curvature. Gap completions linking pairs of segments are then proposed and represented in a graph structure. A second graph, whose nodes consist of paths in the above graph, is constructed, and the nodes of the second graph are probabilistically classified as various object parts. Relaxation is then applied to increase the probability of mutually supporting classifications, and decrease the probability of unsupported decisions. A modified relaxation process using information about the size, spatial position, and orientation of the object parts yielded a high degree of disambiguation.     

Motion segmentation based on motion/brightness integration andoscillatory correlation

A segmentation method based on the integration of motion and brightness is proposed for image sequences. The method is composed of two parallel pathways that process motion and brightness, respectively, Inspired by the visual system, the motion pathway has two stages. The first stage estimates local motion at locations with reliable information. The second stage performs segmentation based on local motion estimates. In the brightness pathway, the input scene is segmented into regions based on brightness distribution. Subsequently, segmentation results from the two pathways are integrated to refine motion estimates. The final segmentation is performed in the motion network based on refined estimates. For segmentation, locally excitatory globally inhibitory oscillator network (LEGION) architecture is employed whereby the oscillators corresponding to a region of similar motion/brightness oscillate in synchrony and different regions attain different phases. Results on synthetic and real image sequences are provided, and comparisons with other methods are made.     

Feature-aligned segmentation using correlation clustering

Computational Visual Media - We present an algorithm for segmenting a mesh into patches whose boundaries are aligned with prominent ridge and valley lines of the shape. Our key insight is that this...     

Image segmentation using evolutionary computation

Image segmentation denotes a process by which a raw input image is partitioned into nonoverlapping regions such that each region is homogeneous and the union of any two adjacent regions is heterogeneous. A segmented image is considered to be the highest domain-independent abstraction of an input image. The image segmentation problem is treated as one of combinatorial optimization. A cost function which incorporates both edge information and region gray-scale uniformity is defined. The cost function is shown to be multivariate with several local minima. The genetic algorithm, a stochastic optimization technique based on evolutionary computation, is explored in the context of image segmentation. A class of hybrid evolutionary optimization algorithms based on a combination of the genetic algorithm and stochastic annealing algorithms such as simulated annealing, microcanonical annealing, and the random cost algorithm is shown to exhibit superior performance as compared with the canonical genetic algorithm. Experimental results on gray-scale images are presented     

Multiseeded segmentation using fuzzy connectedness

Fuzzy connectedness has been effectively used to segment out an object in a badly corrupted image. We generalize the approach by providing a definition which is shown to always determine a simultaneous segmentation of multiple objects. For any set of seed points, the segmentation is uniquely determined by the definition. An algorithm for finding this segmentation is presented and its output is illustrated. The algorithm is fast as compared to other segmentation algorithms in current use. We also report on an evaluation of the accuracy and robustness of the algorithm based on experiments in which several users were repeatedly asked to identify the seed points for the algorithm in a number of images     

Texture segmentation using Voronoi polygons

Textures are defined in terms of primitives called tokens. A texture segmentation algorithm based on the Voronoi tessellation is discussed. The algorithm first builds the Voronoi tessellation of the tokens that make up the textured image. It then computes a feature vector for each Voronoi polygon. These feature vectors are used in a probabilistic relaxation labeling on the tokens, to identify the interior and the border regions of the textures. Some experimental results are shown     

基于时空联合双重约束Snake算法的运动目标分割

提出了一种针对运动目标进行分割的STC（Spatio Temporal Combined） Snake算法。该方法利用待分割帧图像的灰度梯度及其和相邻帧图像的时域信息,构造一种时空联合双重约束的外部能量函数,实现对Snake曲线的变形和收敛。对Snake轮廓进行初始化时,首先将相邻帧图像进行减运算,提取出大致的运动区域,然后再以该区域的外接矩形的长和宽为轴长,在该区域上构造一个椭圆,等间距提取该椭圆形上的N个点,形成Snake的初始化轮廓。实验结果表明,该方法是有效可行的,可精确的分割出非刚体的运动目标。     

Region-based pose tracking with occlusions using 3D models

Despite great progress achieved in 3-D pose tracking during the past years, occlusions and self-occlusions are still an open issue. This is particularly true in silhouette-based tracking where even visible parts cannot be tracked as long as they do not affect the object silhouette. Multiple cameras or motion priors can overcome this problem. However, multiple cameras or appropriate training data are not always readily available. We propose a framework in which the pose of 3-D models is found by minimising the 2-D projection error through minimisation of an energy function depending on the pose parameters. This framework makes it possible to handle occlusions and self-occlusions by tracking multiple objects and object parts simultaneously. Therefore, each part is described by its own image region each of which is modeled by one probability density function. This allows to deal with occlusions explicitly, which includes self-occlusions between different parts of the same object as well as occlusions between different objects. The results we present for simulations and real-world scenes demonstrate the improvements achieved in monocular and multi-camera settings. These improvements are substantiated by quantitative evaluations, e.g. based on the HumanEVA benchmark.