Generalised Epipolar Constraints

In this paper we will discuss structure and motion problems for curved surfaces. These will be studied using the silhouettes or apparent contours in the images. The problem of determining camera motion from the apparent contours of curved three-dimensional surfaces, is studied. It will be shown how special points, called epipolar tangency points or frontier points, can be used to solve this problem. A generalised epipolar constraint is introduced, which applies to points, curves, as well as to apparent contours of surfaces. The theory is developed for both continuous and discrete motion, known and unknown orientation, calibrated and uncalibrated, perspective, weak perspective and orthographic cameras. Results of an iterative scheme to recover the epipolar line structure from real image sequences using only the outlines of curved surfaces, is presented. A statistical evaluation is performed to estimate the stability of the solution. It is also shown how the motion of the camera from a sequence of images can be obtained from the relative motion between image pairs.     

Image Registration, Optical Flow and Local Rigidity

We address the theoretical problems of optical flow estimation and image registration in a multi-scale framework in any dimension. Much work has been done based on the minimization of a distance between a first image and a second image after applying deformation or motion field. Usually no justification is given about convergence of the algorithm used. We start by showing, in the translation case, that convergence to the global minimum is made easier by applying a low pass filter to the images hence making the energy convex enough. In order to keep convergence to the global minimum in the general case, we introduce a local rigidity hypothesis on the unknown deformation. We then deduce a new natural motion constraint equation (MCE) at each scale using the Dirichlet low pass operator. This transforms the problem to solving the energy minimization in a finite dimensional subspace of approximation obtained through Fourier Decomposition. This allows us to derive sufficient conditions for convergence of a new multi-scale and iterative motion estimation/registration scheme towards a global minimum of the usual nonlinear energy instead of a local minimum as did all previous methods. Although some of the sufficient conditions cannot always be fulfilled because of the absence of the necessary a priori knowledge on the motion, we use an implicit approach. We illustrate our method by showing results on synthetic and real examples in dimension 1 (signal matching, Stereo) and 2 (Motion, Registration, Morphing), including large deformation experiments.     

Learned Models for Estimation of Rigid and Articulated Human Motion from Stationary or Moving Camera

We propose an approach for modeling, measurement and tracking of rigid and articulated motion as viewed from a stationary or moving camera. We first propose an approach for learning temporal-flow models from exemplar image sequences. The temporal-flow models are represented as a set of orthogonal temporal-flow bases that are learned using principal component analysis of instantaneous flow measurements. Spatial constraints on the temporal-flow are then incorporated to model the movement of regions of rigid or articulated objects. These spatio-temporal flow models are subsequently used as the basis for simultaneous measurement and tracking of brightness motion in image sequences. Then we address the problem of estimating composite independent object and camera image motions. We employ the spatio-temporal flow models learned through observing typical movements of the object from a stationary camera to decompose image motion into independent object and camera motions. The performance of the algorithms is demonstrated on several long image sequences of rigid and articulated bodies in motion.     

A Continuous Probabilistic Framework for Image Matching

In this paper we describe a probabilistic image matching scheme in which the image representation is continuous and the similarity measure and distance computation are also defined in the continuous domain. Each image is first represented as a Gaussian mixture distribution and images are compared and matched via a probabilistic measure of similarity between distributions. A common probabilistic and continuous framework is applied to the representation as well as the matching process, ensuring an overall system that is theoretically appealing. Matching results are investigated and the application to an image retrieval system is demonstrated.     

A New Linear Method for Camera Self-Calibration with Planar Motion

We consider the self-calibration (affine and metric reconstruction) problem from images acquired with a camera with unchanging internal parameters undergoing planar motion. The general self-calibration methods (modulus constraint, Kruppa equations) are known to fail with this camera motion. In this paper we give two novel linear constraints on the coordinates of the plane at infinity in a projective reconstruction for any camera motion. In the planar case, we show that the two constraints are equivalent and easy to compute, giving us a linear version of the quartic modulus constraint. Using this fact, we present a new linear method to solve the self-calibration problem with planar motion of the camera from three or more images. This work was partly supported by project BFM2003-02914 from the Ministerio de Ciencia y Tecnología (Spain). Ferran Espuny received the MSc in Mathematics in 2002 from the Universitat de Barcelona, Spain. He is currently a PhD student and associate professor in the Departament d’àlgebra i Geometria at Universitat de Barcelona, Spain. His research, supervised by Dr. José Ignacio Burgos Gil, is focussed on self-calibration and critical motions for both pinhole and generic camera models.     

半稠密光流法的图像跟踪匹配算法

针对传统光流法图像特征缺失、边界及遮挡等处容易导致目标跟踪质量降低或丢失的问题,提出一种半稠密光流法实现图像特征的稳定跟踪.首先计算出图像中像素梯度变化较大的像素区域;其次利用时变图像灰度的时空梯度函数来计算像素的速度矢量,进而实现像素区域的跟踪;最后将状态向量作为剔除跟踪失败的依据,保留跟踪质量优良的像素区域.结果表...     

Self-Calibration from Image Derivatives

This study investigates the problem of estimating camera calibration parameters from image motion fields induced by a rigidly moving camera with unknown parameters, where the image formation is modeled with a linear pinhole-camera model. The equations obtained show the flow to be separated into a component due to the translation and the calibration parameters and a component due to the rotation and the calibration parameters. A set of parameters encoding the latter component is linearly related to the flow, and from these parameters the calibration can be determined.However, as for discrete motion, in general it is not possible to decouple image measurements obtained from only two frames into translational and rotational components. Geometrically, the ambiguity takes the form of a part of the rotational component being parallel to the translational component, and thus the scene can be reconstructed only up to a projective transformation. In general, for full calibration at least four successive image frames are necessary, with the 3D rotation changing between the measurements.The geometric analysis gives rise to a direct self-calibration method that avoids computation of optical flow or point correspondences and uses only normal flow measurements. New constraints on the smoothness of the surfaces in view are formulated to relate structure and motion directly to image derivatives, and on the basis of these constraints the transformation of the viewing geometry between consecutive images is estimated. The calibration parameters are then estimated from the rotational components of several flow fields. As the proposed technique neither requires a special set up nor needs exact correspondence it is potentially useful for the calibration of active vision systems which have to acquire knowledge about their intrinsic parameters while they perform other tasks, or as a tool for analyzing image sequences in large video databases.     

Gradient Based Image Motion Estimation Without Computing Gradients

Computing an optical flow field using the classical image motion constraint equation is difficult owing to the aperture problem and the need to compute the image intensity derivatives via numerical differentiation—an extremely unstable operation. We integrate the above constraint equation over a significant spatio-temporal support and use Gauss's Divergence theorem to replace the volume integrals by surface integrals, thereby eliminating the intensity derivatives and numerical differentiation. We tackle the aperture problem by fitting an affine flow field model to a small space-time window. Using this affine model our new integral motion constraint approach leads to a robust and accurate algorithm to compute the optical flow field. Extensive experimentation confirms that the algorithm is indeed robust and accurate.     

基于多角度图像匹配的手机拍照场景识别研究

针对目前手机拍照识别过程中难以识别大角度旋转和俯仰角度变化目标的问题,设计并实现了能够容忍图像旋转和扭转变化的改进型SIFT关键点匹配算法;该算法通过模拟目标任意多角度水平旋转和垂直扭转,提前计算多组SIFT特征;在手机拍照识别景观建筑时,匹配目标图像的SIFT特征与多组特征从而识别目标多个侧面;实验结果表明,该识别系统对任意角度拍摄的手机图像的识别准确率达80%以上,计算耗时仅10s,具有一定的实用性和推广价值。     

Temporal Multi-Scale Models for Flow and Acceleration

A model for computing image flow in image sequences containing a very wide range of instantaneous flows is proposed. This model integrates the spatio-temporal image derivatives from multiple temporal scales to provide both reliable and accurate instantaneous flow estimates. The integration employs robust regression and automatic scale weighting in a generalized brightness constancy framework. In addition to instantaneous flow estimation the model supports recovery of dense estimates of image acceleration and can be readily combined with parameterized flow and acceleration models. A demonstration of performance on image sequences of typical human actions taken with a high frame-rate camera is given.     

Design and Use of Linear Models for Image Motion Analysis

Linear parameterized models of optical flow, particularly affine models, have become widespread in image motion analysis. The linear model coefficients are straightforward to estimate, and they provide reliable estimates of the optical flow of smooth surfaces. Here we explore the use of parameterized motion models that represent much more varied and complex motions. Our goals are threefold: to construct linear bases for complex motion phenomena; to estimate the coefficients of these linear models; and to recognize or classify image motions from the estimated coefficients. We consider two broad classes of motions: i) generic motion features such as motion discontinuities and moving bars; and ii) non-rigid, object-specific, motions such as the motion of human mouths. For motion features we construct a basis of steerable flow fields that approximate the motion features. For object-specific motions we construct basis flow fields from example motions using principal component analysis. In both cases, the model coefficients can be estimated directly from spatiotemporal image derivatives with a robust, multi-resolution scheme. Finally, we show how these model coefficients can be use to detect and recognize specific motions such as occlusion boundaries and facial expressions.     

How Optimal Depth Cue Integration Depends on the Task

The aim of this paper is to explore a linear geometric algorithm for recovering the three dimensional motion of a moving camera from image velocities. Generic similarities and differences between the discrete approach and the differential approach are clearly revealed through a parallel development of an analogous motion estimation theory previously explored in Vieville, T. and Faugeras, O.D. 1995. In Proceedings of Fifth International Conference on Computer Vision, pp. 750–756; Zhuang, X. and Haralick, R.M. 1984. In Proceedings of the First International Conference on Artificial Intelligence Applications, pp. 366–375. We present a precise characterization of the space of differential essential matrices, which gives rise to a novel eigenvalue-decomposition-based 3D velocity estimation algorithm from the optical flow measurements. This algorithm gives a unique solution to the motion estimation problem and serves as a differential counterpart of the well-known SVD-based 3D displacement estimation algorithm for the discrete case. Since the proposed algorithm only involves linear algebra techniques, it may be used to provide a fast initial guess for more sophisticated nonlinear algorithms (Ma et al., 1998c. Electronic Research Laboratory Memorandum, UC Berkeley, UCB/ERL(M98/37)). Extensive simulation results are presented for evaluating the performance of our algorithm in terms of bias and sensitivity of the estimates with respect to different noise levels in image velocity measurements.     

基于抽象匹配流的彩色图像配准模型

在抽象匹配流框架下,构造能够克服大色差问题的彩色图像配准模型.该模型中,数据项采用互相关函数作为2幅图像间的相似性度量,以解决大色差问题;正则项采用各向异性扩散滤波器约束图像演化,从而实现在演化过程中对图像特征的有效保持.扩散滤波器中的扩散系数定义为关于彩色结构张量的函数,以使图像演化能够综合各通道信息,解决了各通道所得位移场不一致而引起的色彩混迭问题.实验结果表明,文中模型对具有大色差的彩色图像能够实现有效配准.     

基于卫星图像数据的大范围风场重建

在机载气象雷达早期开发中,数值天气模型是一个非常重要的环节,而天气模型中风场的准确性直接影响整个模型的准确性。为解决数值天气模型中的连续图像运动分析问题,提出一种基于运动场Helmholtz分解的低维流体运动估计方法。通过少量涡流粒子和源粒子的演化,得到光流场的低维参数化表达,即非旋旋转量和螺旋量基函数的线性组合,其中基函数由格林核梯度构成,系数值和基函数参数通过最小化代价函数获得。实验结果表明,与传统光流法相比,该方法的计算速度快了近4倍,风场更为准确地反映了实际的天气状况,在风场反演中更为可靠。     

Image Processing for Sequences of Oceanographic Images

This paper is concerned with three main problems of image processing occuring on temporal sequences of satellite oceanographic images: approximate localization of the interesting structures on the images; segmentation or determination of the boundary of the structures; and temporal tracking of these boundaries to illustrate their evolution. In this paper, application is done on vortices and results are displayed all over the paper. For this purpose, we propose a new method for optical flow computation and interpretation and a geometric modelling of the structures. Oceanographic images obtained from environmental satellite platforms present a new challenge in computer science. The huge amount of data collected each day and the need for characterizing some specific structures on these images for oceanographic monitoring justify our approach for the detection and tracking of vortices on oceanographic images.     

心脏序列图像运动估计新方法:基于广义模糊梯度矢量流场的形变曲线运动估计与跟踪

应用动态轮廓线模型(ACM)解决心脏运动估计问题是该领域的主要研究方法之一．采用经典外力和传统ACM模型对感兴趣边缘进行搜索及跟踪时,普遍存在模型的局部适应性程度不高的缺陷．为解决这一挑战性难题,该文提出了广义模糊梯度矢量流(GFGVF)的概念,并构造出一组新的Snake平衡方程,该方程可对心脏内部边缘逐帧进行鲁棒跟踪．为进一步跟踪每一特征点的运动,该文将前一步的轮廓跟踪结果作为似然条件,结合一致性和连续性先验条件,通过最大后验概率(MAP)的方法对整个过程进行了优化计算．通过对MR及CT两类心脏序列图像进行运动跟踪实验并对计算结果进行多种比较,此方法显示了较好的鲁棒性．     

A region-based selective optical flow back-projection for genuine motion vector estimation

Motion vector plays one significant feature in moving object segmentation. However, the motion vector in this application is required to represent the actual motion displacement, rather than regions of visually significant similarity. In this paper, region-based selective optical flow back-projection (RSOFB) which back-projects optical flows in a region to restore the region's motion vector from gradient-based optical flows, is proposed to obtain genuine motion displacement. The back-projection is performed based on minimizing the projection mean square errors of the motion vector on gradient directions. As optical flows of various magnitudes and directions provide various degrees of reliability in the genuine motion restoration, the optical flows to be used in the RSOFB are optimally selected based on their sensitivity to noises and their tendency in causing motion estimation errors. In this paper a deterministic solution is also derived for performing the minimization and obtaining the genuine motion magnitude and motion direction.     

基于时空结构张量的高速公路视频车辆检测

详细分析高速公路车辆运动模式与视频监控序列图像的特征,本文针对目标运动速度较大时,背景差分法运算复杂和连续帧差法容易产生虚影的情况,提出基于时空结构张量的光流分析法进行高速公路视频车辆检测。实验结果表明,该算法简洁并且能对高速公路视频序列中的运动车辆进行较准确的分割,为下一步的运动跟踪提供了可靠的依据。