3D Motion Estimation and Motion Fusion by Affine Region Matching

In this paper,a new method is presented for 3D motion estimation by image region correspondences using stereo cameras.Under the weak perspectivity assumption.we first employ the moment tensor theory (Cyganski and Orr^[11]) to compute the monocular affine transformations relating images taken by the same camera at different time instants and the binocular affine transformations relating images taken by different cameras at the same time instant.We then show that 3D motion can be recovered from these 2D transformations.A space-time fusion strategy is proposed to aim at robust results.No knowledge of point correspondences if requred in the above processes and the computations involved are linear.To find corresponding image regions,new affine invariants,which show stronger invariance,are derived in term of tensor contraction theory.Experiments on real motion images are conducted to verify the proposed method.     

Motion synthesis through 1D affine matching

We present the study of a data-driven motion synthesis approach based on a 1D affine image-matching equation. We start by deriving the relevant properties of the exact matching operator, such as the existence of a singular point. Next, we approximate such operator by the Green’s function of a second-order differential equation, finding that it leads to a more compelling motion impression, due to the incorporation of blur. We then proceed to show that, by judicious choice of the matching parameters, the 1D affine Green’s filter allows the simulation of a broad class of effects, such as zoom-in and zoom-out, and of complex nonrigid motions such as that of a pulsating heart.

Epipolar geometry for prism-based single-lens stereovision

In order to simplify the design and implementation of a stereo vision system, prism has been used to capture stereo images with a single camera. This kind of system not only provides advantages over traditional two-camera stereo, but also reduces the complexity and cost of acquiring stereoscopic image. This paper investigated the characteristics of epipolar geometry for a single-lens prism-based stereovision. The prism was considered as a single optical lens. By analyzing each plane individually and then combining them together, an affine transformation matrix which can express the relationship between an object point and its image was derived. Then, the homography between object point and its image was established. Finally, the epipolar geometry as well as the epipolar rectification method was proposed. Experimental results verify that rectification of the image pair based on our proposed model can achieve better performance with much less geometric distortion.     

运动传感驱动的3D直观手势交互

为了使手势交互方式较少受到场地和光线的限制,提出利用加速度传感器作为输入设备进行手势识别的方法.对每种手势只要求用户做一次示范表演,通过添加噪声等手段来提高训练数据生成的自动化程度;将训练数据经过预处理和特征提取之后用于训练机器学习模型(隐马尔科夫模型和支持向量机).在包含70种手势的测试集上进行实验,平均识别率超过90%;并开发了幻灯片手势控制和手势拨号2个基于手势的人机交互原型系统,结果表明文中方法能够显著地提升用户在人机交互中的体验.     

3D sparse signal recovery via 3D orthogonal matching pursuit

Though many three-dimensional (3D) compressive sensing schemes have been proposed, recovery algorithms in most of these schemes are designed for 1D or 2D signals, which cause some serious drawbacks, e.g., huge memory usage, and high decoder complexity. This paper proposes a 3D separable operator (3DSO) which is able to completely exploit the spatial and spectral correlation to sparsify and samples the 3D signal in three dimensions. A 3D orthogonal matching pursuit (3D-OMP) algorithm is then employed to recover the 3D sparse signal, which is able to reduce the computational complexity of the decoder significantly with guaranteed accuracy. In the proposed algorithm, we represent each 3D signal as a weighted sum of 3D atoms, which allow us to sample the 3D signal with 3D separable sensing operator. Then the best matched atoms are selected to construct the 3D support set, and the 3D signal is optimally recovered from the 3D support set in the sense of the least squares. Experimental results show that the 3D-OMP approach achieves higher recovery quality but requires less computational time than the Kronecker Compressive Sensing (KCS) scheme.     

Epipolar geometry of opti-acoustic stereo imaging

Optical and acoustic cameras are suitable imaging systems to inspect underwater structures, both in regular maintenance and security operations. Despite high resolution, optical systems have limited visibility range when deployed in turbid waters. In contrast, the new generation of high-frequency (MHz) acoustic cameras can provide images with enhanced target details in highly turbid waters, though their range is reduced by one to two orders of magnitude compared to traditional low-/midfrequency (10s-100s KHz) sonar systems. It is conceivable that an effective inspection strategy is the deployment of both optical and acoustic cameras on a submersible platform, to enable target imaging in a range of turbidity conditions. Under this scenario and where visibility allows, registration of the images from both cameras arranged in binocular stereo configuration provides valuable scene information that cannot be readily recovered from each sensor alone. We explore and derive the constraint equations for the epipolar geometry and stereo triangulation in utilizing these two sensing modalities with different projection models. Theoretical results supported by computer simulations show that an opti-acoustic stereo imaging system outperforms a traditional binocular vision with optical cameras, particularly for increasing target distance and (or) turbidity.     

Epipolar geometry estimation based on evolutionary agents

This paper presents a novel approach based on the use of evolutionary agents for epipolar geometry estimation. In contrast to conventional nonlinear optimization methods, the proposed technique employs each agent to denote a minimal subset to compute the fundamental matrix, and considers the data set of correspondences as a 1D cellular environment, in which the agents inhabit and evolve. The agents execute some evolutionary behavior, and evolve autonomously in a vast solution space to reach the optimal (or near optima) result. Then three different techniques are proposed in order to improve the searching ability and computational efficiency of the original agents. Subset template enables agents to collaborate more efficiently with each other, and inherit accurate information from the whole agent set. Competitive evolutionary agent (CEA) and finite multiple evolutionary agent (FMEA) apply a better evolutionary strategy or decision rule, and focus on different aspects of the evolutionary process. Experimental results with both synthetic data and real images show that the proposed agent-based approaches perform better than other typical methods in terms of accuracy and speed, and are more robust to noise and outliers.     

Epipolar geometry from profiles under circular motion

Addresses the problem of motion estimation from profiles (apparent contours) of an object rotating on a turntable in front of a single camera. A practical and accurate technique for solving this problem from profiles alone is developed. It is precise enough to reconstruct the shape of the object. No correspondences between points or lines are necessary. Symmetry of the surface of revolution swept out by the rotating object is exploited to obtain the image of the rotation axis and the homography relating epipolar lines in two views robustly and elegantly. These, together with geometric constraints for images of rotating objects, are used to obtain first the image of the horizon, which is the projection of the plane that contains the camera centers, and then the epipoles, thus fully determining the epipolar geometry of the image sequence. The estimation of this geometry by this sequential approach avoids many of the problems found in other algorithms. The search for the epipoles, by far the most critical step, is carried out as a simple 1D optimization. Parameter initialization is trivial and completely automatic at all stages. After the estimation of the epipolar geometry, the Euclidean motion is recovered using the fixed intrinsic parameters of the camera obtained either from a calibration grid or from self-calibration techniques. Finally, the spinning object is reconstructed from its profiles using the motion estimated in the previous stage. Results from real data are presented, demonstrating the efficiency and usefulness of the proposed methods     

Guided Sampling via Weak Motion Models and Outlier Sample Generation for Epipolar Geometry Estimation

The problem of automatic robust estimation of the epipolar geometry in cases where the correspondences are contaminated with a high percentage of outliers is addressed. This situation often occurs when the images have undergone a significant deformation, either due to large rotation or wide baseline of the cameras. An accelerated algorithm for the identification of the false matches between the views is presented. The algorithm generates a set of weak motion models (WMMs). Each WMM roughly approximates the motion of correspondences from one image to the other. The algorithm represents the distribution of the median of the geometric distances of a correspondence to the WMMs as a mixture model of outlier correspondences and inlier correspondences. The algorithm generates a sample of outlier correspondences from the data. This sample is used to estimate the outlier rate and to estimate the outlier pdf. Using these two pdfs the probability that each correspondence is an inlier is estimated. These probabilities enable guided sampling. In the RANSAC process this guided sampling accelerates the search process. The resulting algorithm when tested on real images achieves a speedup of between one or two orders of magnitude. This work was supported partly by grant 01-99-08430 of the Israeli Space Agency through the Ministry of Science Culture and Sports of Israel.     

三维场景建模的全景外极面图像分析方法

文章提出由非精确摄像机运动下的图像序列建立3D环境模型的全景外极面图像方法,实现了 无特征提取的时空纹理方向精确估计、深度边界确定和遮挡恢复算法.该方法推广并结合了 外极面图像方法和全景图像方法,避免了现有运动分层方法迭代过程中的局部最小化问题,具 有计算和存储效率高、适应性强、算法鲁棒性好的优点.建立的自然景物的真实感三维环境 模型,可用于机器人全局定位的自然路标提取和真实环境虚拟再现的图像合成.     

Anamorphic 3D geometry

An anamorphic image appears distorted from all but a few viewpoints. They have been studied by artists and architects since the early fifteenth century. Computer graphics opens the door to anamorphic 3D geometry. We are not bound by physical reality nor a static canvas. Here we describe a simple method for achieving anamorphoses of 3D objects by utilizing a variation of a simple projective map that is well-known in the computer graphics literature. The novelty of this work is the creation of anamorphic 3D digital models, resulting in a tool for artists and architects.     

基于概率模型的行人四肢自遮挡的检测

针对人体运动跟踪领域中的自遮挡现象, 提出了一种基于概率模型的行人四肢自遮挡的检测算法. 首先, 该方法定义了基于马尔可夫模型的自遮挡状态概率模型; 其次, 利用椭圆肤色模型, 定义了层次肤色模型; 最后, 通过上述模型, 算法将行人四肢自遮挡状态的识别转换为计算自遮挡状态转换概率的过程. 实验表明, 该方法具有较高的准确性.     

基于混合跟踪模型的室内步行人体3D运动估计

针对步行人体3D运动估计过程中的自遮挡问题, 提出了基于混合跟踪模型的粒子滤波算法. 首先, 利用自遮挡状态检测模型, 将步行人体运动划分为四种自遮挡状态; 其次, 根据混合跟踪模型, 针对不同的自遮挡状态, 算法采用不同的跟踪模型; 最后, 为了估计遮挡状态下的人体运动, 算法提出了基于M-估计的在线训练方法 以训练肢体运动相关系数. 经过实验分析, 算法对处 于自遮挡状态下的人体3D运动估计有着良好的效果, 人体3D运动的估计精度得到了提高.     

Structure from Motion: Beyond the Epipolar Constraint

The classic approach to structure from motion entails a clear separation between motion estimation and structure estimation and between two-dimensional (2D) and three-dimensional (3D) information. For the recovery of the rigid transformation between different views only 2D image measurements are used. To have available enough information, most existing techniques are based on the intermediate computation of optical flow which, however, poses a problem at the locations of depth discontinuities. If we knew where depth discontinuities were, we could (using a multitude of approaches based on smoothness constraints) accurately estimate flow values for image patches corresponding to smooth scene patches; but to know the discontinuities requires solving the structure from motion problem first. This paper introduces a novel approach to structure from motion which addresses the processes of smoothing, 3D motion and structure estimation in a synergistic manner. It provides an algorithm for estimating the transformation between two views obtained by either a calibrated or uncalibrated camera. The results of the estimation are then utilized to perform a reconstruction of the scene from a short sequence of images.The technique is based on constraints on image derivatives which involve the 3D motion and shape of the scene, leading to a geometric and statistical estimation problem. The interaction between 3D motion and shape allows us to estimate the 3D motion while at the same time segmenting the scene. If we use a wrong 3D motion estimate to compute depth, we obtain a distorted version of the depth function. The distortion, however, is such that the worse the motion estimate, the more likely we are to obtain depth estimates that vary locally more than the correct ones. Since local variability of depth is due either to the existence of a discontinuity or to a wrong 3D motion estimate, being able to differentiate between these two cases provides the correct motion, which yields the least varying estimated depth as well as the image locations of scene discontinuities. We analyze the new constraints, show their relationship to the minimization of the epipolar constraint, and present experimental results using real image sequences that indicate the robustness of the method.     

Robust structure from motion with affine camera via low-rank matrix recovery

We present a novel approach to structure from motion that can deal with missing data and outliers with an affine camera. We model the corruptions as sparse error. Therefore the structure from motion problem is reduced to the problem of recovering a low-rank matrix from corrupted observations. We first decompose the matrix of trajectories of features into low-rank and sparse components by nuclear-norm and l1-norm minimization, and then obtain the motion and structure from the low-rank components by the classical factorization method. Unlike pervious methods, which have some drawbacks such as depending on the initial value selection and being sensitive to the large magnitude errors, our method uses a convex optimization technique that is guaranteed to recover the low-rank matrix from highly corrupted and incomplete observations. Experimental results demonstrate that the proposed approach is more efficient and robust to large-scale outliers.     

Modeling 3D Euclidean geometry

This article compares five models of 3D Euclidean geometry-not theoretically, but by demonstrating how to implement a simple recursive ray tracer in each of them. It's meant as a tangible case study of the profitability of choosing an appropriate model, discussing the trade-offs between elegance and performance for this particular application. The models we compare are 3D linear algebra, 3D geometric algebra, 4D linear algebra, 4D geometric algebra, and 5D geometric algebra.     

利用仿射几何的仿射不变特征提取方法

提出一种新的基于仿射几何的仿射不变特征提取方法,适合于目标的识别和图像的匹配。算法分3步执行：首先,提取区域的质心和扩展质心,质心和扩展质心的连线把目标区域分割成两部分,再分别计算两部分区域的质心,如此迭代,直到提取出满足要求的质心个数;然后,依次计算四边形的面积,其中四边形的顶点分别为其连线分割目标区域的两个质心和两个分割区域的质心;最后,依次计算各个四边形的面积比,得到仿射不变特征矢量; 另外,仿射变换的参数也可以通过计算提取的质心坐标得到。实验表明,提取的不变特征矢量稳健性好、计算速度快、分类精度高。     

Observability of 3D Motion

This paper examines the inherent difficulties in observing 3D rigid motion from image sequences. It does so without considering a particular estimator. Instead, it presents a statistical analysis of all the possible computational models which can be used for estimating 3D motion from an image sequence. These computational models are classified according to the mathematical constraints that they employ and the characteristics of the imaging sensor (restricted field of view and full field of view). Regarding the mathematical constraints, there exist two principles relating a sequence of images taken by a moving camera. One is the epipolar constraint, applied to motion fields, and the other the positive depth constraint, applied to normal flow fields. 3D motion estimation amounts to optimizing these constraints over the image. A statistical modeling of these constraints leads to functions which are studied with regard to their topographic structure, specifically as regards the errors in the 3D motion parameters at the places representing the minima of the functions. For conventional video cameras possessing a restricted field of view, the analysis shows that for algorithms in both classes which estimate all motion parameters simultaneously, the obtained solution has an error such that the projections of the translational and rotational errors on the image plane are perpendicular to each other. Furthermore, the estimated projection of the translation on the image lies on a line through the origin and the projection of the real translation. The situation is different for a camera with a full (360 degree) field of view (achieved by a panoramic sensor or by a system of conventional cameras). In this case, at the locations of the minima of the above two functions, either the translational or the rotational error becomes zero, while in the case of a restricted field of view both errors are non-zero. Although some ambiguities still remain in the full field of view case, the implication is that visual navigation tasks, such as visual servoing, involving 3D motion estimation are easier to solve by employing panoramic vision. Also, the analysis makes it possible to compare properties of algorithms that first estimate the translation and on the basis of the translational result estimate the rotation, algorithms that do the opposite, and algorithms that estimate all motion parameters simultaneously, thus providing a sound framework for the observability of 3D motion. Finally, the introduced framework points to new avenues for studying the stability of image-based servoing schemes.     

Robust affine structure matching for 3D object recognition

We consider model-based object localization based on local geometric feature matching between the model and the image data. The method is based on geometric constraint analysis, working in transformation space. We present a formal method which guarantees finding all feasible matchings in polynomial time. From there we develop more computationally feasible algorithms based on conservative approximations of the formal method. Additionally, our formalism relates object localization, affine model indexing, and structure from multiple views to one another