Pose estimation from multiple cameras based on Sylvester’s equation

In this paper, we introduce a method to estimate the object’s pose from multiple cameras. We focus on direct estimation of the 3D object pose from 2D image sequences. Scale-Invariant Feature Transform (SIFT) is used to extract corresponding feature points from adjacent images in the video sequence. We first demonstrate that centralized pose estimation from the collection of corresponding feature points in the 2D images from all cameras can be obtained as a solution to a generalized Sylvester’s equation. We subsequently derive a distributed solution to pose estimation from multiple cameras and show that it is equivalent to the solution of the centralized pose estimation based on Sylvester’s equation. Specifically, we rely on collaboration among the multiple cameras to provide an iterative refinement of the independent solution to pose estimation obtained for each camera based on Sylvester’s equation. The proposed approach to pose estimation from multiple cameras relies on all of the information available from all cameras to obtain an estimate at each camera even when the image features are not visible to some of the cameras. The resulting pose estimation technique is therefore robust to occlusion and sensor errors from specific camera views. Moreover, the proposed approach does not require matching feature points among images from different camera views nor does it demand reconstruction of 3D points. Furthermore, the computational complexity of the proposed solution grows linearly with the number of cameras. Finally, computer simulation experiments demonstrate the accuracy and speed of our approach to pose estimation from multiple cameras.     

Estimation of object motion parameters from noisy images

An approach is presented for the estimation of object motion parameters based on a sequence of noisy images. The problem considered is that of a rigid body undergoing unknown rotational and translational motion. The measurement data consists of a sequence of noisy image coordinates of two or more object correspondence points. By modeling the object dynamics as a function of time, estimates of the model parameters (including motion parameters) can be extracted from the data using recursive and/or batch techniques. This permits a desired degree of smoothing to be achieved through the use of an arbitrarily large number of images. Some assumptions regarding object structure are presently made. Results are presented for a recursive estimation procedure: the case considered here is that of a sequence of one dimensional images of a two dimensional object. Thus, the object moves in one transverse dimension, and in depth, preserving the fundamental ambiguity of the central projection image model (loss of depth information). An iterated extended Kalman filter is used for the recursive solution. Noise levels of 5-10 percent of the object image size are used. Approximate Cramer-Rao lower bounds are derived for the model parameter estimates as a function of object trajectory and noise level. This approach may be of use in situations where it is difficult to resolve large numbers of object match points, but relatively long sequences of images (10 to 20 or more) are available.     

On recognizing and positioning curved 3-D objects from imagecontours

An approach for explicitly relating the shape of image contours to models of curved three-dimensional objects is presented. This relationship is used for object recognition and positioning. Object models consist of collections of parametric surface patches and their intersection curves; this includes nearly all representations used in computer-aided geometric design and computer vision. The image contours considered are the projections of surface discontinuities and occluding contours. Elimination theory provides a method for constructing the implicit equation of these contours for an object observed under orthographic or perspective projection. This equation is parameterized by the object's position and orientation with respect to the observer. Determining these parameters is reduced to a fitting problem between the theoretical contour and the observed data points. The proposed approach readily extends to parameterized models. It has been implemented for a simple world composed of various surfaces of revolution and tested on several real images     

In the wild image retrieval and clustering for 3D cultural heritage landmarks reconstruction

One of the main characteristics of Internet era is the free and online availability of extremely large collections of images located on distributed and heterogeneous platforms over the web. The proliferation of millions of shared photographs spurred the emergence of new image retrieval techniques based not only on images’ visual information, but on geo-location tags and camera exif data. These huge visual collections provide a unique opportunity for cultural heritage documentation and 3D reconstruction. The main difficulty, however, is that the internet image datasets are unstructured containing many outliers. For this reason, in this paper a new content-based image filtering is proposed to discard image outliers that either confuse or significantly delay the followed e-documentation tools, such as 3D reconstruction of a cultural heritage object. The presented approach exploits and fuses two unsupervised clustering techniques: DBSCAN and spectral clustering. DBSCAN algorithm is used to remove outliers from the initially retrieved dataset and spectral clustering discriminate the noise free image dataset into different categories each representing characteristic geometric views of cultural heritage objects. To discard the image outliers, we consider images as points onto a multi-dimensional manifold and the multi-dimensional scaling algorithm is adopted to relate the space of the image distances with the space of Gram matrices through which we are able to compute the image coordinates. Finally, structure from motion is utilized for 3D reconstruction of cultural heritage landmarks. Evaluation on a dataset of about 31,000 cultural heritage images being retrieved from internet collections with many outliers indicate the robustness and cost effectiveness of the proposed method towards a reliable and just-in-time 3D reconstruction than existing state-of-the-art techniques.     

Dynamic procedures of image reconstruction from projections (computer tomography)

A method of reconstruction of images from projections is suggested. In contrast to the static procedure conventionally used for the solution of problems of image reconstruction from projections, this method presupposes the use of a dynamic procedure. Such an approach, in combination with the use of special noise-immune algorithms of image reconstruction, permits obtaining images of the internal structure of the object under study of a high quality by irradiating it sequentially in time by the flows of photons of a small intensity. The effectiveness of the suggested method is illustrated by the example of results of a computer experiment.     

Topological equivalence between a 3D object and the reconstruction of its digital image

Digitization is not as easy as it looks. If one digitizes a 3D object even with a dense sampling grid, the reconstructed digital object may have topological distortions and, in general, there exists no upper bound for the Hausdorff distance. This explains why so far no algorithm has been known which guarantees topology preservation. However, as we will show, it is possible to repair the obtained digital image in a locally bounded way so that it is homeomorphic and close to the 3D object. The resulting digital object is always well-composed, which has nice implications for a lot of image analysis problems. Moreover, we will show that the surface of the original object is homeomorphic to the result of the marching cubes algorithm. This is really surprising since it means that the well-known topological problems of the marching cubes reconstruction simply do not occur for digital images of r-regular objects. Based on the trilinear interpolation, we also construct a smooth isosurface from the digital image that has the same topology as the original surface. Finally, we give a surprisingly simple topology preserving reconstruction method by using overlapping balls instead of cubical voxels. This is the first approach of digitizing 3D objects which guarantees topology preservation and gives an upper bound for the geometric distortion. Since the output can be chosen as a pure voxel presentation, a union of balls, a reconstruction by trilinear interpolation, a smooth isosurface, or the piecewise linear marching cubes surface, the results are directly applicable to a huge class of image analysis algorithms. Moreover, we show how one can efficiently estimate the volume and the surface area of 3D objects by looking at their digitizations. Measuring volume and surface area of digital objects are important problems in 3D image analysis. Good estimators should be multigrid convergent, i.e., the error goes to zero with increasing sampling density. We will show that every presented reconstruction method can be used for volume estimation and we will give a solution for the much more difficult problem of multigrid-convergent surface area estimation. Our solution is based on simple counting of voxels and we are the first to be able to give absolute bounds for the surface area.     

一种基于非线性扩散方程的PET重建方法

正电子发射断层重建是一类不适定问题,因此正则化方法常用来抑制噪声提高重建的质量.依据Mumford-Shah泛函模型提出了该类重建问题的统一变分模型.基于这一模型,提出了一种新的基于扩散方程的PET重建算法,同时给出了线性和非线性两种方法.实验结果表明,两种方法均有良好的重建性能,并且非线性方法通过合理的设置参数,能体现出较为优越的抗噪声性能和边缘保持特性.     

摄像机简化模型对三维重构的影响--分析与实验

讨论了摄像机简化模型对三维重构的影响．主要结论有：当摄像机在两幅图像间的运动为纯平移运动时,从理论上证明了使用摄像机简化模型重构空间点与实际空间点之间满足仿射变换;当摄像机在两幅图像间的运动为一般刚体运动时,使用简化模型的重构只有在一定条件下才能较好地保持原物体的形状;在简化模型下,基于Kruppa方程的方法所估计的焦距精度不能满足三维重构的要求．实验结果表明：在三维重构中不能盲目地使用简化模型,必须对摄像机内参数进行全面标定．     

On using CAD models to compute the pose of curved 3D objects

A new approach is presented for explicitly relating image observables to models of curved three-dimensional objects. This relationship is used for object recognition and positioning. Object models consist of collections of parametric surface patches. The image observables considered are raw range data, surface normal and Gaussian curvature, raw image intensity and intensity gradient, raw image contours, and contour orientation and curvature. Elimination theory provides a method for constructing an implicit equation that relates these observables to the three-dimensional position and orientation of object models. Determining the unknown pose parameters is reduced to a fitting problem between the implicit equation and the observed data points. By considering translation-independent observables such as surface normal and curvature, this process is further decomposed into first determining orientation and then determining translation. Applications to object recognition are described, and an implementation is presented.     

基于反对称双正交小波重构的图像增强方法

详细给出了基于反对称双正交小波重构的多尺度边缘检测方法的相关理论基础, 即推导了反对称双正交小波变换所具有的卷积运算性质; 分析了反对称双正交小波变换的微分算子功能; 提出了一种针对图像多尺度边缘提 取的小波重构算法. 在此基础上, 提出了基于反对称双正交小波重构的图像锐化增强方法. 首先对图像进行多尺度小波分解; 然后在小波重构中, 计算模值图和相角图, 提取各尺度边缘图像, 并根据边缘图像, 增强半重构图像的对应边缘点; 最后继续逐级重构,实现图像增强. 该方法在小波塔式分解数据的重构过程中有针对性地实现对图像边缘的锐化增强, 对图像增强和图像滤噪增强提供了一种新的解决问题的思路. 实验结果验证了 该方法的有效性.     

Conics-based stereo,motion estimation,and pose determination

Stereo vision, motion and structure parameter estimation, and pose determination are three important problems in 3-D computer vision. The first step in all of these problems is to choose and to extract primitives and their features in images. In most of the previous work, people usually use edge points or straight line segments as primitives and their local properties as features. Few methods have been presented in the literature using more compact primitives and their global features. This article presents an approach using conics as primitives. For stereo vision, a closed-form solution is provided for both establishing the correspondence of conics in images and the reconstruction of conics in space. With this method, the correspondence is uniquely determined and the reconstruction is global. It is shown that the method can be extended for higher degree (degree3) planar curves.For motion and structure parameter estimation, it is shown that, in general, two sequential images of at least three conics are needed in order to determine the camera motion. A complicated nonlinear system must be solved in this case. In particular, if we are given two images of a pair of coplanar conics, a closed-form solution of camera motion is presented. In a CAD-based vision system, the object models are available, and this makes it possible to recognize 3-D objects and to determine their poses from a single image.For pose determination, it is shown that if there exist two conics on the surface of an object, the object's pose can be determined by an efficient one-dimensional search. In particular, if two conics are coplanar, a closed-form solution of the object's pose is presented.Uniqueness analysis and some experiments with real or synthesized data are presented in this article.     

基于形态学重建的粘连物体分割

提出一种基于形态学重建（Morphological Reconstruction）的图像分割方法。该方法先对待分割图像进行预处理,使边界点具有局部极大的灰度值;然后利用灰度形态学重建提取穹顶（Dome）,并根据其特性利用阚值对穹顶进行二值化获得候选边界点集;再利用二值形态学重建确定候选边界点集中的边界点,得到分割边界。实验结果表明,本分割方法所得边界连续性好、假边界少;该方法受噪声和对象内部灰度变化的影响较小,适合用于分割含有粘连对象的图像。     

多相主动轮廓模型的眼底图像杯盘分割

目的 视盘及视杯的检测对于分析眼底图像和视网膜视神经疾病计算机辅助诊断来说十分重要,利用医学眼底图像中视盘和视杯呈现椭圆形状这一特征,提出了椭圆约束下的多相主动轮廓模型,实现视盘视杯的同时精确分割。方法 该算法根据视盘视杯在灰度图像中具有不同的区域亮度,建立多相主动轮廓模型,然后将椭圆形约束内嵌于该模型中。通过对该模型的能量泛函进行求解,得到椭圆参数的演化方程。分割时首先设定两条椭圆形初始曲线,根据演化方程,驱动曲线分别向视盘和视杯方向进行移动。当轮廓线到达视盘、视杯边缘时,曲线停止演化。结果 在不同医学眼底图像中对算法进行验证,对算法抗噪性、不同初始曲线选取等进行了实验,并与多种算法进行了对比。实验结果表明,本文模型能够同时分割出视盘及视杯,与其他模型的分割结果相比,本文算法的分割结果更加准确。结论 本文算法可以精确分割医学眼底图像中的视盘和视杯,该算法不需要预处理,具有较强的鲁棒性和抗噪性。     

垂直建筑物红外图象的灰度值预测

文章提出了一种通过处理不同时间和天气情况下拍摄到的室外垂直墙面的长波红外图象来预测其灰度值的方法。讨论了长波红外(8～12μm)成象系统中图象灰度值与目标温度关系;研究了室外墙面与其背景间的能量交换,建立了墙面红外图象灰度值与时间、天气参数和地理信息之间的方程;通过已知拍摄条件的建筑物的长波红外图象数据拟合方程中的常量,文章最后给出了实验结果。     

Geometric techniques for 3D tracking of ultrasound sensor,tumor segmentation in ultrasound images,and 3D reconstruction

This paper presents different methods, some based on geometric algebra, for ultrasound probe tracking in endoscopic images, 3D allocation of the ultrasound probe, ultrasound image segmentation (to extract objects like tumors), and 3D reconstruction of the surface defined by a set of points. The tracking of the ultrasound probe in endoscopic images is done with a particle filter and an auxiliary method based on thresholding in the HSV space. The 3D pose of the ultrasound probe is calculated using conformal geometric algebra (to locate each slide in 3D space). Each slide (ultrasound image) is segmented using two methods: the level-set method and the morphological operators approach in order to obtain the object we are interested in. The points on the object of interest are obtained from the segmented ultrasound images, and then a 3D object is obtained by refining the convex hull. To do that, a peeling process with an adaptive radius is applied, all of this in the geometric algebra framework. Results for points from ultrasound images, as well as for points from objects from the AimatShape Project, are presented (A.I.M.A.T.S.H.A.P.E. – Advanced an Innovative Models And Tools for the development of Semantic-based systems for Handling, Acquiring, and Processing knowledge Embedded in multidimensional digital objects).     

Application of the finite-element method for the forward and inverse models in optical tomography

The development of an optical tomographic imaging system for biological tissue based on time-resolved near-infrared transillumination has received considerable interest recently. The reconstruction problem is ill posed because of scatter-dominated photon propagation, and hence it requires both an accurate and fast transport model and a robust solution convergence scheme. The iterative image recovery algorithm described in this paper uses a numerical finite-element solution to the diffusion equation as the photon propagation model. The model itself is used to compare the influence of absorbing and scattering inhomogeneities embedded in a homogeneous tissue sample on boundary measurements to estimate the possibility of separating absorption and scattering images. Images of absorbers and scatterers reconstructed from both mean-time-of-flight and logarithmic intensity data are presented. It is found that mean-time-of-flight data offer increased resolution for reconstructing the scattering coefficient, whereas intensity data are favorable for reconstructing absorption.     

Spline-Based Hybrid Image Registration using Landmark and Intensity Information based on Matrix-Valued Non-radial Basis Functions

We introduce a new approach for spline-based elastic image registration using both point landmarks and intensity information. With this approach, both types of information as well as a regularization based on the Navier equation are directly integrated in a single energy minimizing functional. For this functional we have derived an analytic solution, which is based on matrix-valued non-radial basis functions. With our approach the full 3D intensity information is exploited, i.e., all voxels are considered and subsampling using a grid is not required. A special case of our hybrid approach is obtained by disregarding the landmark information, which results in a pure intensity-based elastic registration approach. We have successfully applied our approach to 3D synthetic images, 2D MR images of the human brain, 2D gel electrophoresis images, and 3D CT lung images.     

Undersampled Light Field Rendering by a Plane Sweep

Images synthesized by light field rendering exhibit aliasing artifacts when the light field is undersampled; adding new light field samples improves the image quality and reduces aliasing but new samples are expensive to acquire. Light field rays are traditionally gathered directly from the source images, but new rays can also be inferred through geometry estimation. This paper describes a light field rendering approach based on this principle that estimates geometry from the set of source images using multi‐baseline stereo reconstruction to supplement the existing light field rays to meet the minimum sampling requirement. The rendering and reconstruction steps are computed over a set of planes in the scene volume, and output images are synthesized by compositing results from these planes together. The planes are each processed independently and the number of planes can be adjusted to scale the amount of computation to achieve the desired frame rate. The reconstruction fidelity (and by extension image quality) is improved by a library of matching templates to support matches along discontinuities in the image or geometry (e.g. object profiles and concavities). Given a set of silhouette images, the visual hull can be constructed and applied to further improve reconstruction by removing outlier matches. The algorithm is efficiently implemented by a set of image filter operations on commodity graphics hardware and achieves image synthesis at interactive rates.     

A Full Curvature Based Algorithm for Image Registration

Image registration, i.e., finding an optimal displacement field u which minimizes a distance functional D(u) is known to be an ill-posed problem. In this paper a novel variational image registration method is presented, which matches two images acquired from the same or from different medical imaging modalities. The approach proposed here is also independent of the image dimension. The proposed variational penalty against oscillations in the solutions is the standard H²(Ω) Sobolev semi-inner product for each component of the displacement. We investigate the associated Euler-Lagrange equation of the energy functional. Furthermore, we approach the solution of the underlying system of biharmonic differential equations with higher order boundary conditions as the steady-state solution of a parabolic partial differential equation (PDE). One of the important aspects of this approach is that the kernel of the Euler-Lagrange equation is spanned by all rigid motions. Hence, the presented approach includes a rigid alignment. Experimental results on both synthetic and real images are presented to illustrate the capabilities of the proposed approach. Stefan Henn obtained his diploma (1997) and his Ph.D. in mathematics (2001), both from the Heinrich-Heine University (HHU) of Düsseldorf (Germany). From 1997–1999 he had a researcher position at the Institute for Brain Research at the HHU Düsseldorf. Since 1999 he is a research assistant at the Institute of Mathematics at the HHU Düsseldorf. He received the SIAM outstanding paper prize in 2003 for the paper (Iterative Multigrid Regularization Techniques for Image Matching, SIAM Journal on Scientific Computing, 23(4), pp. 1077-1093). His research interests include Multiscale methods in Scientific Computing and Image Processing, nonlinear large-scale optimization, and numerical analysis of partial differential equations.     

Object-based image labeling through learning by example and multi-level segmentation

We propose a method for automatic extraction and labeling of semantically meaningful image objects using “learning by example” and threshold-free multi-level image segmentation. The proposed method scans through images, each of which is pre-segmented into a hierarchical uniformity tree, to seek and label objects that are similar to an example object presented by the user. By representing images with stacks of multi-level segmentation maps, objects can be extracted in the segmentation map level with adequate detail. Experiments have shown that the proposed multi-level image segmentation results in significant reduction in computation complexity for object extraction and labeling (compared to a single fine-level segmentation) by avoiding unnecessary tests of combinations in finer levels. The multi-level segmentation-based approach also achieves better accuracy in detection and labeling of small objects.