Automated 3-D reconstruction of the surface of live early-stage amphibian embryos

Although three-dimensional (3-D) reconstructions of the surfaces of live embyos are vital to understanding embryo development, morphogenetic tissue movements and other factors have prevented the automation of this task. Here, we report an integrated set of software algorithms that overcome these challenges, making it possible to completely automate the reconstruction of embryo surfaces and other textured surfaces from multiview images. The process involves: 1) building accurate point correspondences using a robust deformable template block matching algorithm; 2) removing outliers using fundamental matrix calculations in conjunction with a RANSAC algorithm; 3) generating 3-D point clouds using a bundle adjustment algorithm that includes camera position and distortion corrections; 4) meshing the point clouds into triangulated surfaces using a Tight Cocone algorithm that produces water tight models; 5) refining surfaces using midpoint insertion and Laplacian smoothing algorithms; and 6) repeating these steps until a measure of convergence G, the rms difference between successive reconstructions, is below a specified threshold. Reconstructions were made of 2.2-mm diameter, neurulation-stage axolotl (amphibian) embryos using 44 multiview images collected with a robotic microscope. A typical final model (sixth iteration) contained 3787 points and 7562 triangles and had an error measure of G = 5.9 microm.     

Registration and tracking to integrate X-ray and MR images in an XMR facility

We describe a registration and tracking technique to integrate cardiac X-ray images and cardiac magnetic resonance (MR) images acquired from a combined X-ray and MR interventional suite (XMR). Optical tracking is used to determine the transformation matrices relating MR image coordinates and X-ray image coordinates. Calibration of X-ray projection geometry and tracking of the X-ray C-arm and table enable three-dimensional (3-D) reconstruction of vessel centerlines and catheters from bi-plane X-ray views. We can, therefore, combine single X-ray projection images with registered projection MR images from a volume acquisition, and we can also display 3-D reconstructions of catheters within a 3-D or multi-slice MR volume. Registration errors were assessed using phantom experiments. Errors in the combined projection images (two-dimensional target registration error--TRE) were found to be 2.4 to 4.2 mm, and the errors in the integrated volume representation (3-D TRE) were found to be 4.6 to 5.1 mm. These errors are clinically acceptable for alignment of images of the great vessels and the chambers of the heart. Results are shown for two patients. The first involves overlay of a catheter used for invasive pressure measurements on an MR volume that provides anatomical context. The second involves overlay of invasive electrode catheters (including a basket catheter) on a tagged MR volume in order to relate electrophysiology to myocardial motion in a patient with an arrhythmia. Visual assessment of these results suggests the errors were of a similar magnitude to those obtained in the phantom measurements.     

Accelerated iterative transmission CT reconstruction using an ordered subsets convex algorithm

Iterative maximum likelihood (ML) transmission computed tomography algorithms have distinct advantages over Fourier-based reconstruction, but unfortunately require increased computation time. The convex algorithm [1] is a relatively fast iterative ML algorithm but it is nevertheless too slow for many applications. Therefore, an acceleration of this algorithm by using ordered subsets of projections is proposed [ordered subsets convex algorithm (OSC)]. OSC applies the convex algorithm sequentially to subsets of projections. OSC was compared with the convex algorithm using simulated and physical thorax phantom data. Reconstructions were performed for OSC using eight and 16 subsets (eight and four projections/subset, respectively). Global errors, image noise, contrast recovery, and likelihood increase were calculated. Results show that OSC is faster than the convex algorithm, the amount of acceleration being approximately proportional to the number of subsets in OSC, and it causes only a slight increase of noise and global errors in the reconstructions. Images and image profiles of the reconstructions were in good agreement. In conclusion, OSC and the convex algorithm result in similar image quality but OSC is more than an order of magnitude faster.     

3-D reconstruction of microcalcification clusters using stereo imaging: algorithm and mammographic unit calibration

The three-dimensional (3-D) shape of microcalcification clusters is an important indicator in early breast cancer detection. In fact, there is a relationship between the cluster topology and the type of lesion (malignant or benign). This paper presents a 3-D reconstruction method for such clusters using two 2-D views acquired during standard mammographic examinations. For this purpose, the mammographic unit was modeled using a camera with virtual optics. This model was used to calibrate the acquisition unit and then to reconstruct the clusters in the 3-D space after microcalcification segmentation and matching. The proposed model is hardware independent since it is suitable for digital mammographic units with different geometries and with various physical acquisition principles. Three-dimensional reconstruction results are presented here to prove the validity of the method. Tests were first performed using a phantom with a well-known geometry. The latter contained X-ray opaque glass balls representing microcalcifications. The positions of these balls were reconstructed with a 16.25-microm mean accuracy. This very high inherent algorithm accuracy is more than enough for a precise 3-D cluster representation. Further validation tests were carried out using a second phantom including a spherical cluster. This phantom was built with materials simulating the behavior of both mammary tissue and microcalcifications toward Xrays. The reconstructed shape was effectively spherical. Finally, reconstructions were carried out for real clusters and their results are also presented.     

A generalized EM algorithm for 3-D Bayesian reconstruction from Poisson data using Gibbs priors

A generalized expectation-maximization (GEM) algorithm is developed for Bayesian reconstruction, based on locally correlated Markov random-field priors in the form of Gibbs functions and on the Poisson data model. For the M-step of the algorithm, a form of coordinate gradient ascent is derived. The algorithm reduces to the EM maximum-likelihood algorithm as the Markov random-field prior tends towards a uniform distribution. Three different Gibbs function priors are examined. Reconstructions of 3-D images obtained from the Poisson model of single-photon-emission computed tomography are presented.     

基于三维集成成像相机阵列获取的元素图像校正

利用相机阵列获取三维信息实现三维集成成像与显示时,为消除相机阵列空间位置偏差对元素图像阵列的影响,提高再现三维图像的质量,以相机阵列记录系统为基础提出了一种元素图像阵列校正方法。通过特征点位置坐标以及相机位置平移误差和旋转误差的计算,分析了相机阵列位置平移误差和旋转误差与元素图像间的关系,以及校正算法的精度。利用光学实验对该算法进行了验证,结果表明,此方法可有效消除相机阵列位置偏差对元素图像阵列的影响,并且校正后再现三维图像质量明显优于误差图像,峰值信噪比提高了33.6%,实现了基于三维集成成像相机阵列获取的元素图像校正,满足了集成成像的显示要求。     

Exact reduced-complexity maximum likelihood reconstruction of multiple 3-D objects from unlabeled unoriented 2-D projections and electron microscopy of viruses.

In cryo-electron microscopy, the data is comprised of noisy 2-D projection images of the 3-D electron scattering intensity of the object where the orientation of the projections is unknown. Often, the images show randomly selected objects from a mixture of different types of objects. Objects of different type may be unrelated, e.g., different species of virus, or related, e.g., different conformations of the same species of virus. Due to the low SNR and the 2-D nature of the data, it is challenging to determine the type of the object shown in an individual image. A statistical model and maximum likelihood estimator that computes simultaneous 3-D reconstruction and labels using an expectation maximization algorithm exists but requires extensive computation due to the numerical evaluation of 3-D or 5-D integrations of a square matrix of dimension equal to the number of degrees of freedom in the 3-D reconstruction. By exploiting the geometry of rotations in 3-D, the estimation problem can be transformed so that the inner-most numerical integral has a scalar rather than a matrix integrand. This leads to a dramatic reduction in computation, especially as the number of degrees of freedom in the 3-D reconstruction increases. Numerical examples of the 3-D reconstructions are provided based on synthetic and experimental images where the objects are small spherical viruses.     

Composition of a Dewarped and Enhanced Document Image From Two View Images

In this paper, we propose an algorithm to compose a geometrically dewarped and visually enhanced image from two document images taken by a digital camera at different angles. Unlike the conventional works that require special equipments or assumptions on the contents of books or complicated image acquisition steps, we estimate the unfolded book or document surface from the corresponding points between two images. For this purpose, the surface and camera matrices are estimated using structure reconstruction, 3-D projection analysis, and random sample consensus-based curve fitting with the cylindrical surface model. Because we do not need any assumption on the contents of books, the proposed method can be applied not only to optical character recognition (OCR), but also to the high-quality digitization of pictures in documents. In addition to the dewarping for a structurally better image, image mosaic is also performed for further improving the visual quality. By finding better parts of images (with less out of focus blur and/or without specular reflections) from either of views, we compose a better image by stitching and blending them. These processes are formulated as energy minimization problems that can be solved using a graph cut method. Experiments on many kinds of book or document images show that the proposed algorithm robustly works and yields visually pleasing results. Also, the OCR rate of the resulting image is comparable to that of document images from a flatbed scanner.     

Optimal CT scanning plan for long-bone 3-D reconstruction

Digital computed tomographic (CT) data are widely used in three-dimensional (3-D) construction of bone geometry and density features for 3-D modelling purposes. During in vivo CT data acquisition the number of scans must be limited in order to protect patients from the risks related to X-ray absorption. The aim of this work is to automatically define, given a finite number of CT slices, the scanning plan which returns the optimal 3-D reconstruction of a bone segment from in vivo acquired CT images. An optimization algorithm based on a Discard-Insert-Exchange technique has been developed. In the proposed method the optimal scanning sequence is searched by minimizing the overall reconstruction error of a two-dimensional (2-D) prescanning image: an anterior-posterior (AP) X-ray projection of the bone segment. This approach has been validated in vitro on 3 different femurs. The 3-D reconstruction errors obtained through the optimization of the scanning plan on the 3-D prescanning images and on the corresponding 3-D data sets have been compared. 2-D and 3-D data sets have been reconstructed by linear interpolation along the longitudinal axis. Results show that direct 3-D optimization yields root mean square reconstruction errors which are only 4%-7% lower than the 2-D-optimized plan, thus proving that 2-D-optimization provides a good suboptimal scanning plan for 3-D reconstruction. Further on, 3-D reconstruction errors given by the optimized scanning plan and a standard radiological protocol for long bones have been compared. Results show that the optimized plan yields 20%-50% lower 3-D reconstruction errors     

嫦娥一号卫星CCD立体相机影像超分辨率重建算法

针对嫦娥一号卫星CCD立体相机空间分辨率不足的问题,运用最大后验概率估计法(MAP)实现了月表影像的超分辨率重建。介绍了嫦娥一号卫星CCD立体相机的成像模型,分析了图像获取过程中的主要影响因素,并建立了相应的超分辨率重建模型。基于该模型,首先采用误差-参数分析法估计嫦娥一号卫星CCD立体相机动态成像光学系统的点扩散函数(PSF);然后将估计的PSF应用到MAP算法所建立的目标函数中,采用共轭梯度法对目标函数进行最值求解;再通过VC软件平台编程实现了对单帧正视月表影像的超分辨率重建;最后从信息熵、清晰度和频谱等方面对重建图像进行评价,结果表明重建图像像质优良。     

Improved image quality and computation reduction in 4-D reconstruction of cardiac-gated SPECT images

Spatiotemporal reconstruction of cardiac-gated SPECT images permits us to obtain valuable information related to cardiac function. However, the task of reconstructing this four-dimensional (4-D) data set is computation intensive. Typically, these studies are reconstructed frame-by-frame: a nonoptimal approach because temporal correlations in the signal are not accounted for. In this work, we show that the compression and signal decorrelation properties of the Karhunen-Loève (KL) transform may be used to greatly simplify the spatiotemporal reconstruction problem. The gated projections are first KL transformed in the temporal direction. This results in a sequence of KL-transformed projection images for which the signal components are uncorrelated along the time axis. As a result, the 4-D reconstruction task is simplified to a series of three-dimensional (3-D) reconstructions in the KL domain. The reconstructed KL components are subsequently inverse KL transformed to obtain the entire spatiotemporal reconstruction set. Our simulation and clinical results indicate that KL processing provides image sequences that are less noisy than are conventional frame-by-frame reconstructions. Additionally, by discarding high-order KL components that are dominated by noise, we can achieve savings in computation time because fewer reconstructions are needed in comparison to conventional frame-by-frame reconstructions.     

An evaluation of maximum likelihood reconstruction for SPECT

A reconstruction method for SPECT (single photon emission computerized tomography) that uses the maximum likelihood (ML) criterion and an iterative expectation-maximization (EM) algorithm solution is examined. The method is based on a model that incorporates the physical effects of photon statistics, nonuniform photon attenuation, and a camera-dependent point-spread response function. Reconstructions from simulation experiments are presented which illustrate the ability of the ML algorithm to correct for attenuation and point-spread. Standard filtered backprojection method reconstructions, using experimental and simulated data, are included for reference. Three studies were designed to focus on the effects of noise and point-spread, on the effect of nonuniform attenuation, and on the combined effects of all three. The last study uses a chest phantom and simulates Tl-201 imaging of the myocardium. A quantitative analysis of the reconstructed images is used to support the conclusion that the ML algorithm produces reconstructions that exhibit improved signal-to-noise ratios, improved image resolution, and image quantifiability.     

Improved tomographic reconstruction in seven-pinhole imaging

Cardiac emission tomography using a seven-pinhole collimator has received only little appreciation as a diagnostic imaging technique. The main reasons are the limited angular sampling of the seven-pinhole device and the difficulties encountered in properly positioning the patient relative to the collimator/camera system. In order to overcome these problems, we have developed a modified ART3 algorithm for reconstruction of the radioactivity distribution in the heart. The method is very appropriate for seven-pinhole tomography, as demonstrated by the quality of the reconstructions, by the excellent point source resolution of the system response, and by a comparison to two other suitable reconstruction techniques, viz., SMART and SIRT.     

A survey of motion-parallax-based 3-D reconstruction algorithms

The task of recovering three-dimensional (3-D) geometry from two-dimensional views of a scene is called 3-D reconstruction. It is an extremely active research area in computer vision. There is a large body of 3-D reconstruction algorithms available in the literature. These algorithms are often designed to provide different tradeoffs between speed, accuracy, and practicality. In addition, even the output of various algorithms can be quite different. For example, some algorithms only produce a sparse 3-D reconstruction while others are able to output a dense reconstruction. The selection of the appropriate 3-D reconstruction algorithm relies heavily on the intended application as well as the available resources. The goal of this paper is to review some of the commonly used motion-parallax-based 3-D reconstruction techniques and make clear the assumptions under which they are designed. To do so efficiently, we classify the reviewed reconstruction algorithms into two large categories depending on whether a prior calibration of the camera is required. Under each category, related algorithms are further grouped according to the common properties they share.     

流体中微粒污染物立体尺寸测量方法

利用“近景痒体摄影测量”方法,对流体中污染颗粒的立体形状及立体尺寸进行测量。通过两套摄影系统,从不同角度摄取颗粒的立体像对,并将像对转换成数字图像对,然后利用计算机软件程序对数字图像对进行处理,以获得颗粒的立体信息,证明这种方法的可行性。     

3-D maximum a posteriori estimation for single photon emission computed tomography on massively-parallel computers

A fully three-dimensional (3-D) implementation of the maximum a posteriori (MAP) method for single photon emission computed tomography (SPECT) is demonstrated. The 3-D reconstruction exhibits a major increase in resolution when compared to the generation of the series of separate 2-D slice reconstructions. As has been noted, the iterative EM algorithm for 2-D reconstruction is highly computational; the 3-D algorithm is far worse. To accommodate the computational complexity, previous work in the 2-D arena is extended, and an implementation on the class of massively parallel processors of the 3-D algorithm is demonstrated. Using a 16000- (4000-) processor MasPar/DECmpp-Sx machine, the algorithm is demonstrated to execute at 2.5 (7.8) s/EM-iteration for the entire 64x64x64 cube of 96 planar measurements obtained from the Siemens Orbiter rotating camera operating in the high-resolution mode.     

单幅高分辨率SAR图像建筑物三维模型重构

提出了一种利用高分辨率SAR图像进行建筑物提取和三维重构的方法.首先,分析了高分辨率SAR图像建筑物产生的电磁散射的类型,给出了不同类型散射区域的后向散射计算方法,并在此基础上给出了一种利用建筑物三维CAD模型进行SAR建筑物特征区域图像仿真的方法;其次,给出了利用建筑物的二次散射结构确定建筑物底部轮廓位置和方向的方法,并提出了一种基于分布密度函数差异的仿真图像迭代匹配方法,进行建筑物高度的反演.仿真SAR图像后向散射系数用来划分建筑物不同的散射区域,通过计算特征区域之间的分布密度函数差异,以取得最大匹配度值的仿真图像对应的检验高度作为建筑物的反演高度;最后,选用了两幅不同屋顶类型的实际机载高分辨率SAR图像进行建筑物提取和三维重构实验,试验结果较为理想,验证了所提方法的可行性和有效性.     

Improved SPECT quantitation using fully three-dimensional iterative spatially variant scatter response compensation

The quality and quantitative accuracy of iteratively reconstructed SPECT images improves when better point spread function (PSF) models of the gamma camera are used during reconstruction. Here, inclusion in the PSF model of photon crosstalk between different slices caused by limited gamma camera resolution and scatter is examined. A three-dimensional (3-D) projector back-projector (proback) has been developed which models both the distance dependent detector point spread function and the object shape-dependent scatter point spread function of single photon emission computed tomography (SPECT). A table occupying only a few megabytes of memory is sufficient to represent this scatter model. The contents of this table are obtained by evaluating an analytical expression for object shape-dependent scatter. The proposed approach avoids the huge memory requirements of storing the full transition matrix needed for 3-D reconstruction including object shape-dependent scatter. In addition, the method avoids the need for lengthy Monte Carlo simulations to generate such a matrix. In order to assess the quantitative accuracy of the method, reconstructions of a water filled cylinder containing regions of different activity levels and of simulated 3-D brain projection data have been evaluated for technetium-99m. It is shown that fully 3-D reconstruction including complete detector response and object shape-dependent scatter modeling clearly outperforms simpler methods that lack a complete detector response and/or a complete scatter response model. Fully 3-D scatter correction yields the best quantitation of volumes of interest and the best contrast-to-noise curves.     

Reconstruction of sculpture from its profiles with unknown camera positions

Profiles of a sculpture provide rich information about its geometry, and can be used for shape recovery under known camera motion. By exploiting correspondences induced by epipolar tangents on the profiles, a successful solution to motion estimation from profiles has been developed in the special case of circular motion. The main drawbacks of using circular motion alone, namely the difficulty in adding new views and part of the object always being invisible, can be overcome by incorporating arbitrary general views of the object and registering its new profiles with the set of profiles resulted from the circular motion. In this paper, we describe a complete and practical system for producing a three-dimensional (3-D) model from uncalibrated images of an arbitrary object using its profiles alone. Experimental results on various objects are presented, demonstrating the quality of the reconstructions using the estimated motion.     

Camera calibration using symmetric objects.

This paper proposes a novel method for camera calibration using images of a mirror symmetric object. Assuming unit aspect ratio and zero skew, we show that interimage homographies can be expressed as a function of only the principal point. By minimizing symmetric transfer errors, we thus obtain an accurate solution for the camera parameters. We also extend our approach to a calibration technique using images of a 1-D object with a fixed pivoting point. Unlike existing methods that rely on orthogonality or pole-polar relationship, our approach utilizes new inter-image constraints and does not require knowledge of the 3-D coordinates of feature points. To demonstrate the effectiveness of the approach, we present results for both synthetic and real images.