An automatic method of detecting and tracking fiducial markers for alignment in electron tomography

We presented an automatic method for detecting and tracking colloidal gold fiducial markers for alignment in electron tomography (ET). The second-order derivative of direction was used to detect a fiducial marker accurately. The detection was optimized to be selective to the size of fiducial markers. A preliminary tracking result from the normalized correlation coefficient was refined using the detector. A constraint model considering the relationship among the fiducial markers on different images was developed for removing outlier. The three-dimensional positions of the detected fiducial markers and the projection parameters of tilt images were calculated for post process. The accuracy of detection and tracking results was evaluated from the residues by the software IMOD. Application on transmission electron microscopic images also indicated that the presented method could provide a useful approach to automatic alignment in ET.     

Automatic acquisition of fiducial markers and alignment of images in tilt series for electron tomography.

Three-dimensional reconstruction of a section of biological tissue by electron tomography requires precise alignment of a series of two-dimensional images of the section made at numerous successive tilt angles. Gold beads on or in the section serve as fiducial markers. A scheme is described that automatically detects the position of these markers and indexes them from image to image. The resulting set of position vectors are arranged in a matrix representation of the tilt geometry and, by inversion, alignment information is obtained. The scheme is convenient, requires little operator time and provides an accuracy of < 2 pixels RMS. A tilt series of 60-70 images can be aligned in approximately 30 min on any modern desktop computer.     

Tomographic reconstruction using an adaptive tetrahedral mesh defined by a point cloud

Medical images in nuclear medicine are commonly represented in three dimensions as a stack of two-dimensional images that are reconstructed from tomographic projections. Although natural and straightforward, this may not be an optimal visual representation for performing various diagnostic tasks. A method for three-dimensional (3-D) tomographic reconstruction is developed using a point cloud image representation. A point cloud is a set of points (nodes) in space, where each node of the point cloud is characterized by its position and intensity. The density of the nodes determines the local resolution allowing for the modeling of different parts of the image with different resolution. The reconstructed volume, which in general could be of any resolution, size, shape, and topology, is represented by a set of nonoverlapping tetrahedra defined by the nodes. The intensity at any point within the volume is defined by linearly interpolating inside a tetrahedron from the values at the four nodes that define the tetrahedron. This approach creates a continuous piecewise linear intensity over the reconstruction domain. The reconstruction provides a distinct multiresolution representation, which is designed to accurately and efficiently represent the 3-D image. The method is applicable to the acquisition of any tomographic geometry, such as parallel-, fan-, and cone-beam; and the reconstruction procedure can also model the physics of the image detection process. An efficient method for evaluating the system projection matrix is presented. The system matrix is used in an iterative algorithm to reconstruct both the intensity and location of the distribution of points in the point cloud. Examples of the reconstruction of projection data generated by computer simulations and projection data experimentally acquired using a Jaszczak cardiac torso phantom are presented. This work creates a framework for voxel-less multiresolution representation of images in nuclear medicine.     

A reconstruction algorithm from truncated projections

A new tomographic reconstruction method is proposed which permits the reconstruction of a region of interest within a slice from partially truncated scanning data. This method utilizes two types of source data, namely a series of truncated projections and the outline of the object's cross section. The principle of this algorithm is to estimate the outside area of truncation in one projection from the projection data of the other viewing angles and the outline data of the object. The above estimation is accomplished by following two repeated procedures: 1) the modification of the calculated projection data compared each time with the already measured projection data of the truncated area, and 2) the modification of the reconstructed image compared also each time with the shape of the object. Computer simulation shows the convergence of the results obtained by this algorithm thus verifying its validity, and a reconstructed image after iterative processes exhibits good quality.     

A statistically harmonized alignment-classification in image space enables accurate and robust alignment of noisy images in single particle analysis

In determining the three-dimensional (3D) structure of macromolecular assemblies in single particle analysis, a large representative dataset of two-dimensional (2D) average images from huge number of raw images is a key for high resolution. Because alignments prior to averaging are computationally intensive, currently available multireference alignment (MRA) software does not survey every possible alignment. This leads to misaligned images, creating blurred averages and reducing the quality of the final 3D reconstruction. We present a new method, in which multireference alignment is harmonized with classification (multireference multiple alignment: MRMA). This method enables a statistical comparison of multiple alignment peaks, reflecting the similarities between each raw image and a set of reference images. Among the selected alignment candidates for each raw image, misaligned images are statistically excluded, based on the principle that aligned raw images of similar projections have a dense distribution around the correctly aligned coordinates in image space. This newly developed method was examined for accuracy and speed using model image sets with various signal-to-noise ratios, and with electron microscope images of the Transient Receptor Potential C3 and the sodium channel. In every data set, the newly developed method outperformed conventional methods in robustness against noise and in speed, creating 2D average images of higher quality. This statistically harmonized alignment-classification combination should greatly improve the quality of single particle analysis.     

Voxel-based 2-D/3-D registration of fluoroscopy images and CT scansfor image-guided surgery

Registration of intraoperative fluoroscopy images with preoperative 3D CT images can he used for several purposes in image-guided surgery. On the one hand, it can be used to display the position of surgical instruments, which are being tracked by a localizer, in the preoperative CT scan. On the other hand, the registration result can be used to project preoperative planning information or important anatomical structures visible in the CT image on to the fluoroscopy image. For this registration task, a novel voxel-based method in combination with a new similarity measure (pattern intensity) has been developed. The basic concept of the method is explained at the example of 2D/3D registration of a vertebra in an X-ray fluoroscopy image with a 3D CT image. The registration method is described, and the results for a spine phantom are presented and discussed. Registration has been carried out repeatedly with different starting estimates to study the capture range. Information about registration accuracy has been obtained by comparing the registration results with a highly accurate “ground-truth” registration, which has been derived from fiducial markers attached to the phantom prior to imaging. In addition, registration results for different vertebrae have been compared. The results show that the rotation parameters and the shifts parallel to the projection plane can accurately be determined from a single projection. Because of the projection geometry, the accuracy of the height above the projection plane is significantly lower     

基于卷积神经网络的大姿态人脸对齐方法

大姿态人脸对齐是人脸识别和三维人脸重构等很多重要视觉任务的先决条件.现有的对齐方法大多使用二维界标位置来进行对齐,且使用的界标数量有限,影响大姿态人脸对齐的准确性.提出一种采用三维形变模型(3DMM)来表示二维人脸图像,将具有任意姿态的人脸对齐问题建模为基于3DMM的拟合问题.采用基于卷积神经网络(CNN)的级联回归方...     

The distribution of target registration error in rigid-body point-based registration

Guidance systems designed for neurosurgery, hip surgery, spine surgery and for approaches to other anatomy that is relatively rigid can use rigid-body transformations to accomplish image registration. These systems often rely on point-based registration to determine the transformation and many such systems use attached fiducial markers to establish accurate fiducial points for the registration, the points being established by some fiducial localization process. Accuracy is important to these systems, as is knowledge of the level of that accuracy. An advantage of marker-based systems, particularly those in which the markers are bone-implanted, is that registration error depends only on the fiducial localization and is, thus, to a large extent independent of the particular object being registered. Thus, it should be possible to predict the clinical accuracy of marker-based systems on the basis of experimental measurements made with phantoms or previous patients. For most registration tasks, the most important error measure is target registration error (TRE), which is the distance after registration between corresponding points not used in calculating the registration transform. In this paper, we derive an approximation to the distribution of TRE; this is an extension of previous work that gave the expected squared value of TRE. We show the distribution of the squared magnitude of TRE and that of the component of TRE in an arbitrary direction. Using numerical simulations, we show that our theoretical results are a close match to the simulated ones.     

On the possibility of direct Fourier reconstruction from divergent-beam projections

A great number of tomographic systems, especially those equipped with fast data acquisition techniques, scan the objects investigated by divergent (fan) X-ray beams. Fan-beam projection data require special reconstruction techniques to be implemented. Among reconstruction techniques from parallel projection data, the direct Fourier method (DFM) proved to be one of the most promising ones, especially for high-speed image reconstruction in the high-end 3-D and dynamic tomographic systems. The goal of this work is to answer the topical question: how would direct use of the DFM influence the quality of image reconstruction from the fan-beam data? The formula describing the error caused by such an approximation is derived. The conclusions deduced from the formula are confirmed by computer simulations. The boundary values of data acquisition geometry parameters have been estimated for the case of using the DFM without recalculating the fan-beam data.     

Influence of the image quality deterioration of a tilted thick specimen on electron tomography

In electron tomography (ET) based on a transmission electron microscope, the effective thickness of a specimen increases with the tilt angle and, therefore, the projection quality may deteriorate because of electron scattering. The information-missing region, however, can be reduced by broadening the specimen tilt range. To clarify the general influence of these effects on ET, the projection quality varying with the tilt angle was quantitatively evaluated for a 5-microm thick specimen observed with a 3 MV ultrahigh-voltage electron microscope. Simulations of three-dimensional reconstruction were then performed for different specimen thicknesses and tilt ranges. As a result, the ET accuracy was shown to decrease as the specimen thickness increased. However, an optimum specimen-tilt range, at which ET could reach its highest accuracy, was found to exist and become small with the increase of the specimen thickness. The presented results are helpful for determining the specimen thickness limitation on the ET resolution and improving the ET fidelity of thick specimens.     

Registration and three-dimensional reconstruction of autoradiographic images by the disparity analysis method

Quantitative autoradiography is a powerful radioisotopic-imaging method for neuroscientists to study local cerebral blood flow and glucose-metabolic rate at rest, in response to physiologic activation of the visual, auditory, somatosensory, and motor systems, and in pathologic conditions. Most autoradiographic studies analyze glucose utilization and blood flow in two-dimensional (2D) coronal sections. With modern digital computer and image-processing techniques, a large number of closely spaced coronal sections can be stacked appropriately to form a three-dimensional (3D) image. 3D autoradiography allows investigators to observe cerebral sections and surfaces from any viewing angle. A fundamental problem in 3D reconstruction is the alignment (registration) of the coronal sections. A new alignment method based on disparity analysis is presented, which can overcome many of the difficulties encountered by previous methods. The disparity analysis method can deal with asymmetric, damaged, or tilted coronal sections under the same general framework, and it can be used to match coronal sections of different sizes and shapes. Experimental results on alignment and 3D reconstruction are presented.     

基于非局部双边随机投影低秩逼近图像去噪算法

该文提出一种基于非局部双边随机投影的低秩逼近图像去噪新方法。首先,对每个图像块通过非局部搜索寻找相似匹配块簇,然后对相似匹配块簇进行双边随机投影,用投影后的低秩结构恢复原图像。实验结果表明,所提方法比奇异值分解方法有较低的计算复杂度,比单边随机投影方法有较小的重构误差。特别是和3维块匹配方法相比,所提方法能保持相近的信噪比和较好的视觉质量。     

基于栅格投影的地基合成孔径雷达三维地形匹配俯仰角模糊处理方法

近距离对高大建筑物进行监测时,基于栅格投影的地基合成孔径雷达(ground-based synthetic aperture radar,GB-SAR)三维地形匹配过程中易出现俯仰角模糊现象,影响投影结果的解译.针对此问题,文中提出一种俯仰角模糊现象的处理方法.首先推导了任意GB-SAR观测几何条件下的图像距离-方位坐...     

Structure-from-motion without correspondence from tomographic projections by Bayesian inversion theory

In conventional tomography, the interior of an object is reconstructed from tomographic projections such as X-ray or transmission electron microscope images. All the current reconstruction methods assume that projection geometry of the imaging device is either known or solved in advance by using e.g., fiducial or nonfiducial feature points in the images. In this paper, we propose a novel approach where the imaging geometry is solved simultaneously with the volume reconstruction problem while no correspondence information is needed. Our approach is a direct application of Bayesian inversion theory and produces the maximum likelihood or maximum a posteriori estimates for the motion parameters under the selected noise and prior distributions. In this paper, the method is implemented for a two-dimensional model problem with one-dimensional affine projection data. The performance of the method is tested with simulated and measured X-ray projection data.     

图像代数重建算法的并行性研究

针对CT图像重建中不完备投影数据重建问题,提出了基于迭代的代数重建(ART)算法。为了解决ART算法重建速度慢这一问题,文中采用计算机集群的并行加速处理技术,将投影角度均匀地分配给适当的多个CPU,再将多个CPU并行独立运行的重建结果进行加权平均。实验数据表明,图像重建速度提高近6倍(CPU个数为6),图像重建质量优于传统的串行重建质量,且提高了ART算法的实现效率。     

分步投影光刻机对准系统应用与研究

分步投影光刻设备对准系统是由嵌在工作台上的一组基准标记、一套离轴对准系统和一套掩模对准系统组成。当工作台上的基准标记运行到投影镜头下面时,通过掩模对准系统在预定范围内扫描测量标记位置值。当工作台上的基准标记或硅片上的标记运行到离轴对准系统测量光束下面时,用系统扫描可以测量出标记的坐标值。即可通过测量值计算出每个曝光场的中心坐标值。通过EGA对准数据,可计算出硅片的平移偏移量、旋转量、比例量和正交性量的补偿值。     

Exact image reconstruction from a limited number of projections

A new method for the exact reconstruction of any gray-scale image from its projections is proposed. The original image is projected into several view angles and the projection samples are stored in an accumulator array. In order to reconstruct the image, the accumulator array is considered as an accumulation of sinusoidal contributions each one corresponding to a certain pixel of the original image. The proposed method defines conditions for the necessary number of projections and the density of ray samples on the projection axis. These conditions insure that, for each pixel, there is at least one sample in the accumulator array where only this particular pixel contributes. This characteristic projection sample is used during the reconstruction phase to determine the coordinates and the gray-scale value of the corresponding image pixel. A variation of the method is also proposed where the reconstruction is performed using a limited number of projection samples in certain view angles. Specifically, the number of necessary samples equals at most the overall number of pixels in the original image. This approach leads to a significant reduction of memory and processing time requirements since it provides exact image reconstruction using one projection sample per pixel.     

Characterization of pinhole SPECT acquisition geometry

A method is presented to estimate the acquisition geometry of a pinhole single photon emission computed tomography (SPECT) camera with a circular detector orbit. This information is needed for the reconstruction of tomographic images. The calibration uses the point source projection locations of a tomographic acquisition of three point sources located at known distances from each other. It is shown that this simple phantom provides the necessary and sufficient information for the proposed calibration method. The knowledge of two of the distances between the point sources proves to be essential. The geometry is estimated by fitting analytically calculated projections to the measured ones, using a simple least squares Powell algorithm. Some mild a priori knowledge is used to constrain the solutions of the fit. Several of the geometrical parameters are however highly correlated. The effect of these correlations on the reconstructed images is evaluated in simulation studies and related to the estimation accuracy. The highly correlated detector tilt and electrical shift are shown to be the critical parameters for accurate image reconstruction. The performance of the algorithm is finally demonstrated by phantom measurements. The method is based on a single SPECT scan of a simple calibration phantom, executed immediately after the actual SPECT acquisition. The method is also applicable to cone-beam SPECT and X-ray CT.     

最小特征值的迭代非刚体三维射影重建方法

为了从图像序列中重建出非刚体三维射影重建,本文提出了一种最小特征值的迭代非刚体射影重建方法.该方法利用所有的图像点和深度因子组成一个低秩图像矩阵的特性,将投影求解转化为矩阵特征值及特征向量的求解,迭代地求解深度因子,实现非刚体的三维射影重建.该方法能够保证算法能够收敛到全局最优解.模拟实验和真实实验结果表明,本文方法具有收敛性速度快、误差小等优点.     

Parameter distribution models for estimation of population based left ventricular deformation using sparse fiducial markers

We present a method to estimate left ventricular (LV) motion based on three-dimensional (3-D) images that can be derived from any anatomical tomographic or 3-D modality, such as echocardiography, computed tomography, or magnetic resonance imaging. A finite element mesh of the LV was constructed to fit the geometry of the wall. The mesh was deformed by optimizing the nodal parameters to the motion of a sparse number of fiducial markers that were manually tracked in the images through the cardiac cycle. A parameter distribution model (PDM) of LV deformations was obtained from a database of MR tagging studies. This was used to filter the calculated deformation and incorporate a priori information on likely motions. The estimated deformation obtained from 13 normal untagged studies was compared with the deformation obtained from MR tagging. The end systolic (ES) circumferential and longitudinal strain values matched well with a mean difference of 0.1 +/- 3.2% and 0.3 +/- 3.0%, respectively. The calculated apex-base twist angle at ES had a mean difference of 1.0 +/- 2.3 degrees. We conclude that fiducial marker fitting in conjunction with a PDM provides accurate reconstruction of LV deformation in normal subjects.