2D/3D image registration using regression learning

In computer vision and image analysis, image registration between 2D projections and a 3D image that achieves high accuracy and near real-time computation is challenging. In this paper, we propose a novel method that can rapidly detect an object’s 3D rigid motion or deformation from a 2D projection image or a small set thereof. The method is called CLARET (Correction via Limited-Angle Residues in External Beam Therapy) and consists of two stages: registration preceded by shape space and regression learning. In the registration stage, linear operators are used to iteratively estimate the motion/deformation parameters based on the current intensity residue between the target projection(s) and the digitally reconstructed radiograph(s) (DRRs) of the estimated 3D image. The method determines the linear operators via a two-step learning process. First, it builds a low-order parametric model of the image region’s motion/deformation shape space from its prior 3D images. Second, using learning-time samples produced from the 3D images, it formulates the relationships between the model parameters and the co-varying 2D projection intensity residues by multi-scale linear regressions. The calculated multi-scale regression matrices yield the coarse-to-fine linear operators used in estimating the model parameters from the 2D projection intensity residues in the registration. The method’s application to Image-guided Radiation Therapy (IGRT) requires only a few seconds and yields good results in localizing a tumor under rigid motion in the head and neck and under respiratory deformation in the lung, using one treatment-time imaging 2D projection or a small set thereof.     

虚拟地球下的小比例尺制图研究

目前地图上的所有投影方法(无论是墨卡托投影、高斯—克吕格投影还是伪投影)都存在着投影变形(包括角度、面积、长度)，这是由于无法平面投影，无法将椭球面没有裂隙和褶叠地表示在平面上。以虚拟地球为研究基点，直接进行从地球球面要素到计算机虚拟球面要素的映射，进而实现小比例尺制图，以纠正平面制图中存在的变形问题。     

基于投影近似不变性的3D-2D医学图像配准

研究了同一病人的MRA和DSA图像之间的3D-2D配准问题，提出了一种基于投影近似不变性的配准算法，即在同一视角下，同一病人的三维MRA血管骨架的投影与二维DSA的骨架保持一致。通过定义二者之间的代价函数，并利用牛顿迭代法可得到视角参数的最优解。针对临床采集的MRA和DSA数据进行了实验，取得了较为满意的结果。     

基于正交投影的三维模型相似性比较算法研究*

提出了一种基于二维正交投影图像的三维模型相似性比较算法。首先计算三维模型的二维正交投影图像,然后提取二维正交投影图像的边缘轮廓夹角特征,并比较这些二维正交投影图像的相似性,最后通过二维正交投影图像的轮廓相似性来获取三维模型的相似性。实验验证了该算法的可行性,并且其对于三维模型检索所要求的平移、旋转和缩放不变性具有较好的鲁棒性。     

3D face reconstruction using images from cameras with varying parameters

In this paper, we present a new technique of 3D face reconstruction from a sequence of images taken with cameras having varying parameters without the need to grid. This method is based on the estimation of the projection matrices of the cameras from a symmetry property which characterizes the face, these projections matrices are used with points matching in each pair of images to determine the 3D points cloud, subsequently, 3D mesh of the face is constructed with 3D Crust algorithm. Lastly, the 2D image is projected on the 3D model to generate the texture mapping. The strong point of the proposed approach is to minimize the constraints of the calibration system: we calibrated the cameras from a symmetry property which characterizes the face, this property gives us the opportunity to know some points of 3D face in a specific well-chosen global reference, to formulate a system of linear and nonlinear equations according to these 3D points, their projection in the image plan and the elements of the projections matrix. Then to solve these equations, we use a genetic algorithm which consists of finding the global optimum without the need of the initial estimation and allows to avoid the local minima of the formulated cost function. Our study is conducted on real data to demonstrate the validity and the performance of the proposed approach in terms of robustness, simplicity, stability and convergence.     

基于空间仿射对应点列的双曲抛物面三维构建及分析

利用计算机3D 软件构建基于空间仿射对应点列的双曲抛物面。对双曲抛 物面进行不同方向的投射,可以准确得到双曲抛物面不同方位和角度的投影图。从这些投影 图中,可以更直观、清楚的了解双曲抛物面的投影特点。使用多个不同位置的平面对双曲抛 物面进行截切,并通过分析、观察截交线的变化趋势,得出几点规律性的结论。     

3D object recognition using invariants of 2D projection curves

This paper presents a new method for recognizing 3D objects based on the comparison of invariants of their 2D projection curves. We show that Euclidean equivalent 3D surfaces imply affine equivalent 2D projection curves that are obtained from the projection of cross-section curves of the surfaces onto the coordinate planes. Planes used to extract cross-section curves are chosen to be orthogonal to the principal axes of the defining surfaces. Projection curves are represented using implicit polynomial equations. Affine algebraic and geometric invariants of projection curves are constructed and compared under a variety of distance measures. Results are verified by several experiments with objects from different classes and within the same class.     

Recovering the position and orientation of free-form objects fromimage contours using 3D distance maps

The accurate matching of 3D anatomical surfaces with sensory data such as 2D X-ray projections is a basic problem in computer and robot assisted surgery, In model-based vision, this problem can be formulated as the estimation of the spatial pose (position and orientation) of a 3D smooth object from 2D video images. The authors present a new method for determining the rigid body transformation that describes this match. The authors' method performs a least squares minimization of the energy necessary to bring the set of the camera-contour projection lines tangent to the surface. To correctly deal with projection lines that penetrate the surface, the authors consider the minimum signed distance to the surface along each line (i.e., distances inside the object are negative). To quickly and accurately compute distances to the surface, the authors introduce a precomputed distance map represented using an octree spline whose resolution increases near the surface. This octree structure allows the authors to quickly find the minimum distance along each line using best-first search. Experimental results for 3D surface to 2D projection matching are presented for both simulated and real data. The combination of the authors' problem formulation in 3D, their computation of line to surface distances with the octree-spline distance map, and their simple minimization technique based on the Levenberg-Marquardt algorithm results in a method that solves the 3D/2D matching problem for arbitrary smooth shapes accurately and quickly     

基于极限投影变换的3D模型检索

基于认知理论和投影理论,提出极限投影面积方法对三维模型进行坐标标准化:将模型不断绕过模型重心的基向量旋转,采集每次坐标面投影面积,在多次迭代后得到最大面积投影,用同样办法获得另一坐标面上的最小面积投影,建立模型的坐标系.实验结果表明:该方法标准化的坐标系处理对象范围宽,适用于网格模型、点云模型和各类曲面模型,针对模型噪声、简化、攻击等有很强的鲁棒性.经该方法标准化的三维模型在3个坐标面的投影作为模型的特征描述子,并在实验中为实验模型库建立了对应的特征描述子库,使得对模型检索转化为特征描述子间的比较检索.检索实验表明:该方法针对增加了特征描述子的模型库具有较快的检索速度;同时具有较强的鲁棒性,但检索的精度稍差.     

多特征动态融合的三维模型检索方法

提出一种基于二维正交投影图像的多特征动态融合的三维模型检索方法.首先计算三维模型的二维正交投影图像,然后提取二维正交投影图像的投影直方图和Zernike矩特征,通过加权求和在输出层融合,得到总体上模型间的相似度.每种特征采用动态权值,针对输入查询模型的不同,根据用户反馈自动更新知识库中的权值.实验表明,该方法在提高检索准确率的同时,也能保证检索效率.     

基于二维视图特征的三维重建

在分析现有ＣＡＤ二维参数化设计,三维参数化设计以及三维重建方法的基础上,基于工程图图形的整体宏观性,图形拓扑性及三维视图的投影规律,提出了二维视图特征的概念及一种新的工程图处理方法。     

基于机器学习和几何变换的实时2D/3D脊椎配准

在图像引导的脊柱手术中,实时高效的2D/3D配准是一项重要且具有挑战性的任务.通常的2D/3D配准一般是将三维图像投影到二维平面,然后进行2D/2D的配准.由于投影空间涉及到3个平移以及3个旋转参数,其投影空间的复杂度为O（n6）,使得配准很难兼具高准确性和高实时性.本文提出了一个结合机器学习与几何变换的2D/3D配准方法,首先,使用统计形状模型对目标脊椎进行建模,并构建了一种新的投影方式,使得6个投影参数中的4个可以使用几何的方法计算出来;接下来利用回归学习的方法学习目标脊椎的形状与投影参数之间的关系;最终,结合学到的关系和几何变换完成配准.本方法的两个姿态参数的平均预测误差为0.84°和0.81°,平均目标配准误差（Mean target registration error,mTRE）为0.87mm,平均配准时间为0.9s.实验结果表明本方法具有很好的实时性和准确性.     

正多边形剖分球面投影及其应用

提出了利用正多面体顶点分布来剖分球面,得到球面上物体的投影点,利用这些投影点得到物体的二维投影。这种二维投影的获取方式具有投影点分布均匀的特点,并且根据三维物体的特点对特定区域的投影点进行加密,以获取更多的二维投影信息。该方法已经应用到三维模型及三维动作的识别中,取得了良好的效果。     

Wireframe Projections: Physical Realisability of Curved Objects and Unambiguous Reconstruction of Simple Polyhedra

The reconstruction of an object from a single 2D projection of a 3D wireframe model is a vision problem with applications in CAD/CAM and computer graphics.We propose an algorithm for the interpretation of wireframe projections based on assigning semantic and numerical depth labels to lines. This method allows us to state necessary and sufficient conditions for the physical realisability of a wireframe projection of a curved object. The presence of linear features provides further constraints on the positions of object vertices. For example, each straight line gives rise to a coplanarity constraint between a set of object vertices.We show that extra information, such as vanishing points, parallel lines or user-entered depth-parity information, is sufficient to uniquely determine the face-circuits in wireframe projections of polyhedra with simple trihedral vertices. In fact, a polyhedron with simple trihedral vertices can be unambiguously reconstructed from its 3D wireframe model.     

结构光视觉三维点云逐层三角网格化算法

针对结构光视觉恢复的大规模三维点云的可投影特点,提出一种基于投影网格的底边驱动逐层网格化曲面重建算法。该算法首先将点云投影到一个二维平面上;然后基于点云投影区域建立规则投影网格,并将投影点映射到规则二维投影网格上,建立二维网格点与三维点云间的映射关系;接着对投影网格进行底边驱动的逐层网格化,建立二维三角网格;最后根据二维投影点与三维点的对应关系及二维三角网格拓扑关系获得最终的三维网格曲面。实验结果表明,算法曲面重建速度快,可较好地保持曲面细节特征。     

Calibration of Central Catadioptric Cameras Using a DLT-Like Approach

In this study, we present a calibration technique that is valid for all single-viewpoint catadioptric cameras. We are able to represent the projection of 3D points on a catadioptric image linearly with a 6×10 projection matrix, which uses lifted coordinates for image and 3D points. This projection matrix can be computed from 3D–2D correspondences (minimum 20 points distributed in three different planes). We show how to decompose it to obtain intrinsic and extrinsic parameters. Moreover, we use this parameter estimation followed by a non-linear optimization to calibrate various types of cameras. Our results are based on the sphere camera model which considers that every central catadioptric system can be modeled using two projections, one from 3D points to a unitary sphere and then a perspective projection from the sphere to the image plane. We test our method both with simulations and real images, and we analyze the results performing a 3D reconstruction from two omnidirectional images.     

Pseudo two-dimensional shape normalization methods for handwritten Chinese character recognition

The nonlinear normalization (NLN) method based on line density equalization is popularly used in handwritten Chinese character recognition. To overcome the insufficient shape restoration capability of one-dimensional NLN, a pseudo two-dimensional NLN (P2DNLN) method has been proposed and has yielded higher recognition accuracy. The P2DNLN method, however, is very computationally expensive because of the line density blurring of each row/column. In this paper, we propose a new pseudo 2D normalization method using line density projection interpolation (LDPI), which partitions the line density map into soft strips and generate 2D coordinate mapping function by interpolating the 1D coordinate functions that are obtained by equalizing the line density projections of these strips. The LDPI method adds little computational overhead to one-dimensional NLN yet performs comparably well with P2DNLN. We also apply this strategy to extending other normalization methods, including line density projection fitting, centroid-boundary alignment, moment, and bi-moment methods. The latter three methods are directly based on character image instead of line density map. Their 2D extensions provide real-time computation and high recognition accuracy, and are potentially applicable to gray-scale images and online trajectories.     

三维表面与平面数据间的自适应配准

三维体数据与二维平面数据间的配准是手术导航的基础。与以往的思路有所不同,论文通过把三维表面投影到平面上,利用光照模型计算投影平面的灰度值,实现投影图像与实拍照片间的匹配,在匹配中调整投影参数使得两平面图像达到最佳相似,进而实现三维到二维的自适应配准。     

基于形状特征描述算子的3D模型检索

针对投影图像的形状特征,结合傅里叶描述算子、Zernke不变矩、形状参数、离心率的特点,提出一种基于权重关系的合成的形状特征描述算子,将其应用到3D模型投影图像检索算法中。实验结果表明,相比其他4种形状特征描述算子,该算子能较好描述投影图像的轮廓、区域以及整体特征信息,并可提高算法的检索效率。     

冠脉树三维重建优化方法的改进

在冠脉造影中,两个不同角度成像坐标系间的几何变换矩阵是实现冠脉树三维重建的关键.然而,由于心脏的运动及平台的移动等,直接从造影系统中获得的几何变换矩阵不能很好地反映两个不同角度成像坐标系间的空间几何关系.几何变换矩阵中的平移向量包括两个不同角度成像坐标系间的平移向量和造影系统平台移动向量,由于冠脉树三维重建误差主要是由于平台移动影响造成的,因此,本文利用血管分支点和端点来优化几何变换矩阵中的平移向量.临床造影图像实验结果表明,该方法不仅能有效地提高冠脉树的三维重建精度,而且可获得重建的冠脉树在第三角度下的精确投影图像.