2D/3D image registration using regression learning

In computer vision and image analysis, image registration between 2D projections and a 3D image that achieves high accuracy and near real-time computation is challenging. In this paper, we propose a novel method that can rapidly detect an object’s 3D rigid motion or deformation from a 2D projection image or a small set thereof. The method is called CLARET (Correction via Limited-Angle Residues in External Beam Therapy) and consists of two stages: registration preceded by shape space and regression learning. In the registration stage, linear operators are used to iteratively estimate the motion/deformation parameters based on the current intensity residue between the target projection(s) and the digitally reconstructed radiograph(s) (DRRs) of the estimated 3D image. The method determines the linear operators via a two-step learning process. First, it builds a low-order parametric model of the image region’s motion/deformation shape space from its prior 3D images. Second, using learning-time samples produced from the 3D images, it formulates the relationships between the model parameters and the co-varying 2D projection intensity residues by multi-scale linear regressions. The calculated multi-scale regression matrices yield the coarse-to-fine linear operators used in estimating the model parameters from the 2D projection intensity residues in the registration. The method’s application to Image-guided Radiation Therapy (IGRT) requires only a few seconds and yields good results in localizing a tumor under rigid motion in the head and neck and under respiratory deformation in the lung, using one treatment-time imaging 2D projection or a small set thereof.     

A computational model of bounded developable surfaces with application to image‐based three‐dimensional reconstruction

Developable surfaces have been extensively studied in computer graphics because they are involved in a large body of applications. This type of surfaces has also been used in computer vision and document processing in the context of three‐dimensional (3D) reconstruction for book digitization and augmented reality. Indeed, the shape of a smoothly deformed piece of paper can be very well modeled by a developable surface. Most of the existing developable surface parameterizations do not handle boundaries or are driven by overly large parameter sets. These two characteristics become issues in the context of developable surface reconstruction from real observations. Our main contribution is a generative model of bounded developable surfaces that solves these two issues. Our model is governed by intuitive parameters whose number depends on the actual deformation and including the “flat shape boundary”. A vast majority of the existing image‐based paper 3D reconstruction methods either require a tightly controlled environment or restricts the set of possible deformations. We propose an algorithm for reconstructing our model's parameters from a general smooth 3D surface interpolating a sparse cloud of 3D points. The latter is assumed to be reconstructed from images of a static piece of paper or any other developable surface. Our 3D reconstruction method is well adapted to the use of keypoint matches over multiple images. In this context, the initial 3D point cloud is reconstructed by structure‐from‐motion for which mature and reliable algorithms now exist and the thin‐plate spline is used as a general smooth surface model. After initialization, our model's parameters are refined with model‐based bundle adjustment. We experimentally validated our model and 3D reconstruction algorithm for shape capture and augmented reality on seven real datasets. The first six datasets consist of multiple images or videos and a sparse set of 3D points obtained by structure‐from‐motion. The last dataset is a dense 3D point cloud acquired by structured light. Our implementation has been made publicly available on the authors' web home pages. Copyright © 2012 John Wiley & Sons, Ltd.     

基于实拍图像的人脸真实感重建

给出了基于实拍人脸图像的三维逼真人脸模型的重建算法,该算法首先在两幅人脸图像上交互标识特征点对和输入摄像机的广角参数来实现摄像机定标,进而匹配出两幅人脸图像上的其它对应点,实现模型的三维重建,作者用半自动垭达到匹配目的。用手工编辑建立的二维对应网格,得到初始人脸外开和鲁棒的最大拟然立体虎法自动匹配出稠密的对应点,重建出表示人脸的散乱三维数据点团;最后利用这些稠密的三维数据点去迭代矫正和自适应细分手     

基于二维超声- 三维 CT 配准的肾脏穿刺定位方法可行性研究

医学影像引导的经皮肾穿刺手术是经皮肾镜取石术中建立手术通道的重要手段,高质量的 实时医学图像可以提高术中穿刺精度,减少手术风险。针对自由呼吸下经皮肾穿刺靶点的导航定位问 题,该研究提出了一种基于术中二维超声-术前三维 CT 实时配准的肾脏穿刺定位方法。首先对肾脏轮 廓特征的二维超声图像和三维 CT 图像进行由粗到精的快速配准,然后通过超声探头标定,将配准融合 后的图像注册到手术空间,实现穿刺靶点的实时定位。人体腹部模型实验结果表明,超声探头标定均 方根误差为 0.998 mm,二维超声和三维 CT 数据配准误差为 0.709 mm,平均配准运算时间为 1.15 s,最 终的平均定位误差为 2.265 mm。     

三次B样条插值的网格拼接和融合

目的 网格模型的拼接和融合是3维模型编辑的一个重要方面。为了提高3维模型之间拼接曲面的精度和效率,提出一种基于三次均匀B样条曲线曲面的网格融合方法。方法 首先,利用协变分析和数据驱动方法在目标模型上选定融合区域、确定要融合模型的大小及方向;其次,根据选定的3维网格模型,确定待拼接区域的边界,识别并记录边界点集,利用三次B样条插值边界点集;然后,对边界曲线进行双三次B样条曲面插值得到拼接区域连续曲面,并以此作为两模型拼接时的过渡面;最后,对拼接区域重采样,并对其三角化,以实现网格模型的无缝光滑拼接和融合。结果 为了验证本文方法对3维模型拼接的有效性,选取4组不同的模型,分别对其使用本文提出的融合拼接方法进行实验,对前两组模型的拼接效果进行了对比试验,实验结果表明,本文方法可以达到很好的拼接效果,对于融合区域以外的部分能够保持源模型的细节特征,拼接部分的过渡区域光顺平滑,拼接后的模型完整性佳。在运行时间相差0.05 s内,与数据驱动的建模方法相比,本文方法可以处理的节点数至少多2 000个,面片数至少多5 000个。结论 本文方法能够适用于具有任何边界的模型,在选取模型时,对于模型的形状、大小、拓扑结构等的要求较低,适用于新模型的快速建造,因此,该算法可应用于医学、商业广告、动画娱乐以及几何建模和制造等较为广阔的应用领域。     

基于姿态编码器的2D/3D脊椎医学图像实时配准方法

2D/3D医学图像配准是骨科手术三维实时导航中的一项关键技术,然而传统的基于优化迭代的2D/3D配准方法需要经过多次迭代计算,无法满足医生在手术过程中对于实时配准的要求。针对该问题,提出一种基于自编码器的姿态回归网络来通过隐空间解码捕获几何姿态信息,从而快速地回归出术中X射线图像对应的术前脊椎位置的3D姿态,并经过重新投影生成最终的配准图像。通过引入新的损失函数,以“粗细”结合配准的方式对模型进行约束,保证了姿态回归的精确度。在CTSpine1K脊椎数据集中抽取100组CT扫描图像进行10折交叉验证,实验结果表明：所提出的模型所生成的配准结果图像与X射线图像的平均绝对误差（MAE）为0.04,平均目标配准误差（mTRE）为1.16 mm,单帧耗时1.7 s。与基于传统优化的方法相比,该模型配准时间大幅缩短。相较于基于学习的方法,该模型在快速配准的同时,保证了较高的配准精度。可见,所提模型可以满足术中实时高精配准的要求。     

基于拟蒙特卡罗方法的三维医学重建模型体积测量方法研究

测量医学图像三维重建病灶组织与器官的体积, 为临床诊疗与医学研究提供更可靠的数据。通过先对系列二维医学图像进行预处理, 并对处理后的图像进行三维重建, 可以获得较好的只有表面三角网格的三维重建模型; 然后使用拟蒙特卡罗方法在构造的包围盒内生成低差异分布的随机点, 通过计算模型内的点数量与全部随机点数量的比例进行体积测量。分别对四组重建的三维模型进行体积测量并与蒙特卡罗方法相比, 拟蒙特卡罗方法在测量体积方面具有较好的效果。拟蒙特卡罗方法对三维医学重建模型体积测量可以得到较准确的体积数据, 具有一定的实际应用与理论研究价值。     

Constructing 3D human model from front and side images

Constructing 3D human model from 2D images provides a cost-effective approach to visualize digital human in virtual environment. This paper presents a systematic approach for constructing 3D human model using the front and side images of a person. The silhouettes of human body are first detected and the feature points on the silhouettes are subsequently identified. The feature points are further used to obtain the body dimensions that are necessary for identifying a template 3D human model. The shape of the template human model can be modified by the free-form deformation method. Moreover, the proposed approach has been applied for constructing the 3D human models of 30 subjects. The comparisons between the constructed 3D models and the 3D scanning models of the 30 subjects indicate that the proposed system is very effective and robust.     

3D wavelet-based multiresolution object representation

This paper presents a technique for computing multiresolution shape models of 3D objects acquired as clouds of 3D points. The procedure is fully automated and is able to compute approximations for any object, overcoming sampling irregularity if present (sampling irregularity is a common feature of most 3D acquisition techniques; a typical example is stereo vision). The method described here starts by computing an intermediate mesh that meets the subdivision connectivity requirement needed to allow the computation of the wavelet transform. The mesh is then adjusted to the 3D input data using an iterative deformation process. Finally, a spherical wavelet transform is computed to obtain the object's 3D multiresolution model. This paper shows a number of real objects acquired with different techniques, including hand-held 3D digitizers. The paper also gives some examples of how multiresolution representations can be used in tasks such as acquisition noise filtering, mesh simplification and shape labelling.     

双目立体视觉的三维人脸重建方法

创建逼真的三维人脸模型始终是一个极具挑战性的课题.随着三维人脸模型在虚拟现实、视频监控、三维动画、人脸识别等领域的广泛应用,三维人脸重建成为计算机图像学和计算机视觉领域的一个研究热点.针对这一问题,提出一种基于双目立体视觉的三维人脸重建方法,重建过程中无需三维激光扫描仪和通用人脸模型.首先利用标定的2台摄像机获取人脸正面图像对,通过图像校正使图像对的极线对齐并且补偿摄像机镜头的畸变;在立体匹配方面,选择具有准确可靠视差的人脸边缘特征点作为种子像素,以种子像素的视差作为区域生长的视差,在外极线约束、单调性约束以及对应匹配的边缘特征点的约束下,进行水平扫描线上的区域生长,从而得到整个人脸区域的视差图,提高了对应点匹配的速度和准确度;最后,根据摄像机标定结果和立体匹配生成的视差图计算人脸空间散乱点的三维坐标,对人脸的三维点云进行三角剖分、网格细分和光顺处理.实验结果表明,该方法能够生成光滑、逼真的三维人脸模型,证明了该算法的有效性.     

基于网格变形的从图像重建三维人脸

从图像重建高质量三维人脸一直是计算机视觉和图形学的一个重要研究问题.不同于传统的基于立体匹配的窄基线多视几何和数据驱动的人脸形变方法,提出一种结合网格变形技术和立体视觉原理的、从图像重建高质量三维人脸模型方法.给定从不同视角拍摄的几幅人脸图像,基于健壮图像特征获得可靠的相机外部参数和稀疏三维点;在此基础上,提出一种结合几何细节保持和图像一致性约束的三维人脸变形算法重建三维人脸,通过对人脸模板的网格变形,使得变形人脸在多幅图像中的可见投影具有一致性的图像颜色强度.基于模板的人脸变形可以有效地解决三维模型成像中的遮挡问题,采用健壮估计法消除噪声、离群点和光照对目标函数收敛性的影响,对目标函数的多次非线性优化求解进一步改进了人脸重建的质量.采用合成人脸图像和真实人脸图像重建三维人脸的实验结果表明,文中算法可以从几幅宽基线图像重建高质量的三维人脸模型.     

PCA-based 3D shape reconstruction of human foot using multiple viewpoint cameras

This paper describes a multiple camera-based method to reconstruct the 3D shape of a human foot. From a foot database, an initial 3D model of the foot represented by a cloud of points is built. The shape parameters, which can characterize more than 92% of a foot, are defined by using the principal component analysis method. Then, using ＂active shape models＂, the initial 3D model is adapted to the real foot captured in multiple images by applying some constraints （edge points＇ distance and color variance）. We insist here on the experiment part where we demonstrate the efficiency of the proposed method on a plastic foot model, and also on real human feet with various shapes. We propose and compare different ways of texturing the foot which is needed for reconstruction. We present an experiment performed on the plastic foot model and on human feet and propose two different ways to improve the final 3D shapers accuracy according to the previous experiments＇ results. The first improvement proposed is the densification of the cloud of points used to represent the initial model and the foot database. The second improvement concerns the projected patterns used to texture the foot. We conclude by showing the obtained results for a human foot with the average computed shape error being only 1.06 mm.     

基于单幅正面照片的三维人脸重建方法

三维人脸重建算法需要多张照片实现重建且重建效率低下。针对上述问题,提出一种利用单幅正面照片重建三维人脸的方法。采用薄板样条函数对数据库中的三维人脸确立点对点的对应关系,建立平均三维人脸模型,利用LMA算法优化形状系数以恢复其三维形状,人脸颜色纹理信息可通过垂直投影得到。实验结果表明,利用该方法重建得到的三维人脸逼真且时间效率高。     

Three-dimensional reconstruction using multiresolution photoclinometry by deformation

We present a new photoclinometric reconstruction method based on the deformation of a 3D mesh. The optimization process of our method relies on a maximum-likelihood estimation with a density function measuring discrepancies between observed images and the corresponding synthetic images calculated from the progressively deformed 3D mesh. An input mesh is necessary and can be obtained from other methods or created by implementing a multiresolution scheme. We present a 3D shape model of an asteroid obtained by this method and compare it with the models obtained with two high-resolution 3D reconstruction techniques, stereophotogrammetry, and stereophotoclinometry.     

一种分组并行的轻量化实时微观三维形貌重建方法

微观三维形貌重建作为精密制造领域生产制造的关键环节,其重建过程依赖于高分辨率稠密图像的采集.而面对复杂应用场景的高时效性需求,高分辨率稠密图像的输入会导致运算量与计算复杂度呈几何倍增加,无法实现高效率低延时的实时微观三维形貌重建.针对上述现状,本文提出一种分组并行的轻量级实时微观三维形貌重建方法GPLWS-Net,GPLWS-Net以U型网络为基础构造轻量化主干网络,以并行分组式查询加速三维形貌重建过程,并针对神经网络结构进行重参数化设计避免重建微观结构的精度损失.另外,为弥补现有微观三维重建数据集的缺失,本文公开了一组多聚焦微观三维重建数据集(Micro 3D),其标签数据利用多模态数据融合的方式获取场景高精度的三维结构.结果表明,本文提出的GPLWS-Net网络不仅可以保证重建精度,而且在三组公开数据集中相比于其他五类深度学习方法平均耗时降低39.15%,在Micro 3D数据集中平均耗时降低50.55%,能够实现复杂微观场景的实时三维形貌重建.     

一种快速的三维扫描数据自动配准方法

研究了两幅和多幅深度图像的自动配准问题.在配准两幅深度图像时,结合二维纹理图像配准深度图像,具体过程是:首先,从扫描数据中提取纹理图像,特别地,针对不包含纹理图像的扫描数据提出了一种根据深度图像直接生成纹理图像的方法;然后,基于SIFT(scale-invariant feature transform)特征提取纹理图像中的兴趣像素,并通过预过滤和交叉检验兴趣像素等方法从中找出匹配像素对的候选集;之后,使用RANSAC(random sample consensus)算法,根据三维几何信息的约束找出候选集中正确的匹配像素对和相对应的匹配顶点对,并根据这些匹配顶点对计算出两幅深度图像间的刚体置换矩阵;最后,使用改进的ICP(iterative closest point)算法优化这一结果.在配准多幅深度图像时,提出了一种快速构建模型图的方法,可以避免对任意两幅深度图像作配准,提高了配准速度.该方法已成功应用于多种文物的三维逼真建模.     

面向医疗整容的三维人脸重建与编辑系统

针对医疗整容领域中客户在术前无法直观感受术后整容效果的问题, 提出了面向医疗整容领域的三维人脸重建与编辑系统. 该系统首先对用户上传的图片进行特征点标记, 然后结合三维形变模型(3D morphable model, 3DMM)对输入图像进行对齐, 接着将处理后的图像输入预先训练好的三维人脸重建网络中, 便可得到输入图像所对应的三维人脸模型. 系统对此模型进行加载渲染后, 用户可以对模型的脸颊、鼻梁和下巴进行编辑以达到模拟整容的效果, 之后可对模型进行保存并查看诊断结果. 最后, 对重建效果、整形效果和诊断结果可靠性进行测试. 实验结果表明, 该系统对中青年人脸的重建效果好, 重建模型与输入图片相似度高; 对模型整形后的部位依然保持平滑自然, 达到了模拟整容的效果; 在给定正确的人脸尺寸后, 诊断结果给出的整形建议在毫米级范围内, 说明了整形结果具有较高的可靠性.     

3D woven composite design using a flattening simulation

Fiber composite materials have unique, advantageous mechanical properties that have made them highly desirable in a range of industries. In particular, 3D woven-fiber composites are highly resistant to delamination compared with laminated 2D woven-fiber composites and have been adopted in various advanced products. This paper focuses on the design of 3D woven-fiber composite products and proposes a flattening simulation method for designed 3D models with constant thickness. The proposed method estimates the shape of a flat material and the fiber directions in the 3D model design; deformation phenomena of 3D woven-fiber materials are also considered in order to improve the accuracy of the proposed method. CT images are used to compare the simulation results with the actual deformation of 3D woven-fiber materials and confirm the ability of our method to effectively design the fiber direction base on the 3D model and to estimate the shape of flat materials.     

基于Candide模型的人脸深度信息生成技术研究

目前通过2D转3D技术转换的3D电影场景中,绝大部分关于人脸的深度信息生成都不令人满意：深度值不准确、立体效果差。文中为解决特定二维人脸图像转换为三维人脸图像提供了一个比较简单有效的方法。首先,应用Candide模型作为通用的3D人脸模型,然后对通用模型调整控制点进行全局变换和局部变换,通过三维模型与人脸最大限度的匹配得到特定人脸三维网格模型。对网格模型采用顶点规则化和NURBS曲面拟合方法获得光滑的模型表面使其逼近人脸,最后生成准确的人脸深度信息。     

基于形变模型的多角度三维人脸实时重建

摘 要：采用人脸特征点调整三维形变模型的方法应用于面部三维重建,但模型形变的计 算往往会产生误差,且耗时较长。因此运用人脸二维特征点对通用三维形变模型的拟合方法进 行改进,提出了一种视频流的多角度实时三维人脸重建方法。首先利用带有三层卷积网络的 CLNF 算法识别二维特征点,并跟踪特征点位置;然后由五官特征点位置估计头部姿态,更新 模型的表情系数,其结果再作用于 PCA 形状系数,促使当前三维模型发生形变;最后采用 ISOMAP 算法提取网格纹理信息,进行纹理融合形成特定人脸模型。实验结果表明,该方法在 人脸重建过程中具有更好的实时性能,且精确度有所提高。