Implementation,calibration and accuracy testing of an image-enhanced endoscopy system

This paper presents a new method for image-guided surgery called image-enhanced endoscopy. Registered real and virtual endoscopic images (perspective volume renderings generated from the same view as the endoscope camera using a preoperative image) are displayed simultaneously; when combined with the ability to vary tissue transparency in the virtual images, this provides surgeons with the ability to see beyond visible surfaces and, thus, provides additional exposure during surgery. A mount with four photoreflective spheres is rigidly attached to the endoscope and its position and orientation is tracked using an optical position sensor. Generation of virtual images that are accurately registered to the real endoscopic images requires calibration of the tracked endoscope. The calibration process determines intrinsic parameters (that represent the projection of three-dimensional points onto the two-dimensional endoscope camera imaging plane) and extrinsic parameters (that represent the transformation from the coordinate system of the tracker mount attached to the endoscope to the coordinate system of the endoscope camera), and determines radial lens distortion. The calibration routine is fast, automatic, accurate and reliable, and is insensitive to rotational orientation of the endoscope. The routine automatically detects, localizes, and identifies dots in a video image snapshot of the calibration target grid and determines the calibration parameters from the sets of known physical coordinates and localized image coordinates of the target grid dots. Using nonlinear lens-distortion correction, which can be performed at real-time rates (30 frames per second), the mean projection error is less than 0.5 mm at distances up to 25 mm from the endoscope tip, and less than 1.0 mm up to 45 mm. Experimental measurements and point-based registration error theory show that the tracking error is about 0.5-0.7 mm at the tip of the endoscope and less than 0.9 mm for all points in the field of view of the endoscope camera at a distance of up to 65 mm from the tip. It is probable that much of the projection error is due to endoscope tracking error rather than calibration error. Two examples of clinical applications are presented to illustrate the usefulness of image-enhanced endoscopy. This method is a useful addition to conventional image-guidance systems, which generally show only the position of the tip (and sometimes the orientation) of a surgical instrument or probe on reformatted image slices.     

一种新的深度传感器内部参数标定方法研究

针对双镜头深度传感器(以Kinect为例)出厂标 定参数精度不高的问题,提出一种新的标定方法。 对于Kinect 2.0深度镜头,利用空间线定长约束,通过间接平差方法求解待求参数;根据求 解参数,将深度图 像坐标转转换值Kinect坐标并将其与对应的彩色影像坐标点进行关联,基于中心投影方程标 定彩色镜头。实 验结果表明,本方法将深度影像点转换到Kinect坐标时精度优于2.5 mm,深度影像坐标转换至彩色影像坐标 时精度优于1pixel,高于Kinect微软开发包内置参数的计算精度,对一些需要较高参 数精度的应用,本文算法解算的参数更优。     

多光谱遥感卫星图像的精确配准方法研究

针对一组多光谱遥感图像中,各谱段图像之间配准不一致的问题,本文提出了一种基于特征点的快速自动配准方法。在图像信息熵的基础上,利用环形移动窗口,自动快速寻找感兴趣区域, 并利用尺度不变特征转换（SIFT）算法提取特征。为提高精度,文中对特征初匹配方法作了改进,并用余弦定理和空间距离约束条件剔除误匹配点,之后提取最稳定的特征点对计算变换参数,完成配准。最后根据配准前后图像的互信息和特征点的均方根误差（RMSE）来衡量配准的程度。通过对大量中巴地球资源卫星拍摄的多光谱图像进行实验,该方法能达到亚像素级配准精度,并能快速对各谱段图像进行配准。      

一种简易的激光雷达点云与光学影像自动配准方法

将激光雷达点云与光学影像进行配准融合,在数字城市建模、自动驾驶和导航避障等方面有重要的应用价值。当前多数自动配准方法需要专门设计特殊的场景结构,操作复杂。根据激光雷达对玻璃具有穿透的特性,提出利用透明的门窗等室内建筑物场景来实现对激光雷达点云与光学影像的自动配准。首先,将激光点云量化为图像,提取光学影像与量化图像的门窗角点特征,并利用相关系数法实现特征点的自动匹配;然后,对激光雷达与相机之间的配准转换参数进行求解;最后,对两种数据进行配准融合。实验结果表明:利用室内门窗这种规则的半透明场景,可以轻松实现激光点云与光学影像的自动化配准,简单实用,操作性强。     

基于ISS-3DSC的NDT三维点云配准算法研究

工件形貌的三维扫描需求在车间自动化装备中越来越多,其中点云配准作为三维数据处理的重要步骤。现有三维点云配准存在特征点对误配、配准时间长、配准精度差等问题,提出了一种基于内部形状描述子-三维形状上下文特征(ISS-3DSC)的NDT三维点云配准算法。首先通过内部形状描述子(ISS)算法提取三维点云关键点,提高配准效率;然后结合三维形状上下文特征(3DSC)进行关键点的特征描述,并根据特征点对中值距离法剔除错误点对,采用SVD分解计算初始变换矩阵;最后使用NDT算法完成精配准。测试实验结果表明：算法在鞋面、鞋底点云数据配准时的精度可达到0.025 cm,相比传统SAC-IA+NDT算法配准效率提升明显,具有一定的工程应用价值。     

基于高程和地物光谱约束的多光谱图像预处理算法

分视场滤光片型多光谱相机在利用拼接和配准生成多光谱图像时,受云、水等特殊地物的影响,利用现有方法生成的结果图像上容易出现条带噪声和错配现象。提出了一种三维信息约束下的尺度不变特征转换(SIFT)图像配准和地物光谱特性约束下的多光谱图像预处理算法,该算法利用三维信息约束的SIFT算子提高配准精度,同时在地物光谱特性约束下选取有效的辐射校正点提高图像拼接时各条带图像灰度的校正精度。实验结果表明,利用该方法对分视场多光谱相机数据进行预处理时,即使图像上存在云、水等特殊区域的地物,结果图像仍能保持高精度配准且无条带噪声。     

Fast and accurate automatic registration for MR-guided procedures using active microcoils

A fast, robust, accurate, and automatic registration technique based on magnetic resonance (MR) active microcoils (active markers) for registration of tracked medical devices to preprocedural MR-images is presented. This allows for a straight-forward integration of position measurement systems into clinical procedures. The presented method is useful for guidance purposes in clinical applications with high demands on accuracy and ease-of-use (e.g., neurosurgical or orthopedic applications). The determination of the positions of the active markers is integrated into the preparation phase of the actual MR imaging scan. The technique features a generic interface using DICOM standards for communication with navigation workstations linked to an MR system. The position of the active markers is fixed with respect to a reference system of an optical positioning measurement system (OPMS) and thus the coregistration of the MR system and the OPMS is established. In a phantom study, a mean overall targeting accuracy of 0.9+/-0.1 mm was achieved and compared favorably to results obtained from manual registration tests (1.8+/-0.3 mm) carried out in parallel. For a test person trained for both registration methods, workflow improvements of 3-6 min per registration step were found. The need for manual interaction is entirely eliminated thus avoiding user-bias, which is advantageous for the usage in clinical routine. The method improves the ease-of-use of tracking equipment during stereotactic guidance. The method is finally demonstrated in a volunteer study using a model of a Mayfield skull clamp with integrated active and optical reference markers.     

Validation of object-induced MR distortion correction for framelessstereotactic neurosurgery

Spatial fidelity is a paramount issue in image guided neurosurgery. Until recently, three-dimensional computed tomography (3D CT) has been the primary modality because it provides fast volume capture with pixel level (1 mm) accuracy. While three-dimensional magnetic resonance (3D MR) images provide superior anatomic information, published image capture protocols are time consuming and result in scanner- and object-induced magnetic field inhomogeneities which raise inaccuracy above pixel size. Using available scanner calibration software, a volumetric algorithm to correct for object-based geometric distortion, and a Fast Low Angle SHot (FLASH) 3D MR-scan protocol, the authors were able to reduce mean CT to MR skin-adhesed fiducial marker registration error from 1.36 to 1.09 mm. After dropping the worst one or two of six fiducial markers, mean registration error dropped to 0.62 mm (subpixel accuracy). Three dimensional object-induced error maps present highest 3D MR spatial infidelity at the tissue interfaces (skin/air, scalp/skull) where frameless stereotactic fiducial markers are commonly applied. The algorithm produced similar results in two patient 3D MR-scans     

一种基于单目视觉的无人机室内定位方法

目前,无人机定位技术主要依赖以GPS(Global Positioning System)为代表的全球定位系统,然而在室内等GPS信号缺失的地方进行定位则比较困难.另外,传统的室内定位技术主要采用蓝牙、WiFi、基站定位等多种方式融合成一套定位体系,但是该类方法受环境的影响比较大,而且往往需要部署多个设备.此外,这种方...     

基于场景复杂度与不变特征的航拍视频实时配准算法

实时、鲁棒的图像配准是航拍视频电子稳像、全景图拼接和地面运动目标自动检测与跟踪的前提和关键技术.本文以航拍视频序列为处理对象,提出了一种新的基于场景复杂度与不变特征的实时配准算法,其主要特点包括：(1)在对航拍视频配准难点进行详细分析的基础上,有针对性的提出基于积分图的快速图像尺度空间构建、依据场景复杂度的检测特征点数量在线精确控制、基于描述子误差分布统计特性级的联分类器构造等新方法,使得算法配准性能不随场景的复杂度发生改变,能够在各种地貌条件下实时、稳定的进行图像配准;(2)将多尺度Harris角点和SIFT描述子相结合,并通过对帧间变换模型参数进行鲁棒估计,保证了算法具有良好的旋转、尺度、亮度不变性和配准精度.实验结果表明,算法可在场景变化、图像大幅度平移、尺度缩放和任意角度旋转等复杂条件下实时、精确的进行图像配准,对分辨率为320×240的航拍序列的平均处理速度达到20.7帧/秒.     

稳健的图像匹配方法

景象匹配是计算机视觉研究的一个重要方向,如何在复杂的环境下提高匹配概率与定位精度是其亟待解决的难点之一。在分析畸变校正的量化噪声为高斯噪声的基础上,提出了基于特征和时空关联的积相关图像匹配算法。采用基于噪声抑制小波边缘检测方法,提取实时图和基准图的边缘特征;基于归一化积相关实现序列实时图(3帧)与基准图的匹配,再根据位置信息实现相关峰值的数据融合得到匹配点。此方法克服了因面积增大由几何失真导致的匹配概率下降的缺点,提高了匹配概率,具有较好的匹配稳健性。     

基于EPNP算法的单目视觉测量系统研究

利用计算机视觉进行姿态测量的方法已广泛应用于现代控制、导航、跟踪等多个领域中。研究并设计了一种基于P4P矩形分布的平面靶标和EPNP算法结合的单目视觉姿态测量方法。首先,利用单相机获取平面靶标图像,经图像处理后得到四个特征点的像素坐标,并使用EPNP算法进行姿态解算;其次,对姿态角测量误差进行了仿真分析,为提高姿态测量精度提供了理论指导和依据;最后,提出一种与高精度二维转台结合的坐标系配准方法,利用该方法对三个方向姿态角精度进行验证。实验结果表明:当绕x和y轴的转动角度在[-6,6]时,姿态测量误差小于0.1,可以满足测量应用需求。     

An efficient image-registration method based on probability density and global parallax

This paper is concerned with the problem of image registration of video sequence, to meet the accuracy and real-time requirements for video analysis and processing. Despite of the existence of sophisticated registration algorithms, it is still problematic to register images precisely with low computation complexity due to the lack of stable features and effective matching method. In this paper, a probability density gradient based interest point detector is employed to extract stable point features precisely. And a robust technique, namely, global parallax histogram based filter is proposed to discard outliers in the initial candidate matches set found by classical correlation method. The registration matrix can then be accurately and precisely estimated using a well adapted criterion. Several field tests are performed on complex and challenging images to assess the performance, including comparison to conventional algorithms, and both inter-frame registration results and statistical analysis of video sequence. These simulations validate the improvement of proposed method in accuracy and efficiency, and the robustness against camera motions, illumination variations, acquirement conditions, moving objects and image noise.     

Automatic correction of registration errors in surgical navigation systems

Surgical navigation systems are used widely among all fields of modern medicine, including, but not limited to ENT- and maxillofacial surgery. As a fundamental prerequisite for image-guided surgery, intraoperative registration, which maps image to patient coordinates, has been subject to many studies and developments. While registration methods have evolved from invasive procedures like fixed stereotactic frames and implanted fiducial markers toward surface-based registration and noninvasive markers fixed to the patient's skin, even the most sophisticated registration techniques produce an imperfect result. Due to errors introduced during the registration process, the projection of navigated instruments into image data deviates up to several millimeter from the actual position, depending on the applied registration method and the distance between the instrument and the fiducial markers. We propose a method that allows to automatically and continually improve registration accuracy during intraoperative navigation after the actual registration process has been completed. The projections of navigated instruments into image data are inspected and validated by the navigation software. Errors in image-to-patient registration are identified by calculating intersections between the virtual instruments' axes and surfaces of hard bone tissue extracted from the patient's image data. The information gained from the identification of such registration errors is then used to improve registration accuracy by adding an additional pair of registration points at every location where an error has been detected. The proposed method was integrated into a surgical navigation system based on paired points registration with anatomical landmarks. Experiments were conducted, where registrations with deliberately misplaced point pairs were corrected with automatic error correction. Results showed an improvement in registration quality in all cases.     

Real-time multimodal retinal image registration for a computer-assisted laser photocoagulation system

An algorithm for the real-time registration of a retinal video sequence captured with a scanning digital ophthalmoscope (SDO) to a retinal composite image is presented. This method is designed for a computer-assisted retinal laser photocoagulation system to compensate for retinal motion and hence enhance the accuracy, speed, and patient safety of retinal laser treatments. The procedure combines intensity and feature-based registration techniques. For the registration of an individual frame, the translational frame-to-frame motion between preceding and current frame is detected by normalized cross correlation. Next, vessel points on the current video frame are identified and an initial transformation estimate is constructed from the calculated translation vector and the quadratic registration matrix of the previous frame. The vessel points are then iteratively matched to the segmented vessel centerline of the composite image to refine the initial transformation and register the video frame to the composite image. Criteria for image quality and algorithm convergence are introduced, which assess the exclusion of single frames from the registration process and enable a loss of tracking signal if necessary. The algorithm was successfully applied to ten different video sequences recorded from patients. It revealed an average accuracy of 2.47 ± 2.0 pixels (～23.2 ± 18.8 μm) for 2764 evaluated video frames and demonstrated that it meets the clinical requirements.     

一种基于深度和局部特征的分布式光电图像配准方法北大核心CSCD

针对分布式光电成像系统采集的红外和可见光图像在配准时易受噪声影响,配准精度不高问题,提出一种基于卷积神经网络深度特征和RIFT局部特征的图像配准算法。首先基于改进的AVIRnet提取待配准红外和可见光图像的卷积深度特征,利用深度特征进行初匹配,得到初步的空间关系;然后在重叠图像区域内提取RIFT特征点;最后对局部特征点进行修正,得到最终的匹配点对,估算出精确的变换矩阵。实验结果表明:本文方法通过深度特征和局部特征两次匹配,对非线性辐射差异具有不变性,满足了分布式光电红外和可见光图像配准的精度要求。     

Surgical navigation by autostereoscopic image overlay of integral videography

This paper describes an autostereoscopic image overlay technique that is integrated into a surgical navigation system to superimpose a real three-dimensional (3-D) image onto the patient via a half-silvered mirror. The images are created by employing a modified version of integral videography (IV), which is an animated extension of integral photography. IV records and reproduces 3-D images using a microconvex lens array and flat display; it can display geometrically accurate 3-D autostereoscopic images and reproduce motion parallax without the need for special devices. The use of semitransparent display devices makes it appear that the 3-D image is inside the patient's body. This is the first report of applying an autostereoscopic display with an image overlay system in surgical navigation. Experiments demonstrated that the fast IV rendering technique and patient-image registration method produce an average registration accuracy of 1.13 mm. Experiments using a target in phantom agar showed that the system can guide a needle toward a target with an average error of 2.6 mm. Improvement in the quality of the IV display will make this system practical and its use will increase surgical accuracy and reduce invasiveness.     

基于边缘特征的异质图像目标配准方法

目标检测作为计算机视觉的重要研究课题,在视频监控和智能导航方面有重要应用.图像配准技术是刚体目标检测方法之一.异质图像间目标的纹理差异极大,为此,对于目标的可见光俯视图像与机载红外摄像机观测异质图像,通过研究基于边缘特征的配准技术,实现对地面平面目标的配准检测.笔者首先通过建立机载运动摄像机空间模型,解决了红外观测图像的视角变换问题,将其换到与参考图像一致的视角,提出一种新颖的基于稳定区域轮廓合成的边缘提取方法,更好地控制异质图像的边缘提取效果,边缘特征匹配是在距离变换的基础上实现的,提出了一种基于稳定区域距离图像互相关的匹配度量准则.实验表明,算法能有效地实现异质图像间的目标配准检测.     

Robust feature points extraction for image registration based on the nonsubsampled contourlet transform

In this letter, a new feature points extraction method based on the nonsubsampled contourlet transform (NSCT) is proposed for image registration. The primary motivation of this work is to determine the effectiveness of the NSCT transform in extracting feature points for image registration. Preliminary experimental results show that the registration accuracy and robustness of the proposed algorithm is acceptable and very promising, and confirm the success of the proposed NSCT-based feature points extraction approach.