Markerless real-time 3-D target region tracking by motion backprojection from projection images

Accurate and fast localization of a predefined target region inside the patient is an important component of many image-guided therapy procedures. This problem is commonly solved by registration of intraoperative 2-D projection images to 3-D preoperative images. If the patient is not fixed during the intervention, the 2-D image acquisition is repeated several times during the procedure, and the registration problem can be cast instead as a 3-D tracking problem. To solve the 3-D problem, we propose in this paper to apply 2-D region tracking to first recover the components of the transformation that are in-plane to the projections. The 2-D motion estimates of all projections are backprojected into 3-D space, where they are then combined into a consistent estimate of the 3-D motion. We compare this method to intensity-based 2-D to 3-D registration and a combination of 2-D motion backprojection followed by a 2-D to 3-D registration stage. Using clinical data with a fiducial marker-based gold-standard transformation, we show that our method is capable of accurately tracking vertebral targets in 3-D from 2-D motion measured in X-ray projection images. Using a standard tracking algorithm (hyperplane tracking), tracking is achieved at video frame rates but fails relatively often (32% of all frames tracked with target registration error (TRE) better than 1.2 mm, 82% of all frames tracked with TRE better than 2.4 mm). With intensity-based 2-D to 2-D image registration using normalized mutual information (NMI) and pattern intensity (PI), accuracy and robustness are substantially improved. NMI tracked 82% of all frames in our data with TRE better than 1.2 mm and 96% of all frames with TRE better than 2.4 mm. This comes at the cost of a reduced frame rate, 1.7 s average processing time per frame and projection device. Results using PI were slightly more accurate, but required on average 5.4 s time per frame. These results are still substantially faster than 2-D to 3-D registration. We conclude that motion backprojection from 2-D motion tracking is an accurate and efficient method for tracking 3-D target motion, but tracking 2-D motion accurately and robustly remains a challenge.     

基于角膜反射的非侵入式视线跟踪技术

针对现有的视线跟踪技术设备复杂、对姿势限制过多、要头部佩戴专用设备等方面的不足,提出一种实现视线跟踪技术的新方法,采用5个近红外光灯作为角膜反射的光源。该方法不需要带任何头戴设备,并且能适应头部的自然运动,设计的算法计算出来的视线位置精度较高。提出一种新的瞳孔边缘拟合方案,循环拟合剔除假点,最终得到精确的瞳孔中心,提高了映射的精度。利用图像的灰度进行人眼的初定位,快速而准确。利用交比不变原理进行坐标映射,精确计算出注视坐标。     

Constrained registration for motion compensation in atrial fibrillation ablation procedures

Fluoroscopic overlay images rendered from preoperative volumetric data can provide additional anatomical details to guide physicians during catheter ablation procedures for treatment of atrial fibrillation (AFib). As these overlay images are often compromised by cardiac and respiratory motion, motion compensation methods are needed to keep the overlay images in sync with the fluoroscopic images. So far, these approaches have either required simultaneous biplane imaging for 3-D motion compensation, or in case of monoplane X-ray imaging, provided only a limited 2-D functionality. To overcome the downsides of the previously suggested methods, we propose an approach that facilitates a full 3-D motion compensation even if only monoplane X-ray images are available. To this end, we use a training phase that employs a biplane sequence to establish a patient specific motion model. Afterwards, a constrained model-based 2-D/3-D registration method is used to track a circumferential mapping catheter. This device is commonly used for AFib catheter ablation procedures. Based on the experiments on real patient data, we found that our constrained monoplane 2-D/3-D registration outperformed the unconstrained counterpart and yielded an average 2-D tracking error of 0.6 mm and an average 3-D tracking error of 1.6 mm. The unconstrained 2-D/3-D registration technique yielded a similar 2-D performance, but the 3-D tracking error increased to 3.2 mm mostly due to wrongly estimated 3-D motion components in X-ray view direction. Compared to the conventional 2-D monoplane method, the proposed method provides a more seamless workflow by removing the need for catheter model re-initialization otherwise required when the C-arm view orientation changes. In addition, the proposed method can be straightforwardly combined with the previously introduced biplane motion compensation technique to obtain a good trade-off between accuracy and radiation dose reduction.     

Computer-Assisted Preoperative Planning for Reduction of Proximal Femoral Fracture Using 3-D-CT Data

This paper describes procedures for repositioning calculations of fractured bone fragments using 3-D-computed tomography (CT), aimed at preoperative planning for computer-guided fracture reduction of the proximal femur. Fracture boundaries of the bone fragments, as ldquofracture lines (FLs),rdquo and the mirror-transformed contralateral femur shape extracted from 3-D-CT were used for repositioning of the fragments. We first describe a method for extracting FLs based on 3-D curvature analysis and then formulate repositioning methods based on registration of bone fragments using the following three constraints: 1) contralateral (CL) femur shape; 2) FLs; and 3) both CL femur shape and fracture lines, as ldquoboth constraintsrdquo. We performed experiments using CT datasets from five simulated and four real patients with proximal femoral fracture. We evaluated the rotation error in reposition calculations and the contact ratio between repositioned fragment boundaries, which are crucial for the recovery of proper functional axes and bone adhesion of fragments, respectively. Experimental results showed that good accuracy and stability were attainable when registration using both constraints was performed after registration using the fracture-line constraint. On average, 6.0deg plusmn0.8deg in rotation error and 89% plusmn 3% in contact ratio were obtained without providing precise initial values.     

Gaze estimation using a webcam for region of interest detection

In this paper, a real-time gaze estimation system using a webcam is proposed, in which variation of head pose is tracked. At first, variation of head position and pose are estimated by using facial features. Then, an iterative iris center detection method is proposed for tracking iris in eye image. Finally, gaze is estimated by using estimated head pose and position, and iris center position. The proposed gaze estimation system is applied to four different applications. Experimental results show that the proposed iterative iris center detection method has a higher accuracy than conventional ones. Also, the proposed gaze estimation system shows about 98 % accuracy using \(640\times 480\) resolution webcam and 42-inch monitor that are 0.75 m apart.     

Registration of three-dimensional cardiac catheter models to single-plane fluoroscopic images

Transvenous cardiac procedures require accurate positioning of catheters within the geometrically complex cavities of the heart. Recently, nonfluoroscopic catheter tracking technologies have been developed to quantitate the (degrees-of-freedom) three-dimensional positions of intracardiac catheters. This paper presents a projection-Procrustes method to register an animated three-dimensional (3-D) model of multiple intracardiac catheters with a single-plane fluoroscopic image. Applying the computed transformation to the catheter coordinates enables the animated 3-D model of the catheters to be viewed from the same perspective as the fluoroscopic image. Mathematical simulations show that the computed transformation parameters are sensitive to both the position errors in the 3-D catheter coordinates and to the spatial distribution of the catheter-mounted transducers. Simulations with a realistic geometric model of three catheters with four transducers per catheter showed an angular error of 1.91 degrees +/- 0.27 degree for 3-D catheter position errors of 2.0 mm. An in vitro experiment demonstrated the feasibility of the method using a water tank phantom of three catheters and fluoroscopic images taken over an 80 degrees range. The mean angular error was 0.61 degree +/- 0.48 degree. The results of this study indicate that the projection-Procrustes method is a useful tool for registering 3-D catheter tracking models to single-plane fluoroscopic images.     

Gradient-based 2-D/3-D rigid registration of fluoroscopic X-ray to CT

We present a gradient-based method for rigid registration of a patient preoperative computed tomography (CT) to its intraoperative situation with a few fluoroscopic X-ray images obtained with a tracked C-arm. The method is noninvasive, anatomy-based, requires simple user interaction, and includes validation. It is generic and easily customizable for a variety of routine clinical uses in orthopaedic surgery. Gradient-based registration consists of three steps: 1) initial pose estimation; 2) coarse geometry-based registration on bone contours, and; 3) fine gradient projection registration (GPR) on edge pixels. It optimizes speed, accuracy, and robustness. Its novelty resides in using volume gradients to eliminate outliers and foreign objects in the fluoroscopic X-ray images, in speeding up computation, and in achieving higher accuracy. It overcomes the drawbacks of intensity-based methods, which are slow and have a limited convergence range, and of geometry-based methods, which depend on the image segmentation quality. Our simulated, in vitro, and cadaver experiments on a human pelvis CT, dry vertebra, dry femur, fresh lamb hip, and human pelvis under realistic conditions show a mean 0.5-1.7 mm (0.5-2.6 mm maximum) target registration accuracy.     

A robust 3-D IVUS transducer tracking using single-plane cineangiography

During an intravascular ultrasound (IVUS) intervention, a catheter with an ultrasound transducer is introduced in the body through a blood vessel, and then, pulled back to image a sequence of vessel cross sections. Unfortunately, there is no 3-D information about the position and orientation of these cross-section planes, which makes them less informative. To position the IVUS images in space, some researchers have proposed complex stereoscopic procedures relying on biplane angiography to get two X-ray image sequences of the IVUS transducer trajectory along the catheter. To simplify this procedure, we and others have elaborated algorithms to recover the transducer 3-D trajectory with only a single view X-ray image sequence. In this paper, we present an improved method that provides both automated 2-D and 3-D transducer tracking based on pullback speed as a priori information. The proposed algorithm is robust to erratic pullback speed and is more accurate than the previous single-plane 3-D tracking methods.     

Redundant system of passive markers for ultrasound scanhead tracking

Scanhead tracking by opto-electronic (OE) systems allows high accuracy in three-dimensional (3-D) freehand ultrasound imaging. In this paper, a new set of methods is proposed and compared with the standard approach [Gram-Schmidt method (GS)]. Three redundancy-based algorithms are introduced to compensate for possible loss of markers during data acquisition: regression plane (RP), multiple Gram-Schmidt (MGS), and center of mass least square (CMLS). When combined with the ultrasound instrument, the root-mean-squared (RMS) uncertainty in locating target points, over a working volume of 420 mm x 490 mm x 100 mm, improved by 7% and 24% using MGS and CMLS method respectively, compared to GS. A lower improvement was obtained with RP methods (5%), using the best marker configuration. In conclusion, CMLS method provides a robust and accurate procedure for 3-D freehand ultrasound scanhead tracking, able to manage possible loss of markers, with interesting perspectives for image fusion and body referenced 3-D ultrasound.     

Infrared image recognition for seam tracking monitoring during fiber laser welding

Seam-tracking ability of a laser-welding system is important for welding process, and the accurate detection of deviations between the laser-beam focus and the weld seam position is prerequisite for seam-tracking control. Infrared image sensing and visual recognition techniques for real-time seam tracking monitoring during high-power fiber laser welding is researched to improve the accuracy of seam-tracking ability. Molten pool images are caught by an infrared sensitive high speed camera arranged off-axis orientation of a laser-welding head which is fixed to a robot. Through the image processing, the feature detection of a near-infrared image is used in visual tracking. The gray-value gradient of near-infrared image is calculated and the keyhole margin of a molten pool is also detected. Combining the gradient and keyhole margin of a molten pool image, the thermal gradient parameter based on the thermal distribution of a molten pool is extracted. As a visual feature in robot control system, this parameter can be used to determine the deviations between the laser-beam focus and the weld-seam center. In comparison with direct detection of the narrow gap position, this parameter can be measured easily and the delay error resulted from the forerun of the sensor can be eliminated. This provides a practical approach to detect the deviations and the possibility to adjust the laser-beam focus position in real time, which can sensibly promote seam tracking accuracy. The proposed algorithm is tested during a butt-joint laser welding of Type 304 austenitic stainless steel plates at a continuous wave fiber laser power of 6 kW and 10 kW. Its effectiveness is confirmed by the welding experiments.     

带宽自适应Mean Shift跟踪算法

提出了一种先进行空间定位再确定目标尺度的两级跟踪算法,有效地解决了mean shift算法对尺度变化的适应问题.该算法首先在当前帧对应位置进行降分辨率处理,并以基于增量试探的mean shift跟踪算法收敛点作为当前帧目标中心位置,进而利用对数极坐标变换的旋转、尺度不变性,对目标和候选目标分别进行对数极坐标映射,并通过求取最大归一化相关函数确定目标的尺度变化.跟踪实验表明,该算法可以有效的提高mean shift跟踪算法空间和尺度定位准确性.     

Registration algorithm based on image matching for outdoor AR system with fixed viewing position

An image registration method based on the Fourier-Mellin transform is introduced for an outdoor augmented reality (AR) system. For this type of AR system, the observation position is fixed, and a complex 3-D registration problem can be reduced to a 2-D image registration for this fixed viewing position system. An observation globe model for this method is proposed. Under this supposition, a Fourier-Mellin transform is used in image registration, and the architecture of this system is illustrated. Experimental results show that this image registration algorithm is accurate and robust. It is effective for an outdoor AR system with a fixed viewing position.     

基于同名点追踪的空间相机成像拼接配准模型

分析了已有图像配准算法在遥感图像拼接配准方面面临的问题。根据空间相机TDICCD交错拼接的成像特点,提出了一种基于同名点轨迹追踪的成像拼接配准模型。通过建立辅助空间坐标系下的中心投影共线方程,将像点、摄像中心、景物点建立严谨的数学关系,可精确实现对同名像点在像面上的轨迹追踪。结合TDICCD在像面上的位置约束,计算图像上同名像对的纵向偏差像元数和横向偏差像元数。最后结合相机在轨所成条带图像和卫星辅助数据进行分析,选取多组像点进行配准,同名像对配准误差小于1 pixel,经验证模型算法可行。相比传统的遥感图像配准算法,该方法不需要已知图像内容,为一种严格的几何意义上的配准,具有很强的适应性和预测性,已应用在型号相机的地面复算,易移植应用于其他类型空间相机的图像配准与拼接。     

Multiple-object 2-D-3-D registration for noninvasive pose identification of fracture fragments

This paper presents a multiple-object 2-D-3-D registration technique for noninvasively identifying the poses of fracture fragments in the space of a preoperative treatment plan. The plan is made by manipulating and aligning computer models of individual fracture fragments that are segmented from a diagnostic computed tomography. The registration technique iteratively updates the treatment plan and matches its digitally reconstructed radiographs to a small number of intraoperative fluoroscopic images. The proposed approach combines an image similarity metric that integrates edge information with mutual information, and a global-local optimization scheme, to deal with challenges associated with the registration of multiple small fragments and limited imaging orientations in the operating room. The method is easy to use as minimum user interaction is required. Experiments on simulated fractures and two distal radius fracture phantoms demonstrate clinically acceptable target registration errors with capture range as large as 10 mm.     

Automated segmentation of acetabulum and femoral head from 3-d CT images

This paper describes several new methods and software for automatic segmentation of the pelvis and the femur, based on clinically obtained multislice computed tomography (CT) data. The hip joint is composed of the acetabulum, cavity of the pelvic bone, and the femoral head. In vivo CT data sets of 60 actual patients were used in the study. The 120 (60 /spl times/ 2) hip joints in the data sets were divided into four groups according to several key features for segmentation. Conventional techniques for classification of bony tissues were first employed to distinguish the pelvis and the femur from other CT tissue images in the hip joint. Automatic techniques were developed to extract the boundary between the acetabulum and the femoral head. An automatic method was built up to manage the segmentation task according to image intensity of bone tissues, size, center, shape of the femoral heads, and other characters. The processing scheme consisted of the following five steps: 1) preprocessing, including resampling 3-D CT data by a modified Sine interpolation to create isotropic volume and to avoid Gibbs ringing, and smoothing the resulting images by a 3-D Gaussian filter; 2) detecting bone tissues from CT images by conventional techniques including histogram-based thresholding and binary morphological operations; 3) estimating initial boundary of the femoral head and the joint space between the acetabulum and the femoral head by a new approach utilizing the constraints of the greater trochanter and the shapes of the femoral head; 4) enhancing the joint space by a Hessian filter; and 5) refining the rough boundary obtained in step 3) by a moving disk technique and the filtered images obtained in step 4). The above method was implemented in a Microsoft Windows software package and the resulting software is freely available on the Internet. The feasibility of this method was tested on the data sets of 60 clinical cases (5000 CT images).     

Guide wire reconstruction and visualization in 3DRA using monoplane fluoroscopic imaging

A method has been developed that, based on the guide wire position in monoplane fluoroscopic images, visualizes the approximate guide wire position in the three-dimensional (3-D) vasculature, that is obtained prior to the intervention with 3-D rotational X-ray angiography (3DRA). The method assumes the position of the guide wire in the fluoroscopic images is known. A two-dimensional feature image is determined from the 3DRA data. In this feature image, the guide wire position is determined in a two-step approach: a mincost algorithm is used to determine a suitable position for the guide wire, and subsequently a snake optimization technique is applied to move the guide wire to a better position. The resulting guide wire can then be visualized in 3-D in combination with the 3DRA dataset. The reconstruction accuracy of the method has been evaluated using a 3DRA image of a vascular phantom filled with contrast, and monoplane fluoroscopic images of the same phantom without contrast and with a guide wire inserted. The evaluation has been performed for different projection angles, and with different parameters for the method. The final result does not appear to be very sensitive to the parameters of the method. The average mean error of the estimated 3-D guide wire position is 1.5 mm, and the average tip distance is 2.3 mm. The effect of inaccurate C-arm geometry information is also investigated. Small errors in geometry information (up to 1 degrees) will slightly decrease the 3-D reconstruction accuracies, with an error of at most 1 mm. The feasibility of this approach on clinical data is demonstrated.     

基于跟踪微分器的波门跟踪算法的研究

针对目标自动跟踪系统对目标跟踪算法实时性和精确性要求高的特点,提出了一种基于跟踪微分滤波器的运动目标跟踪方法.该算法利用跟踪微分器预测目标中心在下一帧体现在图像中可能出现的位置,以该位置为波门目标检测的中心,减少了目标的搜索范围,缩短了算法的运算时间.仿真结果表明,该方法具有不依赖于目标运动模型,实时性好,精确性高等优...     

Clinical evaluation of respiratory motion compensation for anatomical roadmap guided cardiac electrophysiology procedures

X-ray fluoroscopically guided cardiac electrophysiological procedures are routinely carried out for diagnosis and treatment of cardiac arrhythmias. X-ray images have poor soft tissue contrast and, for this reason, overlay of static 3-D roadmaps derived from preprocedural volumetric data can be used to add anatomical information. However, the registration between the 3-D roadmap and the 2-D X-ray image can be compromised by patient respiratory motion. Three methods were designed and evaluated to correct for respiratory motion using features in the 2-D X-ray images. The first method is based on tracking either the diaphragm or the heart border using the image intensity in a region of interest. The second method detects the tracheal bifurcation using the generalized Hough transform and a 3-D model derived from 3-D preoperative volumetric data. The third method is based on tracking the coronary sinus (CS) catheter. This method uses blob detection to find all possible catheter electrodes in the X-ray image. A cost function is applied to select one CS catheter from all catheter-like objects. All three methods were applied to X-ray images from 18 patients undergoing radiofrequency ablation for the treatment of atrial fibrillation. The 2-D target registration errors (TRE) at the pulmonary veins were calculated to validate the methods. A TRE of 1.6 mm ± 0.8 mm was achieved for the diaphragm tracking; 1.7 mm ± 0.9 mm for heart border tracking, 1.9 mm ± 1.0 mm for trachea tracking, and 1.8 mm ± 0.9 mm for CS catheter tracking. We present a comprehensive comparison between the techniques in terms of robustness, as computed by tracking errors, and accuracy, as computed by TRE using two independent approaches.     

视线跟踪系统中眼睛睁闭检测算法研究

眼睛睁闭检测在视线跟踪系统中具有重要意义.为提高检测的准确性,提出了一种基于Harris算子的检测方法.该方法首先利用Harris算子计算图像的角点量,然后搜索图像中角点量最大的位置,以该位置为中心,设置一个区域,统计该区域内角点量总和占整幅图像角点量总和的比例,通过将该比例与阈值相比较来确定眼睛的状态.实验结果表明,...