State‐of‐the‐art in orthopaedic surgical navigation with a focus on medical image modalities

This paper presents a review of surgical navigation systems in orthopaedics and categorizes these systems according to the image modalities that are used for the visualization of surgical action. Medical images used to be an essential part of surgical education and documentation as well as diagnosis and operation planning over many years. With the recent introduction of navigation techniques in orthopaedic surgery, a new field of application has been opened. Today surgical navigation systems — also known as image‐guided surgery systems — are available for various applications in orthopaedic surgery. They visualize the position and orientation of surgical instruments as graphical overlays onto a medical image of the operated anatomy on a computer monitor. Preoperative image data such as computed tomography scans or intraoperatively generated images (for example, ultrasonic, endoscopic or fluoroscopic images) are suitable for this purpose. A new category of medical images termed ‘surgeon‐defined anatomy’ has been developed that exclusively relies upon the usage of navigation technology. Points on the anatomy are digitized interactively by the surgeon and are used to build up an abstract geometrical model of the bony structures to be operated on. This technique may be used when no other image data is available or appropriate for a given application. Copyright © 2002 John Wiley & Sons, Ltd.     

Exploratory spatio-temporal data mining and visualization

Spatio-temporal data sets are often very large and difficult to analyze and display. Since they are fundamental for decision support in many application contexts, recently a lot of interest has arisen toward data-mining techniques to filter out relevant subsets of very large data repositories as well as visualization tools to effectively display the results. In this paper we propose a data-mining system to deal with very large spatio-temporal data sets. Within this system, new techniques have been developed to efficiently support the data-mining process, address the spatial and temporal dimensions of the data set, and visualize and interpret results. In particular, two complementary 3D visualization environments have been implemented. One exploits Google Earth to display the mining outcomes combined with a map and other geographical layers, while the other is a Java3D-based tool for providing advanced interactions with the data set in a non-geo-referenced space, such as displaying association rules and variable distributions.     

A novel augmented reality visualization in jaw surgery: enhanced ICP based modified rotation invariant and modified correntropy

Image registration, accuracy, processing time and occlusions are the main limitations of augmented reality (AR) based jaw surgery. Therefore, the main aim of this paper is to reduce the registration error, which will help in improving the accuracy and reducing the processing time. Also, it aims to remove outliers and remove the registration outcomes trapped in local minima to improve the alignment problems and remove the occlusion caused by surgery instrument. The enhanced Iterative Closest Point (ICP) algorithm with rotation invariant and correntropy was used for the proposed system. Markerless image registration technique was used for AR-based jaw surgery. The problem of occlusion caused by surgical tools and blood is solved by using stereo based tracing with occlusion handling techniques. This research reduced alignment error 0.59 mm?~?0.62 mm against 0.69?~?0.72 mm of state-of-the-art solution. The processing time of video frames was enhanced to 11.9?~?12.8 fps against 8?~?9.15 fps in state-of-the-art solution. This paper is focused on providing fast and accurate AR-based system for jaw surgery. The proposed system helps in improving the AR visualization during jaw surgery. The combination of methods and technology helped in improving AR visualization for jaw surgery and to overcome the failure caused by a large rotation angle and provides an initial parameter for better image registration. It also enhances performance by removing outliers and noises. The pose refinement stage provides a better result in terms of processing time and accuracy.

     

An optical navigator for brain surgery

Computer-assisted surgery (CAS) helps surgeons find their bearings during operations under difficult visual conditions. These include brain surgery and other head operations. CAS combines coordinate measurements of the operating instruments and superimposes their positions on to images of the operation area so that surgeons can follow instrument movements on a computer screen. Since 1986, the Institute for Measurement Techniques (Lehrstuhl fu¨r Meßtechnik) has been working in close cooperation with the Aachen University Hospital to develop CAS systems. This project has successfully treated numerous patients in the areas of neurosurgery, otorhinolaryngology (ear, nose and throat), ophthalmology, radiotherapy and neurology. Based on this work, Philips Medical Systems has developed a commercial system for neurosurgery. This orientation device to aid physician accuracy in brain surgery has been developed by combining 3D position measurement techniques, digital image processing and 3D display techniques. This article provides an overview of the development process and an outlook on possible applications     

A transparently scalable visualization architecture for exploring the universe

Modern astronomical instruments produce enormous amounts of three-dimensional data describing the physical Universe. The currently available data sets range from the solar system to nearby stars and portions of the Milky Way Galaxy, including the interstellar medium and some extrasolar planets, and extend out to include galaxies billions of light years away. Because of its gigantic scale and the fact that it is dominated by empty space, modeling and rendering the Universe is very different from modeling and rendering ordinary three-dimensional virtual worlds at human scales. Our purpose is to introduce a comprehensive approach to an architecture solving this visualization problem that encompasses the entire Universe while seeking to be as scale-neutral as possible. One key element is the representation of model-rendering procedures using power scaled coordinates (PSC), along with various PSC-based techniques that we have devised to generalize and optimize the conventional graphics framework to the scale domains of astronomical visualization. Employing this architecture, we have developed an assortment of scale-independent modeling and rendering methods for a large variety of astronomical models, and have demonstrated scale-insensitive interactive visualizations of the physical Universe covering scales ranging from human scale to the Earth, to the solar system, to the Milky Way Galaxy, and to the entire observable Universe     

具有力反馈的虚拟膝部手术系统

基于计算机的外科手术仿真系统已被广泛用于医疗教育、手术训练、手术计划,还被作为术中的辅助手段。随着具有力反馈设备在虚拟膝部手术中的应用,拓展了体数据可视化技术在医学实践中的应用范围,使医务人员不但能看到而且可以触摸并可以使用各种虚拟手术器械对三维虚拟人体膝部进行切割,修补,嫁接等操作。该文介绍了虚拟膝部手术系统的组成部分,主要包括：图像数据的获取和分割,三维模型的重建及人机交互操作,并详细说明每一部分的技术要点及其所采用的疗法。     

基于电磁定位的手术导航探针可视化与实时跟踪技术

针对手术导航探针可视化与实时跟踪技术进行了研究.文中对电磁定位方法进行了介绍,设计了电磁定位下具有数据采集功能的探针.首先,应用3维实体造型技术构建探针的模型,完成实体几何模型向3维表面模型的转化.其次,使用旋转.沣册方法对针尖进行注册,并进行了精度测试.然后,应用最近点迭代(ICP)算法实现CT模型与患者位置的术中注...     

An integrated range-sensing, segmentation and registration framework for the characterization of intra-surgical brain deformations in image-guided surgery

Image-guided surgery (IGS) is a technique for localizing anatomical structures on the basis of volumetric image data and for determining the optimal surgical path to reach these structures, by the means of a localization device, or probe, whose position is tracked over time. The usefulness of this technology hinges on the accuracy of the transformation between the image volume and the space occupied by the patient anatomy and spanned by the probe. Unfortunately, in neurosurgery this transformation can be degraded by intra-surgical brain shift, which often measures more than 10 mm and can exceed 25 mm. We propose a method for characterizing brain shift that is based on non-rigid surface registration, and can be combined with a constitutively realistic finite element approach for volumetric displacement estimation. The proposed registration method integrates in a unified framework all of the stages required to estimate the movement of the cortical surface in the operating room: model-based segmentation of the pre-operative brain surface in magnetic resonance image data, range-sensing of the cortex in the OR, range–MR rigid transformation computation, and range-based non-rigid brain motion estimation. The brain segmentation technique is an adaptation of the surface evolution model. Its convergence to the brain boundary is the result of a speed term restricted to white and grey matter voxels made explicit by a classifier, and the final result is post-processed to yield a Closest Point Map of the brain surface in MR space. In turn, this Closest Point Map is used to produce the homologous pairs required to determine a highly efficient, 2D spline-based, Iterative Closest Point (ICP) non-rigid surface registration. The baseline for computing intra-operative brain displacement, as well as the initial starting point of the non-rigid ICP registration, is determined by a very good rigid range–MR transformation, produced by a simple procedure for relating the range coordinate system to that of the probe, and ultimately to that of the MR volume.     

Automatic Registration of Multi‐Projector Domes Using a Single Uncalibrated Camera

In this paper we present a novel technique for easily calibrating multiple casually aligned projectors on spherical domes using a single uncalibrated camera. Using the prior knowledge of the display surface being a dome, we can estimate the camera intrinsic and extrinsic parameters and the projector to display surface correspondences automatically using a set of images. These images include the image of the dome itself and a projected pattern from each projector. Using these correspondences we can register images from the multiple projectors on the dome. Further, we can register displays which are not entirely visible in a single camera view using multiple pan and tilted views of an uncalibrated camera making our method suitable for displays of different size and resolution. We can register images from any arbitrary viewpoint making it appropriate for a single head‐tracked user in a 3D visualization system. Also, we can use several cartographic mapping techniques to register images in a manner that is appropriate for multi‐user visualization. Domes are known to produce a tremendous sense of immersion and presence in visualization systems. Yet, till date, there exists no easy way to register multiple projectors on a dome to create a high‐resolution realistic visualizations. To the best of our knowledge, this is the first method that can achieve accurate geometric registration of multiple projectors on a dome simply and automatically using a single uncalibrated camera.     

色彩渐进插值的矿井预警数据集三维可视化算法

针对矿井预警数据信息表达不完全、基于视觉的统计分析工作繁重、预警数据集庞杂等问题,提出了一种基于色彩渐进插值的矿井预警数据集三维可视化算法。在该算法中,首先根据矿井预警数据集的测点位置和测量值信息进行三维空间模型构造;然后根据灰度级与彩色空间系统的映射关系对矿井预警数据集与彩色空间模型进行颜色映射及三维空间层次分割,对每个层片依据伪图像编码算法及颜色聚类参数特征进行矿井预警数据集的三维可视化伪图像编码;最后根据色彩渐进插值算法对伪图像中相邻层片进行平滑过渡处理。实验证明,该算法处理的矿井预警数据集伪图像色彩渲染层次感强,色彩过渡平滑,有利于矿井预警数据集的信息表达。     

3D navigation and monitoring for spinal milling operation based on registration between multiplanar fluoroscopy and CT images

Milling operations in spinal surgery demand much experience and skill for the surgeon to perform the procedure safely. A 3D navigation method is introduced aiming at providing a monitoring system with enhanced safety and minimal intraoperative interaction. An automatic registration method is presented to establish the 3D-3D transformation between the preoperative CT images and a common reference system in the surgical space, and an intensity-based similarity metric adapted for the multi-planar configuration is introduced in the registration procedure. A critical region is defined for real-time monitoring in order to prevent penetration of the lamina and avoid violation of nerve structures. The contour of the spinal canal is reconstructed as the critical region, and different levels of warning limits are defined. During the milling procedure, the position of the surgical instrument relative to the critical region is provided with augmented display and audio warnings. Timely alarm is provided for surgeons to prevent surgical failure when the mill approaches the critical region. Our validation experiment shows that real-time 3D navigation and monitoring is advantageous for improving the safety of the milling operation.     

Multimodal image registration system for image-guided orthopaedic surgery

We present a novel multimodality image registration system for spinal surgery. The system comprises a surface-based algorithm that performs computed tomography/magnetic resonance (CT/MR) rigid registration and MR image segmentation in an iterative manner. The segmentation/registration process progressively refines the result of MR image segmentation and CT/MR registration. For MR image segmentation, we propose a method based on the double-front level set that avoids boundary leakages, prevents interference from other objects in the image, and reduces computational time by constraining the search space. In order to reduce the registration error from the misclassification of the soft tissue surrounding the bone in MR images, we propose a weighted surface-based CT/MR registration scheme. The resultant weighted surface is registered to the segmented surface of the CT image. Contours are generated from the reconstructed CT surfaces for subsequent MR image segmentation. This process iterates till convergence. The registration method achieves accuracy comparable to conventional techniques while being significantly faster. Experimental results demonstrate the advantages of the proposed approach and its application to different anatomies.     

Reconstruction of medical images under different image acquisition angles

Compared to object-based registration, feature-based registration is much less complex. However, in order for feature-based registration to work, the two image stacks under consideration must have the same acquisition tilt angle and the same anatomical location - two requirements that are not always fulfilled. In this paper, we propose a technique that reconstructs two sets of medical images acquired with different acquisition angles and anatomical cross sections into one set of images of identical scanning orientation and positions. The space correlation information among the two image stacks is first extracted and is used to correct the tilt angle and anatomical position differences in the image stacks. Satisfactory reconstruction results were presented to prove our points.     

A virtual cockpit for a distributed interactive simulation

We developed the virtual cockpit as an inexpensive flight simulator, using off-the-shelf equipment. This system functions as a component of a distributed interactive simulation. Any flight simulator has three principal tasks: image display, image generation, and flight dynamics. We built the flight simulator, the virtual cockpit, using a head-mounted display (HMD) to display the out-the-window imagery and the cockpit interior. The virtual cockpit consists of a Silicon Graphics workstation and supporting hardware components, such as an HMD and position tracker. The software falls into three areas by function: flight dynamics and cockpit instruments; network interface (either SimNet or DIS PDUs); and display of the out-the-window view. The virtual cockpit developed uses a multiprocessor Silicon Graphics Iris (the 4D/440VGXT) connected to a Polhemus Laboratories fiber-optic head-mounted display, a Polhemus magnetic head tracker and a hands-on-throttle-and-stick (HOTAS) by Thrustmaster. The software uses the AT and T C++ translator and the Silicon Graphics C compiler running under Irix 4.0.5. a Unix operating system. We use Software Systems' MultiGen to create the geometric models and the terrain database     

Effective visualization of complex vascular structures using a non-parametric vessel detection method

The effective visualization of vascular structures is critical for diagnosis, surgical planning as well as treatment evaluation. In recent work, we have developed an algorithm for vessel detection that examines the intensity profile around each voxel in an angiographic image and determines the likelihood that any given voxel belongs to a vessel; we term this the "vesselness coefficient" of the voxel. Our results show that our algorithm works particularly well for visualizing branch points in vessels. Compared to standard Hessian based techniques, which are fine-tuned to identify long cylindrical structures, our technique identifies branches and connections with other vessels. Using our computed vesselness coefficient, we explore a set of techniques for visualizing vasculature. Visualizing vessels is particularly challenging because not only is their position in space important for clinicians but it is also important to be able to resolve their spatial relationship. We applied visualization techniques that provide shape cues as well as depth cues to allow the viewer to differentiate between vessels that are closer from those that are farther. We use our computed vesselness coefficient to effectively visualize vasculature in both clinical neurovascular x-ray computed tomography based angiography images, as well as images from three different animal studies. We conducted a formal user evaluation of our visualization techniques with the help of radiologists, surgeons, and other expert users. Results indicate that experts preferred distance color blending and tone shading for conveying depth over standard visualization techniques.     

Autostereoscopic augmented reality visualization for depth perception in endoscopic surgery

Augmented reality (AR) has received increasing attention in minimally invasive surgery (MIS) applications. The goal of applying AR techniques to MIS is to enhance a surgeon's perception of the spatial relationship by overlaying invisible structures (e.g. tumor or vessels) onto the in vivo endoscopic video acquired during the surgery. One of primary issues of AR visualization is to provide correct depth perception for visible and invisible structures. In this paper, we present a video-based AR system consisting of functional modules for real-time 3D surface capture, reconstruction, and registration with pre-operative segmented CT model. The real-time 3D registration allows precise overlay of invisible structures onto 2D video for AR visualization. The AR overlay result is displayed on a multi-view autostereoscopic lenticular LCD. To study and compare the efficacy of AR visualization techniques, we investigated five different AR visualization modes. Both simulated and in vivo experiments were carried out and autostereoscopic AR visualization results were given. Evaluation and comparison for depth perception between five AR visualization modes are presented. Finally, we conclude the characteristics of these visualization modes. The novelty of our work lies in successful implementation of an end-to-end 3D autostereoscopic AR system from real-time reconstruction and registration with our multi-channel 3D endoscope, and systematic evaluation and comparison of five different visualization modes for depth perception.     

Image‐guided robotic navigation system for neurosurgery

This paper presents an image‐guided robotic navigation system for neurosurgery, which can be applied to the electro‐stimulation treatment of Parkinson's leisure, the biopsy of deep tumors, and haematoma evacuation. The system integrates a computer containing CT images for surgical planning, a magnetic tracking device for measuring the coordinates of the markers and surgical instruments, and a robot manipulator for guiding surgical instruments to the preplanned position and orientation. The computer display of brain anatomy offers a convenient tool for surgeons to diagnose brain diseases and to plan safe surgical paths, while the tracking device guides the robot manipulator to automatically move surgical instruments to the preplanned position and orientation. An experiment of using a skull model for simulating a robotic biopsy of brain tumor has been done to verify the performance of the robotic navigation system. The results show that the positioning accuracy of the robot relative to the tracker frame is only related to the positioning resolution of the robot manipulator and the positioning accuracy of the tracking device. In other words, the positioning accuracy of the robot manipulator does not affect the final positioning accuracy of the surgical instruments. Therefore, using a robot manipulator for precise surgical navigation is feasible and reliable. © 2000 John Wiley & Sons, Inc.     

Image display in endoscopic surgery

Abstract— Advances in the technology of optical displays have changed the way surgeons are able to manage different illnesses. Minimally invasive surgery encompasses a wide range of endoscopic procedures, whereby the body cavity (abdomen, thorax, gastrointestinal tract, and joint spaces) is accessed through small incisions and the use of telescopes and fine, long instruments. These techniques have rapidly gained in popularity during the last decades, as patients are experiencing less discomfort after surgery. Visualization of the operative field requires optimal image capture, processing, and display. The introduction of charge‐coupled devices has enabled surgeons to view the operative field on a video monitor, allowing ever‐more‐complex operations to be performed endoscopically. However, limitations include loss of 3‐D perception and tactile sense, poor ergonomics, often suboptimal positioning of image display and image quality that is too dependent on outside influences. These limitations, and possible solutions, are addressed, as is the “ideal” display system for endoscopic surgery.     

Uncertainty visualization using HDR volume rendering

In this paper, we explore a novel idea of using high dynamic range (HDR) technology for uncertainty visualization. We focus on scalar volumetric data sets where every data point is associated with scalar uncertainty. We design a transfer function that maps each data point to a color in HDR space. The luminance component of the color is exploited to capture uncertainty. We modify existing tone mapping techniques and suitably integrate them with volume ray casting to obtain a low dynamic range (LDR) image. The resulting image is displayed on a conventional 8-bits-per-channel display device. The usage of HDR mapping reveals fine details in uncertainty distribution and enables the users to interactively study the data in the context of corresponding uncertainty information. We demonstrate the utility of our method and evaluate the results using data sets from ocean modeling.