Volumetric virtual environments

Driven by fast development of both virtual reality and volume visualization,we discuss some critical techniques towards building a volumetric VR system,specifically the modeling,rendering,and manipulations of a volumetric scene.Techniques such as voxel-based object simplification,accelerated volume rendering,fast stereo volume rendering,and volumetric “collision detection“ are introduced and improved,with the idea of demonstrating the possibilities and potential benefits of incorporating volumetric models into VR systems.     

虚拟内窥镜的路径规划算法研究*

路径规划算法是虚拟内窥镜系统的关键技术之一。论文以分类的形式对虚拟内窥镜的路径规划算法进行了综述性研究。简要介绍了路径规划算法的研究内容及分类,重点分析了中心路径规划算法的性能和特点,并讨论了相关的改进算法。通过对各种路径规划算法的性能比较和特点总结,概括出虚拟内窥镜中,路径规划算法的研究重点和思路。     

虚拟内窥镜系统的关键技术分析

概述了虚拟内窥镜的基本原理、优点及发展现状;针对目前虚拟内窥镜系统存在的问题,结合各种处理算法,以其实现过程为主线,具体分析了虚拟内窥镜系统的几个关键性技术,重点讨论了路径规划、实时绘制及导航的研究内容;最后对虚拟内窥镜的研究特点和趋势作了总结。     

CT辅助消化道胶囊内窥镜图像虚拟外翻

CT辅助消化道胶囊内窥镜图像方法的主要目的在于更为直观地提高胶囊内窥镜的小病灶检出率,减少胶囊内窥镜的重复检查次数,减少患者检查费用,提高医生的工作效率.现有的虚拟外翻技术主要有两种,一种是展平法,一种是镜像法.CT辅助消化道胶囊内窥镜虚拟外翻是以镜像法为指导思想对三维肠道模型进行外翻,实验结果证明了这一方法的有效性.三维图像的显示采用了经典的移动立方体的算法.验证完成后,算法通过TI DaVinciDSP异构双核处理器(DM6467)和接口进行加速实现.通过合理分配双核的任务,算法的运算效率得到了很大提高.     

Sinus endoscopy--application of advanced GPU volume rendering for virtual endoscopy

For difficult cases in endoscopic sinus surgery, a careful planning of the intervention is necessary. Due to the reduced field of view during the intervention, the surgeons have less information about the surrounding structures in the working area compared to open surgery. Virtual endoscopy enables the visualization of the operating field and additional information, such as risk structures (e.g., optical nerve and skull base) and target structures to be removed (e.g., mucosal swelling). The Sinus Endoscopy system provides the functional range of a virtual endoscopic system with special focus on a realistic representation. Furthermore, by using direct volume rendering, we avoid time-consuming segmentation steps for the use of individual patient datasets. However, the image quality of the endoscopic view can be adjusted in a way that a standard computer with a modern standard graphics card achieves interactive frame rates with low CPU utilization. Thereby, characteristics of the endoscopic view are systematically used for the optimization of the volume rendering speed. The system design was based on a careful analysis of the endoscopic sinus surgery and the resulting needs for computer support. As a small standalone application it can be instantly used for surgical planning and patient education. First results of a clinical evaluation with ENT surgeons were employed to fine-tune the user interface, in particular to reduce the number of controls by using appropriate default values wherever possible. The system was used for preoperative planning in 102 cases, provides useful information for intervention planning (e.g., anatomic variations of the Rec. Frontalis), and closely resembles the intraoperative situation.     

Reliable path for virtual endoscopy: ensuring complete examinationof human organs

Virtual endoscopy is a computerized, noninvasive procedure for detecting anomalies inside human organs. Several preliminary studies have demonstrated the benefits and effectiveness of this modality. Unfortunately, previous work cannot guarantee that an existing anomaly will be detected, especially for complex organs with multiple branches. In this paper, we introduce the concept of reliable navigation, which ensures the interior organ surface is fully examined by the physician performing the virtual endoscopy procedure. To achieve this, we propose computing a reliable fly-through path that ensures no blind areas during the navigation. Theoretically, we discuss the criteria of evaluating a reliable path and prove that the problem of generating an optimal reliable path for virtual endoscopy is NP-complete. In practice, we develop an efficient method for the calculation of an effective reliable path. First, a small set of center observation points are automatically located inside the hollow organ. For each observation point, there exists at least one patch of interior surface visible to it, but that cannot be seen from any of the other observation points. These chosen points are then linked with a path that stays in the center of the organ. Finally, new points inside the organ are recursively selected and connected into the path until the entire organ surface is visible from the path. We present encouraging results from experiments on several data sets. For a medium-size volumetric model with several hundred thousand inner voxels, an effective reliable path can be generated in several minutes     

Efficient unfolding of virtual endoscopy using linear ray interpolation

Unfold rendering enables us to understand the internal structure of organs more precisely. The most commonly used unfolding method is to cast rays radially along a pre-computed camera path. However, this may produce images of false pathological structures. Details on the organ surface may be lost when the organ's curvature is relatively large due to adjacent ray planes crossing. Several methods are presented to avoid the penetration of ray planes. These methods result in severe computational overhead. We propose an efficient method of penetration-free ray casting for unfolding. It computes ray planes that do not intersect neighboring planes, using the noble intersection avoidance method. It determines the direction of all the rays fired from intermediate sampling points, by linearly interpolating the direction of the rays on two of the original ray planes. Experimental results show that it does not produce erroneous images since it exploits simple linear interpolation.     

A quantitative analysis of the effectiveness of laparascopy and endoscopy virtual reality simulators

The increasing use of virtual reality (VR) simulators in surgical training makes it imperative that definitive studies be performed to assess their training effectiveness. Indeed in this paper we report the meta-analysis of the efficacy of virtual reality simulators in (1) the transference of skills from the simulator training environment to the operating room and (2) their ability to discriminate between the experience levels of its users. The task completion time and the error score were the two study outcomes collated and analyzed in this meta-analysis. Sixteen studies were identified from a computer-based literature search (1996-2004). The meta-analysis of the random-effects model (because of the heterogeneity of the data) revealed that training on virtual reality simulators did lessen the time taken to complete a given surgical task as also clearly differentiate between the experienced and the novice trainees. Meta-analytic studies such as the one reported here would be very helpful in the planning and setting up of surgical training programs and for the establishment of reference ‘learning curves’ for a specific simulator and surgical task. If any such programs already exist they can then indicate the improvements to be made in the simulator used such as providing for more variety in their case scenarios based on the state and/or rate of learning of the trainee.     

Volumetric Features for Video Event Detection

Real-world actions occur often in crowded, dynamic environments. This poses a difficult challenge for current approaches to video event detection because it is difficult to segment the actor from the background due to distracting motion from other objects in the scene. We propose a technique for event recognition in crowded videos that reliably identifies actions in the presence of partial occlusion and background clutter. Our approach is based on three key ideas: (1) we efficiently match the volumetric representation of an event against oversegmented spatio-temporal video volumes; (2) we augment our shape-based features using flow; (3) rather than treating an event template as an atomic entity, we separately match by parts (both in space and time), enabling robustness against occlusions and actor variability. Our experiments on human actions, such as picking up a dropped object or waving in a crowd show reliable detection with few false positives.     

Volumetric ambient occlusion for volumetric models

This paper presents new algorithms to compute ambient occlusion for volumetric data. Ambient occlusion is used in video-games and film animation to mimic the indirect lighting of global illumination. We extend a novel interpretation of ambient occlusion to volumetric models that measures how big portion of the tangent sphere of a surface belongs to the set of occluded points, and propose statistically robust estimates for the ambient occlusion value. The data needed by this estimate can be obtained by separable filtering of the voxel array. As ambient occlusion is meant to obtain global illumination effects, it can provide decisive clues in interpreting the data. The new algorithms obtain smooth shading and can be computed at interactive rates, being thus appropriate for dynamic models exploration.     

Volumetric video compression for interactive playback

We develop a volumetric video system which supports interactive browsing of compressed time-varying volumetric features (significant isosurfaces and interval volumes). Since the size of even one volumetric frame in a time-varying 3D data set is very large, transmission and on-line reconstruction are the main bottlenecks for interactive remote visualization of time-varying volume and surface data. We describe a compression scheme for encoding time-varying volumetric features in a unified way, which allows for on-line reconstruction and rendering. To increase the run-time decompression speed and compression ratio, we decompose the volume into small blocks and encode only the significant blocks that contribute to the isosurfaces and interval volumes. The results show that our compression scheme achieves high compression ratio with fast reconstruction, which is effective for interactive client-side rendering of time-varying volumetric features.     

A PC-based high-quality and interactive virtual endoscopy navigating system using 3D texture based volume rendering

As an alternative method to optical endoscopy, visual quality and interactivity are crucial for virtual endoscopy. One solution is to use the 3D texture map based volume rendering method that offers high rendering speed without reducing visual quality. However, it is difficult to apply the method to virtual endoscopy. First, 3D texture mapping requires a high-end graphic workstation. Second, texture memory limits reduce the frame-rate. Third, lack of shading reduces visual quality significantly. As 3D texture mapping has become available on personal computers recently, we developed an interactive navigation system using 3D texture mapping on a personal computer. We divided the volume data into small cubes and tested whether the cubes had meaningful data. Only the cubes that passed the test were loaded into the texture memory and rendered. With the amount of data to be rendered minimized, rendering speed increased remarkably. We also improved visual quality by implementing full Phong shading based on the iso-surface shading method without sacrificing interactivity. With the developed navigation system, 256 x 256 x 256 sized brain MRA data was interactively explored with good image quality.     

Intelligent virtual environments for virtual reality art

The development of virtual reality (VR) art installations is faced with considerable difficulties, especially when one wishes to explore complex notions related to user interaction. We describe the development of a VR platform, which supports the development of such installations, from an art+science perspective. The system is based on a CAVE™-like immersive display using a game engine to support visualisation and interaction, which has been adapted for stereoscopic visualisation and real-time tracking. In addition, some architectural elements of game engines, such as their reliance on event-based systems have been used to support the principled definition of alternative laws of Physics. We illustrate this research through the development of a fully implemented artistic brief that explores the notion of causality in a virtual environment. After describing the hardware architecture supporting immersive visualisation we show how causality can be redefined using artificial intelligence technologies inspired from action representation in planning and how this symbolic definition of behaviour can support new forms of user experience in VR.     

Methods for Volumetric Reconstruction of Visual Scenes

In this paper, we present methods for 3D volumetric reconstruction of visual scenes photographed by multiple calibrated cameras placed at arbitrary viewpoints. Our goal is to generate a 3D model that can be rendered to synthesize new photo-realistic views of the scene. We improve upon existing voxel coloring/space carving approaches by introducing new ways to compute visibility and photo-consistency, as well as model infinitely large scenes. In particular, we describe a visibility approach that uses all possible color information from the photographs during reconstruction, photo-consistency measures that are more robust and/or require less manual intervention, and a volumetric warping method for application of these reconstruction methods to large-scale scenes.     

Unsupervised Image Reconstruction for Gradient-Domain Volumetric Rendering

Gradient-domain rendering can highly improve the convergence of light transport simulation using the smoothness in image space. These methods generate image gradients and solve an image reconstruction problem with rendered image and the gradient images. Recently, a previous work proposed a gradient-domain volumetric photon density estimation for homogeneous participating media. However, the image reconstruction relies on traditional L1 reconstruction, which leads to obvious artifacts when only a few rendering passes are performed. Deep learning based reconstruction methods have been exploited for surface rendering, but they are not suitable for volume density estimation. In this paper, we propose an unsupervised neural network for image reconstruction of gradient-domain volumetric photon density estimation, more specifically for volumetric photon mapping, using a variant of GradNet with an encoded shift connection and a separated auxiliary feature branch, which includes volume based auxiliary features such as transmittance and photon density. Our network smooths the images on global scale and preserves the high frequency details on a small scale. We demonstrate that our network produces a higher quality result, compared to previous work. Although we only considered volumetric photon mapping, it's straightforward to extend our method for other forms, like beam radiance estimation.     

Volumetric Vector-Based Representation for Indirect Illumination Caching

This paper introduces a caching technique based on a volumetric representation that captures low-frequency indirect illumination. This structure is intended for efficient storage and manipulation of illumination. It is based on a 3D grid that stores a fixed set of irradiance vectors. During preprocessing, this representation can be built using almost any existing global illumination software. During rendering, the indirect illumination within a voxel is interpolated from its associated irradiance vectors, and is used as additional local light sources. Compared with other techniques, the 3D vector-based representation of our technique offers increased robustness against local geometric variations of a scene. We thus demonstrate that it may be employed as an efficient and high-quality caching data structure for bidirectional rendering techniques such as particle tracing or photon mapping.     

局部特征加强的体绘制算法

最大强度差值累积结合了直接体绘制和最大强度值投影的优势,但其在累积过程中会遗漏一些局部特征.为了绘制体数据中局部特征信息,提出一种局部特征加强的直接体绘制方法.通过查找采样光线上特征边界点来确定局部最大强度的区域,利用局部差值累积的方法加强特征区域的绘制;为了提高特征分界点的查找精度,引入移动最小二乘法来平滑采样光线上的标量值,并利用用户自定义的阈值函数来控制特征的绘制;在绘制过程中,采用特征分析的表面光照模型增强绘制特征的三维立体感,引入深度信息对局部特征累积算法进行优化,并引入了Tone衰减方法使累积颜色值处于正常显示范围.实验结果表明,文中方法可在不需要传输函数的前提下绘制体数据中的特征信息.     

Realization of reconfigurable virtual environments for virtual testing

1 Introduction The scalability of virtual environments (VEs) that allow users to change and modify their contents and context depending on application requirements has been a de facto feature of virtual reality technology. Engineering uses such as virtual testing and exper- imenting applications demand VEs to be highly or fully reconfigurable. In reality, however, satisfactory reconfiguration of a virtual environment is yet to be achieved, due to the high accuracy of spatial data in- put and…     

三维空间曲面消隐的并行算法

并行与分布式图形处理已成为超级计算环境下科学可视化的基础。本文描述我们在银河巨型机上针对一类三维空间曲面设计的并行消隐算法。