Reality Augmentation for Medical Procedures: System Architecture, Single Camera Marker Tracking, and System Evaluation

Augmented Reality is an emerging technology that seeks to enhance a user’s view by overlaying graphical information. We developed a prototype AR system geared for medical applications. It is built around a stereoscopic head-mounted display of the video-see-through variety. The newest generation of this prototype system exhibits high performance on a standard PC platform. Stereoscopic video images are augmented with medical graphics in real-time at 30 frames per second and with XGA (1024× 768) resolution. The system provides a compelling AR perception: the graphics appears firmly anchored in the scene—there is no time lag between video and graphics or any apparent jitter of the graphics. With the head-mounted display, the user has a natural and direct access to understanding the 3D structure of the scene, based on both stereo and kinetic depth cues. In the present paper, we describe in detail the architecture and several features of the AR prototype system. Head tracking is accomplished with a single-camera system, with the dedicated tracker camera placed on the head-mounted display. This configuration is the foundation of achieving a high-accuracy graphics overlay. We are now exploring the use of the prototype system for a variety of medical applications. This paper gives an overview over the pre-clinical tests that we have performed for interventional guidance. Overall, the feedback has been very positive and encouraging, and we are continuing to work towards realizing the clinical potential of the technology.