Image-driven virtual simulation of arthroscopy

In recent years, minimally invasive arthroscopic surgery has replaced a number of conventional open orthopedic surgery procedures on joints. While this achieves a number of advantages for the patient, the surgeons have to learn very different skills, since the surgery is performed with special miniature pencil-like tools and cameras inserted through little incisions while observing the surgical field on video monitor. Therefore, virtual reality simulation becomes an alternative to traditional surgical training based on hundreds years old apprentice–master model that involves either real patients or increasingly difficult to procure cadavers. Normally, 3D simulation of the virtual surgical field requires significant efforts from the software developers but yet remains not always photorealistic. In contrast to this, for photorealistic visualization and haptic interaction with the surgical field we propose to use real arthroscopic images augmented with 3D object models. The proposed technique allows for feeling the joint cavity displayed on video monitor as real 3D objects rather than their images while various surgical procedures, such as menisectomy, are simulated in real time. In the preprocessing stage of the proposed approach, the arthroscopic images are stitched into panoramas and augmented with implicitly defined object models representing deformable menisci. In the simulation loop, depth information from the mixed scene is used for haptic rendering. The scene depth map and visual display are reevaluated only when the scene is modified.     

Effects of vision and friction on haptic perception

OBJECTIVE: Two experiments were conducted to examine the effects of vision and masking friction on contact perception and compliance differentiation thresholds in a simulated tissue-probing task. BACKGROUND: In minimally invasive surgery, the surgeon receives limited haptic feedback because of the current design of the instrumentation and relies on visual feedback to judge the amount of force applied to the tissues. It is suggested that friction forces inherent in the instruments contribute to errors in surgeons' haptic perception. This paper investigated the psychophysics of contact detection and cross-modal sensory processing in the context of minimally invasive surgery. METHOD: A within-subjects repeated measures design was used, with friction, vision, tissue softness, and order of presentation as independent factors, and applied force, detection time, error, and confidence as dependent measures. Eight participants took part in each experiment, with data recorded by a custom force-sensing system. RESULTS: In both detection and differentiation tasks, higher thresholds, longer detection times, and more errors were observed when vision was not available. The effect was more pronounced when haptic feedback was masked by friction forces in the surgical device (p < .05). CONCLUSION: Visual and haptic feedback were equally important for tissue compliance differentiation. APPLICATION: A frictionless endoscopic instrument can be designed to restore critical haptic information to surgeons without having to create haptic feedback artificially.     

A Compact Modular Teleoperated Robotic System for Laparoscopic Surgery

Compared with traditional open surgery, minimally invasive surgical procedures reduce patient trauma and recovery time, but the dexterity of the surgeon in laparoscopic surgery is reduced owing to the small incisions, long instruments and limited indirect visibility of the operative site inside the patient. Robotic surgical systems, teleoperated by surgeons from a master control console with joystick-type manipulation interfaces, have been commercially developed yet their adoption into standard practice may be limited owing to their size, complexity, cost and time-consuming setup, maintenance and sterilization procedures. The goal of our research is to improve the effectiveness of robot-assisted surgery by developing much smaller, simpler, modular, teleoperated robotic manipulator systems for minimally invasive surgery.     

Advanced virtual endoscopic pituitary surgery

Endoscopy has recently been introduced to endonasal transsphenoidal pituitary surgery as a minimally invasive procedure for the removal of various kinds of pituitary tumors. To reduce morbidity and mortality with this new technique, the surgeon must be well-trained and well-prepared. Virtual endoscopy can be beneficial as a tool for training, preoperative planning, and intraoperative support. This paper introduces STEPS, a virtual endoscopy system designed to aid surgeons in getting acquainted with the endoscopic view of the anatomy, the handling of instruments, the transsphenoidal approach, and challenges associated with the procedure. STEPS also assists experienced surgeons in planning a real endoscopic intervention by getting familiar with the individual patient anatomy, identifying landmarks, planning the approach, and deciding upon the ideal target position of the actual surgical activity. The application provides interactive visualization, navigation, and perception aids and the possibility of simulating the procedure, including haptic feedback and simulation of surgical instruments.     

Computer-based virtual reality simulator for phacoemulsification cataract surgery training

Recent research in virtual reality indicates that computer-based simulators are an effective technology to use for surgeons learning to improve their surgical skills in a controlled environment. This article presents the development of a virtual reality simulator for phacoemulsification cataract surgery training, which is the most common surgical technique currently being used to remove cataracts from the patient’s eyes. The procedure requires emulsifying the cloudy natural lens of the eye and restoring vision by implanting an artificial lens through a small incision. The four main procedures of cataract surgery, namely corneal incision, capsulorhexis, phacoemulsification, and intraocular lens implantation, are incorporated in the simulator for virtual surgical training by implementing several surgical techniques. The surgical activity that are applied on the anatomy of the human eye, such as incision, grasping, tearing, emulsification, rotation, and implantation, are simulated in the system by using different types of mesh modifications. A virtual reality surgical simulator is developed, and the main procedures of phacoemulsification cataract surgery are successfully simulated in the system. The simulation results of the training system show that the developed simulator is capable of generating a virtual surgical environment with faithful force feedback for medical residents and trainees to conduct their training lessons via the computer using a pair of force-feedback haptic devices. In addition, the successful simulation of the mesh modifications on the human eyeball with visual realism and faithful force feedback throughout the surgical operation shows that the developed simulator is able to serve as a virtual surgical platform for surgeons to train their surgical skills.     

Understanding the Effect of Handedness on Both-Handed Task Performance: An Experimental Study based on a Haptic-Controlled,Simulation-Based Surgical Skill Training Scenario

Understanding the performance on both-handed tasks, such as endoscopic surgery, is critical to better organize and develop appropriate instructional systems to improve the necessary skills of surgeons. However, in the literature, only a limited number of studies have investigated the effect of handedness on both-handed task performance. This study aimed to provide an understanding of the participants’ performance differences while performing both-handed tasks through a haptic user interface in a simulated virtual environment specifically developed for surgical training purposes. Twenty-four surgeons attending a medical school in Turkey voluntarily participated in the study. The duration, accuracy, and collision measures were automatically recorded by software. The results revealed that the left-handed participants performed the both-handed tasks (camera: nondominant hand, tool: dominant-hand) in a significantly shorter time than the right-handed participants. This study also showed that haptic-controlled simulation-based surgical skill training systems can potentially provide measures for better understanding the individual behaviors and deliver alternative training environments specific to individual requirements.     

Image display in endoscopic surgery

Abstract— Advances in the technology of optical displays have changed the way surgeons are able to manage different illnesses. Minimally invasive surgery encompasses a wide range of endoscopic procedures, whereby the body cavity (abdomen, thorax, gastrointestinal tract, and joint spaces) is accessed through small incisions and the use of telescopes and fine, long instruments. These techniques have rapidly gained in popularity during the last decades, as patients are experiencing less discomfort after surgery. Visualization of the operative field requires optimal image capture, processing, and display. The introduction of charge‐coupled devices has enabled surgeons to view the operative field on a video monitor, allowing ever‐more‐complex operations to be performed endoscopically. However, limitations include loss of 3‐D perception and tactile sense, poor ergonomics, often suboptimal positioning of image display and image quality that is too dependent on outside influences. These limitations, and possible solutions, are addressed, as is the “ideal” display system for endoscopic surgery.     

Surgical robotics: Reviewing the past,analysing the present,imagining the future

This paper presents an overview of the surgical robotics field, highlighting significant milestones and grouping the various propositions into cohorts. The review does not aim to be exhaustive but rather to highlight how surgical robotics is acting as an enabling technology for minimally invasive surgery. As such, there is a focus on robotic surgical solutions which are commercially available; research efforts which have not gained regulatory approval or entered clinical use are mostly omitted. The practice of robotic surgery is currently largely dominated by the da Vinci system of Intuitive Surgical (Sunnyvale, CA, USA) but other commercial players have now entered the market with surgical robotic products or are appearing in the horizon with medium and long term propositions. Surgical robotics is currently a vibrant research topic and new research directions may lead to the development of very different robotic surgical devices in the future—small, special purpose, lower cost, possibly disposable robots rather than the current large, versatile and capital expensive systems. As the trend towards minimally invasive surgery (MIS) increases, surgery becomes more technically demanding for surgeons and more challenging for medical device technologists and it is clear that surgical robotics has now an established foothold in medicine as an enabling technology of MIS.     

Toward the improvement of image-guided interventions for minimally invasive surgery: three factors that affect performance

OBJECTIVES: The objectives were to measure the impact of specific features of imaging devices on tasks relevant to minimally invasive surgery (MIS) and to investigate cognitive and perceptual factors in such tasks. BACKGROUND: Although image-guided interventions used in MIS provide benefits for patients, they pose drawbacks for surgeons, including degraded depth perception and reduced field of view (FOV). It is important to identify design factors that affect performance. METHOD: In two navigation experiments, observers fed a borescope through an object until it reached a target. Task completion time and object shape judgments were measured. In a motion perception experiment, observers reported the direction of a line that moved behind an aperture. A motion illusion associated with reduced FOV was measured. RESULTS: Navigation through an object was faster when a preview of the object's exterior was provided. Judgments about the object's shape were more accurate with a preview (compared with none) and with active viewing (compared with passive viewing). The motion illusion decreased with a rectangular or rotating octagonal viewing aperture (compared with circular). CONCLUSIONS: Navigation performance may be enhanced when surgeons develop a mental model of the surgical environment, when surgeons (rather than assistants) control the camera, and when the shape of the image is designed to reduce visual illusions. APPLICATION: Unintentional contact between surgical tools and healthy tissues may be reduced during MIS when (a) visual aids permit surgeons to maintain a mental model of the surgical environment, (b) images are bound by noncircular apertures, and (c) surgeons manually control the camera.     

Guest Editors' Introduction: Simulators and Closed Interaction Loops

Simulation systems for medical training and treatment planning should create an immersive and realistic feeling similar to treating a real patient. They should provide realistic views, support haptic feedback, and enable instrument handling. In addition, they must simulate tissue deformations and/or tissue removal. Some simulators already exist for isolated, rather simple areas, such as endoscopic or minimally invasive procedures (laparoscopy, colonoscopy, bronchoscopy, and so on), however there is a large need for systems that simulate open surgery (example, on the liver or heart). Simulators will play a fundamental role in teaching, training, and quality assurance for the coming generations.     

Methods for haptic feedback in teleoperated robot-assisted surgery

Teleoperated minimally invasive surgical robots can significantly enhance a surgeon's accuracy, dexterity and visualization. However, current commercially available systems do not include significant haptic (force and tactile) feedback to the operator. This paper describes experiments to characterize this problem, as well as several methods to provide haptic feedback in order to improve surgeon's performance. There exist a variety of sensing and control methods that enable haptic feedback, although a number of practical considerations, e.g. cost, complexity and biocompatibility, present significant challenges. The ability of teleoperated robot-assisted surgical systems to measure and display haptic information leads to a number of additional exciting clinical and scientific opportunities, such as active operator assistance through "virtual fixtures" and the automatic acquisition of tissue properties.     

A haptic interface for computer-integrated endoscopic surgery and training

Haptic feedback has the potential to provide superior performance in computer-integrated surgery and training. This paper discusses the design of a user interface that is capable of providing force feedback in all the degrees of freedom (DOFs) available during endoscopic surgery. Using the Jacobian matrix of the haptic interface and its singular values, methods are proposed for analysis and optimization of the interface performance with regard to the accuracy of force feedback, the range of applicable forces, and the accuracy of control. The haptic user interface is used with a sensorized slave robot to form a master–slave test-bed for studying haptic interaction in a minimally invasive environment. Using the master–slave test-bed, teleoperation experiments involving a single degree of freedom surgical task (palpation) are conducted. Different bilateral control methods are compared based on the transparency of the master–slave system in terms of transmitting the critical task-related information to the user in the context of soft-tissue surgical applications.     

Design and implementation of a PC-based image-guided surgical system

In interactive, image-guided surgery, current physical space position in the operating room is displayed on various sets of medical images used for surgical navigation. We have developed a PC-based surgical guidance system (ORION) which synchronously displays surgical position on up to four image sets and updates them in real time. There are three essential components which must be developed for this system: (1) accurately tracked instruments; (2) accurate registration techniques to map physical space to image space; and (3) methods to display and update the image sets on a computer monitor. For each of these components, we have developed a set of dynamic link libraries in MS Visual C++ 6.0 supporting various hardware tools and software techniques. Surgical instruments are tracked in physical space using an active optical tracking system. Several of the different registration algorithms were developed with a library of robust math kernel functions, and the accuracy of all registration techniques was thoroughly investigated. Our display was developed using the Win32 API for windows management and tomographic visualization, a frame grabber for live video capture, and OpenGL for visualization of surface renderings. We have begun to use this current implementation of our system for several surgical procedures, including open and minimally invasive liver surgery.     

Investigating haptic distance-to-break using linear and nonlinear materials in a simulated minimally invasive surgery task

Accurate detection of mediated haptic information in minimally invasive surgery (MIS) is critical for applying appropriate force magnitudes onto soft tissue with the aim of minimising tissue trauma. Force perception in MIS is a dynamic process, with surgeons’ administration of force into tissue revealing information about the remote surgical site which further informs the surgeons’ haptic interactions. The relationship between applied force and material deformation rate provides biomechanical information specifying the deformation distance remaining until a tissue will fail: which is termed distance-to-break (DTB). The current study demonstrates that observers can detect DTB while deforming simulated tissues and stop before reaching the tissues’ failure points. The design of training simulators, control devices and automated robotic systems for applications outside of MIS is discussed.

Practitioner Summary: In MIS, haptic information is critical for applying appropriate forces onto soft tissue to minimise tissue trauma. Observers used force information to detect how far they could deform a virtual tissue before it would break. The design of training simulators, control devices and automated robotic systems is discussed.     

基于杆-质点模型实现的脑膜瘤术前训练系统

脑膜瘤是中枢神经系统最为常见的原发性肿瘤,手术摘除是该疾病的首选治疗方案。脑神经外科疾病发病紧急、专家少、会诊难度大,且病灶易与周围组织粘连,手术操作空间狭小。因此,设计术前培训系统帮助医生模拟手术过程是非常必要的,而如何实时模拟手术过程中手术器具与脑膜瘤及周边正常脑组织的交互力是亟待解决的问题。采用改进的杆-质点模型建立脑膜瘤摘除过程中手术器具与病灶的交互过程模型,实时计算交互力,并通过手控器反馈给医生,使医生能够真实感受手术过程。基于C#、Unity 3D 等软硬件系统构建虚拟手术系统平台进行实验,验证了所提出的方法具有实时性,可以满足神经外科手术需求。     

VR-Based Simulators for Training in Minimally Invasive Surgery

Simulation-based training using VR techniques is a promising alternative to traditional training in minimally invasive surgery (MIS). Simulators let the trainee touch, feel, and manipulate virtual tissues and organs through the same surgical tool handles used in actual MIS while viewing images of tool-tissue interactions on a monitor as in real laparoscopic procedures     

Force Feedback Benefit Depends on Experience in Multiple Degree of Freedom Robotic Surgery Task

Force feedback has been suggested to provide a number of benefits to surgery. Few studies, however, have addressed the benefit of force feedback in the context of the complexities of true surgical tasks. When information is limited (such as depth information in endoscopically guided tasks), force feedback may provide additional information that improves performance. We investigate a two-handed, six degree of freedom, endoscopically guided, minimally invasive cannulation task (inserting one tube into another tube) to test this hypothesis. We used twelve subjects, six of whom were experienced minimally invasive surgeons. Results suggest that force feedback reduces applied forces for both subject groups, but only the surgically trained group can take advantage of this benefit without a significant increase in trial time. We hypothesize that this training difference is due to the interaction between visual-spatial motor abilities and the information contained in the mechanical interaction forces.     

A novel tool for summarization of arthroscopic videos

Arthroscopic surgery is a minimally invasive procedure that uses a small camera to generate video streams, which are recorded and subsequently archived. In this paper we present a video summarization tool and demonstrate how it can be successfully used in the domain of arthroscopic videos. The proposed tool generates a keyframe-based summary, which clusters visually similar frames based on user-selected visual features and appropriate dissimilarity metrics. We discuss how this tool can be used for arthroscopic videos, taking advantage of several domain-specific aspects, without losing its ability to work on general-purpose videos. Experimental results confirm the feasibility of the proposed approach and encourage extending it to other application domains.     

Effect of visual displays and locations on laparoscopic surgical training task

The number of minimally invasive surgical (MIS) procedures has substantially increased since its introduction due to health and recovery benefits for patients. However, there are potential performance issues in MIS for surgeons due to perceptual processing demands associated with supporting technologies. Monitor location has been identified as a major factor influencing performance in these types of procedures. This study examined the effect of multiple monitors on performance during a laparoscopic surgical training task (peg transfer among instruments). Twenty-four novice subjects were exposed to different monitor conditions including a default position, a biomechanically compatible position, and a position collocated with the operating surface as well as the combination of the latter two. Subjective rankings and cognitive workload were also assessed. Results revealed a significant effect of monitor position on task time when compared to subjects' baseline training task time using the default monitor setup. Collocating the monitor with the operating surface was shown to be superior in terms of task time. There were no significant differences among monitor positions in terms of perceived workload. The results of this study provide an applicable guide for the design of MIS setups in the operating room to promote surgeon performance.