3D interactive system based on vision computing of direct‐flective cameras

A bare‐finger 3D interactive technology for portable devices was developed. Using directive‐flective cameras to reform the field of viewing, a blind working range close to the camera is eliminated. Moreover, the algorithm of vision computing, different from skin color detection, is presented to determine the positions of fingertips. The interactive range is workable from 1.5 to 50 cm above the entire surface of the display. The mean position error of less than 1 cm is achieved. This accuracy realizes a camera‐based 3D interactive system allowing for near‐distance functionality. Therefore, floating 3D images can be touched and interacted with, potentially creating more application and intuitive user‐machine interface.     

Dual layered display that presents auto‐stereoscopic 3D images to multiple viewers in arbitrary positions

Dual layered display or also called tensor display that consists of two panels in a stack can present full‐parallax 3D images with high resolution and continuous motion parallax by reconstructing corresponding light ray field within a viewing angle. The depth range where the 3D images can be displayed with reasonable resolution, however, is limited around the panel stack. In this paper, we propose a dual layered display that can present stereoscopic images to multiple viewers located at arbitrary positions in observer space with high resolution and large depth range. Combined with the viewer tracking system, the proposed method provides a practical way to realize high‐resolution large‐depth auto‐stereoscopic 3D display for multiple observers without restriction on the observer position and the head orientation.     

面向机器人运动跟踪的电磁定位系统

电磁定位具有精度高、速度快和易实现等优点,所以对手术、室内和室外机器人跟踪是一个良好的选择。电磁定位以交变电磁信号作为源信号。交变的电流信号激励发射线圈（信号激励部件）在空间中产生交变的电磁场,感应线圈（信号感应部件）在交变的电磁场中输出频率相同的信号。根据输出信号的幅值和相位信息,我们可以计算出感应线圈相对于发射线圈的位置和方向信息。本文面向机器人定位跟踪,介绍电磁定位系统的原理与实现,包括磁场模型、电磁定位算法与系统软硬件的搭建与实验。本文介绍两种不同的激励模式,分别为分时激励3轴正交发射线圈模式与同时激励2轴正交发射线圈模式,两种模式中的感应线圈均采用3轴正交线圈。实验结果表明,定位系统可以达到1 mm的定位精度。     

General design method for three-dimensional displays with horizontally asymmetric pixel structures

Various 3D displays have been proposed to show realistic and vivid 3D images. Moreover, 3D displays have been applied in various fields including medicine, entertainment, and advertising. Depending on the application, 3D displays have different pixel structures and sizes. In this paper, we present a 3D-display design method that can be applied regardless of the pixel structure and display sizes. The area of the designable 3D display is suggested by the improved 3D image quality. The manufactured displays are used to verify the proposed method. Furthermore, a light field simulation is performed to confirm the area that was not proven by the manufactured displays. With the proposed 3D image-quality model and 3D image simulation by the light field representation, a general design of 3D displays with various pixel structures can be developed.     

Analysis of head pose accuracy in augmented reality

A method is developed to analyze the accuracy of the relative head-to-object position and orientation (pose) in augmented reality systems with head-mounted displays. From probabilistic estimates of the errors in optical tracking sensors, the uncertainty in head-to-object pose can be computed in the form of a covariance matrix. The positional uncertainty can be visualized as a 3D ellipsoid. One useful benefit of having an explicit representation of uncertainty is that we can fuse sensor data from a combination of fixed and head-mounted sensors in order to improve the overall registration accuracy. The method was applied to the analysis of an experimental augmented reality system, incorporating an optical see-through head-mounted display, a head-mounted CCD camera, and a fixed optical tracking sensor. The uncertainty of the pose of a movable object with respect to the head-mounted display was analyzed. By using both fixed and head mounted sensors, we produced a pose estimate that is significantly more accurate than that produced by either sensor acting alone     

A next-best-view system for autonomous 3-D object reconstruction

The focus of this paper is to design and implement a system capable of automatically reconstructing a prototype 3D model from a minimum number of range images of an object. Given an ideal 3D object model, the system iteratively renders range and intensity images of the model from a specified position, assimilates the range information into a prototype model, and determines the sensor pose (position and orientation) from which an optimal amount of previously unrecorded information may be acquired. Reconstruction is terminated when the model meets a given threshold of accuracy. Such a system has applications in the context of robot navigation, manufacturing, or hazardous materials handling. The system has been tested successfully on several synthetic data models, and each set of results was found to be reasonably consistent with an intuitive human search. The number of views necessary to reconstruct an adequate 3D prototype depends on the complexity of the object or scene and the initial data collected. The prototype models which the system recovers compare well with the ideal models     

基于光场扫描的真三维立体显示系统的开发

近年来,计算机视觉领域出现了一种新型的三维显示技术,即光场三维显示技术。基于光场扫描的真三维立体显示系统通过重构物体光强的空间分布,减少了信息冗余,采用高速投影仪、定向散射反射镜和高速旋转马达等设计而成。通过分析光场三维显示原理和系统工作原理、显示系统架构和立体成像原理等,详细论述了显示系统开发的可行性,并通过实验证明了当达到一定投影输出功率和马达旋转功率时可实现立体成像。观测者可360度裸眼观测立体图像,并且无需佩戴任何辅助工具。     

Tabletop integral imaging 3D display system based on annular point light source

When the viewers sitting around the table observe 3D images, the viewing direction is generally oblique and the viewpoints should be distributed as annular. In this paper, a tabletop integral imaging (II) three-dimensional (3D) display system based on annular point light sources is demonstrated, which can present 3D images to multiple viewers in a standard annular viewing area with oblique viewing direction. The proposed system consists of annular point light sources, a Fresnel lens, a lens array, a two-dimensional (2D) display panel, and a diffuser screen. Each point light source illuminates the Fresnel lens to form parallel light and then illuminates the lens array and the display panel. A viewing sub-area is generated at the position of the diffuser screen, in which the 3D images can be viewed. Multiple viewing sub-areas are created in a way of time-division multiplexing to form a 360° annular viewing area. Compared with the previous tabletop 3D display, the viewing area can be concentrated at an oblique angle near the tabletop. The experimental results demonstrate the feasibility of the tabletop II 3D display system.     

特征重加权U-Net超声图像分割的骨结构重建与增强现实显示

在现有骨科微创手术中,医生通过观察屏幕上显示的二维骨结构X射线图像,引导手术器械置入并完成治疗.但这种方式不仅缺乏直观的三维信息,而且导致病人和医生暴露在大量的射线电离辐射下.为了解决该问题,提出特征重加权超声分割的骨结构重建与增强现实显示技术,将骨结构直观展示给医生,为医生提供无辐射的手术引导.首先,开发了一个特征重加权U-Net,从Free-hand超声图像中精确分割提取骨表面.然后,根据超声图像的姿态,对分割后的骨表面进行重建,获得三维骨结构.最后,使用立体全像技术对重建的骨结构进行增强现实显示,展示给医生.在仿体和真实患者超声数据集上进行了实验,所获得的骨结构分割精度为(88.51±1.44)%,骨结构重建误差为(1.29±0.11)mm.实验结果表明,所提出的基于特征重加权U-Net超声图像分割的骨结构重建与增强现实显示技术可以为外科医生提供直观的术中三维导航信息.     

一种硬币鉴别系统的设计与实现

介绍了一种可用于我国现行流通硬币的鉴别系统.该系统以电涡流传感器、光电传感器为检测手段,以LM3S1138微控制器为控制核心,设计并实现了对硬币面值的快速识别及实时显示、对假币的辨伪和清退,系统鉴别精度高,成本低,具有一定的应用价值.     

Image‐guided robotic navigation system for neurosurgery

This paper presents an image‐guided robotic navigation system for neurosurgery, which can be applied to the electro‐stimulation treatment of Parkinson's leisure, the biopsy of deep tumors, and haematoma evacuation. The system integrates a computer containing CT images for surgical planning, a magnetic tracking device for measuring the coordinates of the markers and surgical instruments, and a robot manipulator for guiding surgical instruments to the preplanned position and orientation. The computer display of brain anatomy offers a convenient tool for surgeons to diagnose brain diseases and to plan safe surgical paths, while the tracking device guides the robot manipulator to automatically move surgical instruments to the preplanned position and orientation. An experiment of using a skull model for simulating a robotic biopsy of brain tumor has been done to verify the performance of the robotic navigation system. The results show that the positioning accuracy of the robot relative to the tracker frame is only related to the positioning resolution of the robot manipulator and the positioning accuracy of the tracking device. In other words, the positioning accuracy of the robot manipulator does not affect the final positioning accuracy of the surgical instruments. Therefore, using a robot manipulator for precise surgical navigation is feasible and reliable. © 2000 John Wiley & Sons, Inc.     

3D navigation and monitoring for spinal milling operation based on registration between multiplanar fluoroscopy and CT images

Milling operations in spinal surgery demand much experience and skill for the surgeon to perform the procedure safely. A 3D navigation method is introduced aiming at providing a monitoring system with enhanced safety and minimal intraoperative interaction. An automatic registration method is presented to establish the 3D-3D transformation between the preoperative CT images and a common reference system in the surgical space, and an intensity-based similarity metric adapted for the multi-planar configuration is introduced in the registration procedure. A critical region is defined for real-time monitoring in order to prevent penetration of the lamina and avoid violation of nerve structures. The contour of the spinal canal is reconstructed as the critical region, and different levels of warning limits are defined. During the milling procedure, the position of the surgical instrument relative to the critical region is provided with augmented display and audio warnings. Timely alarm is provided for surgeons to prevent surgical failure when the mill approaches the critical region. Our validation experiment shows that real-time 3D navigation and monitoring is advantageous for improving the safety of the milling operation.     

基于坐标逆映射的增强型车辆三维全景影像

当前最先进的车辆三维全景影像虽然可以较好地对车身周边环境进行三维立体的拟真显示,但仍然会对车身近处的三维物体造成显示畸变,极大地影响显示效果、降低实用性。针对该问题,提出一种增强型车辆三维全景影像的合成方法。首先利用YOLOv4网络检测出车辆及行人在图像中的位置,之后基于坐标升维逆映射将检测出的物体位置升维映射至世界坐标系下,最后将三维模型渲染在相应的逆映射位置上来代替显示畸变的三维物体,从而给驾驶员提供有效的周边物体位置信息。实验结果表明,所提方法生成的增强型车辆三维全景影像具有很好的实时性和显示效果,能够有效解决当前车辆三维全景影像的显示缺陷。     

基于CT图像的肝脏及其内部管道可视化系统

建立肝脏及其内部管道系统的三维模型并进行切割手术模拟可以提高肝脏外科手术的精确性和安全性,达到最大限度地减小手术创伤、缩短手术时间、提高手术成功率、减少医疗费用及促进病人康复的目的。论文阐述了作者开发基于CT图像的肝脏及其内部管道系统三维重建、虚拟切割采用的技术和方法———用直接体视化方法建立肝脏的三维体视化模型,在OPENGL环境下采用空间点绘制的方法显示肝脏的管道系统并进行切割手术模拟。最后,采用小波变化进行了医学图像数据的压缩。     

A three-dimensional measurement system for robot applications

This paper presents a 3D noncontacting sensor system designed to measure the position and orientation of a robot end effector. This measurement system includes two parts: a tridimensional object including four spheres placed along the axes of a tetrahedron and a set of three orthogonally pointed cameras. The purpose is to design a measurement system characterized by easy relationships in order to satisfy real-time constraints. The system has been used in two experiments: first, to calibrate a parallel robot and validate the geometrical control performance, then as an exteroceptive sensor in an assembly task. The system computes position and orientation of the tetrahedron in 100 ms time. The position and orientation accuracy are, respectively, 0.6 mm and 0.2 deg in a workspace, being a cube with 0.3 m sides.     

Automated identification of thoracolumbar vertebrae using orthogonal matching pursuit

A computer-assisted system that can automatically provide rapid localization and accurate labeling of vertebral disks and bodies is a highly desirable tool due to the large demand for the diagnostic imaging and surgical planning of the vertebral column structures. However, a reliable detection and definitive labeling of vertebrae can be difficult due to factors such as the limited imaging coverage and various vertebral anomalies particularly in the thoracolumbar and lumbosacral junctions. In this paper, we investigate the problem of identifying the last thoracic and first lumbar vertebrae in CT images. The main purpose of this study is to improve the accuracy of labeling vertebrae of an automatic spine labeling system especially when the field of view is limited in the lower spine region. We present a dictionary-based classification method using a cascade of simultaneous orthogonal matching pursuit classifiers on 2D vertebral regions extracted from the maximum intensity projection images. The performance of the proposed method in terms of accuracy and speed has been validated by experimental results on hundreds of CT images collected from various clinical sites.     

Transpost: a novel approach to the display and transmission of 360 degrees-viewable 3D solid images

Three-dimensional displays are drawing attention as next-generation devices. Some techniques which can reproduce three-dimensional images prepared in advance have already been developed. However, technology for the transmission of 3D moving pictures in real-time is yet to be achieved. In this paper, we present a novel method for 360-degrees viewable 3D displays and the Transpost system in which we implement the method. The basic concept of our system is to project multiple images of the object, taken from different angles, onto a spinning screen. The key to the method is projection of the images onto a directionally reflective screen with a limited viewing angle. The images are reconstructed to give the viewer a three-dimensional image of the object displayed on the screen. The display system can present images of computer-graphics pictures, live pictures, and movies. Furthermore, the reverse optical process of that in the display system can be used to record images of the subject from multiple directions. The images can then be transmitted to the display in real-time. We have developed prototypes of a 3D display and a 3D human-image transmission system. Our preliminary working prototypes demonstrate new possibilities of expression and forms of communication.     

An optical navigator for brain surgery

Computer-assisted surgery (CAS) helps surgeons find their bearings during operations under difficult visual conditions. These include brain surgery and other head operations. CAS combines coordinate measurements of the operating instruments and superimposes their positions on to images of the operation area so that surgeons can follow instrument movements on a computer screen. Since 1986, the Institute for Measurement Techniques (Lehrstuhl fu¨r Meßtechnik) has been working in close cooperation with the Aachen University Hospital to develop CAS systems. This project has successfully treated numerous patients in the areas of neurosurgery, otorhinolaryngology (ear, nose and throat), ophthalmology, radiotherapy and neurology. Based on this work, Philips Medical Systems has developed a commercial system for neurosurgery. This orientation device to aid physician accuracy in brain surgery has been developed by combining 3D position measurement techniques, digital image processing and 3D display techniques. This article provides an overview of the development process and an outlook on possible applications     

卷筒纸纸带跑偏检测与纠正系统

针对卷筒纸印刷机对纸带横向位置要居中的要求，给出了光电式传感器的纸带跑偏检测与纠正系统。该系统采用了光敏三极管传感器，把纸带横向偏移量变换为电信号，经放大器、A／D转换器等信号处理后送到AT89C51单片机进行分析、处理和判断，把结果输出给显示器进行显示，并控制步进电机转动相应的步数，步进电机驱动导纸辊机构运动，使纸带跑偏得到纠正。实验结果表明：该系统具有测量线性度好、纠正精度高，能满足印刷中横向套印的要求。