Floating autostereoscopic display with in situ interaction

Floating three‐dimensional (3D) display implements direct interaction between human hands and virtual 3D images, which offers natural and effective augmented reality interaction. In this study, we propose a novel floating autostereoscopic display, combining head tracking lenticular display with an image projection system, to offer the observers with an accurate 3D image floating in midair without any optical elements between observers and the virtual 3D image. Combined with a gesture recognition device, the proposed system can achieve in situ augmented reality interaction with the floating 3D image. A distortion correction method is developed to achieve 3D display with accurate spatial information. Moreover, a coordinate calibration method is designed to improve the accuracy in the in situ interaction. Experiments were performed to prove the feasibility of the proposed system, and the good results show the potential of human‐computer interaction in medicine and life sciences.     

基于多投影的多视点自动立体三维显示系统

多视点自动立体显示有望成为今后主流的三维显示技术,它是一种无需借助任何辅助观察设备的多视点、多观察区、高分辨率、显示效果逼真的三维显示方式。阐述了基于多投影的多视点自动立体显示系统的设计原理,详细地描述了系统的软硬件构架,建立了基于多投影仪和水平光学各向异性显示结构的自动立体显示样机,开发了投影仪阵列自动校准系统,提高了投影仪的校准精度,避免了因投影仪数目多而导致的繁琐的校准过程。实验结果能够给观众带来逼真的三维视觉体验。     

基于Kinect体感交互的多人在线虚拟实验系统

为了满足多人异地进行真实感虚拟实验的需求,使用Kinect体感设备和Unity 3D引擎搭建了一个多人在线虚拟实验系统。在该系统中,使用Unity 3D引擎搭建虚拟实验场景,通过导入3D Max制作的实验器材模型进行实验搭建,并通过网络通信技术实现远距离多人在线操作。对于真实感部分,采用Kinect体感技术捕捉的身体姿势被用来控制虚拟场景中第一人称角色的走动、抓取和操作实验器材以及选取虚拟场景中的菜单。实验结果证明,Kinect姿势识别具有很高的准确性和鲁棒性,并且不容易被光照条件和复杂的背景所影响,服务器与客户端的通信对于建立远程虚拟实验系统来说足够稳定。该系统具有成本低、真实感较强的优点。     

Visual gesture interfaces for virtual environments

Virtual environments provide a whole new way of viewing and manipulating 3D data. Current technology moves the images out of desktop monitors and into the space immediately surrounding the user. Users can literally put their hands on the virtual objects. Unfortunately, techniques for interacting with such environments are yet to mature. Gloves and sensor-based trackers are unwieldy, constraining and uncomfortable to use. A natural, more intuitive method of interaction would be to allow the user to grasp objects with their hands and manipulate them as if they were real objects.We are investigating the use of computer vision in implementing a natural interface based on hand gestures. A framework for a gesture recognition system is introduced along with results of experiments in colour segmentation, feature extraction and template matching for finger and hand tracking, and simple hand pose recognition. Implementation of a gesture interface for navigation and object manipulation in virtual environments is presented.     

Distributed real-time rendering for ultrahigh-resolution multiscreen 3D display

Large-scale autostereoscopic three-dimensional (3D) displays can give audiences a truly immersive feeling with strong visual impact. However, the traditional autostereoscopic 3D display systems are limited by the display hardware, making it difficult to directly achieve large-scale 3D displays with high resolution. Multiscreen splicing with laser backlights can be used for large-scale and ultrahigh-resolution 3D display, but it normally results in subscreen image asynchronization, view zone error, or obvious edge overlapping. To solve the problems mentioned above, a distributed real-time rendering system for ultrahigh-resolution multiscreen 3D display is proposed. Fifteen 3D LCD display devices are driven through a host, cooperating with laser backlights, a lenticular lens array (LLA), and a directional diffuser to display high resolution, high frame rate, large size, and wide-viewing angle 3D images. The resolution of the whole display system can reach 23,040 × 21,600. The rendering system provides a large-scale and real-time 3D scene image with an ultrahigh-definition resolution at a speed of 40 frames per second and high quality.     

基于Leap Motion的手势识别在虚拟交互中的研究*

针对虚拟场景中的自然手势交互进行了研究,提出了基于Leap Motion的动态指尖手势轨迹识别方法。首先借助Leap Motion设备采集指尖在场景中运动时产生的坐标并同时对数据进行预处理,然后从这一系列坐标中找出起始和结束位置并提取出有效的手势轨迹,再进行轨迹优化和手势初步分类,基于加权欧氏距离将轨迹和手势模板进行相似度计算,得到识别结果。采集200组手势数据进行实验,结果证明提出的方法具有很高的识别率,将方法应用在手势交互系统中,实现使用自然手势和虚拟物品进行交互,增加了交互乐趣,改善了交互体验。     

裸眼3D LED显示的全息透镜板在线拼接的检测方法

全息透镜板的高精度拼接与装配是基于全息透镜技术的大屏幕LED裸眼3D显示系统搭建中的关键问题。理论计算与实验结果表明,全息透镜板与LED显示模组横向相对位置误差小于1.332mm时,可以满足显示的要求。基于裸眼3D显示系统的投射条纹,提出了基于投射条纹的全息透镜板位置实时调整方法。依据此方法提出了基于极大值测量条纹中心间距的图像处理算法,并结合LabView编写了图像处理程序。实验结果表明,使用该方法测得的亮暗条纹间距的测量精度为0.1mm,反算出全息透镜板与LED屏之间的位置误差小于0.03mm,满足实时调整全息透镜板位置的要求,可以作为全息透镜板在线拼接的检测方法。     

Tracking,recognition, and distance detection of hand gestures for a 3‐D interactive display

Abstract— This study proposes an interactive system for displays, the technologies of which consists of three main parts: hand‐gesture tracking, recognition, and depth measurement. The proposed interactive system can be applied to a general 3‐D display. In this interactive system, for hand‐gesture tracking, Haar‐like features are employed to detect a specific hand gesture to start tracking, while the mean‐shift algorithm and Kalman filter are adopted for fast tracking. First, for recognizing hand gestures, a principal component analysis (PCA) algorithm is used to localize colored areas of skin, and then hand gestures are identified by comparison with a prepared database. Second, a simple optical system is set up with an infrared laser source and a grid mask in order to project a proposed horizontal stripe pattern. Third, the projected patterns are deciphered to extract the depth information using the Hough‐transform algorithm. The system containing hand‐gesture localization, recognition, and associated depth detection (the distance between the display and the hand), was included in a prototype of an interactive display. Demonstration of rotation recognition of a finger‐pointing hand gesture was successful by using the algorithm of radar‐like scanning.     

VR interactive dialog system with verbal and nonverbal communication

We have constructed a dialog environment between a human and a virtual agent. With commercial off-the-shelf VR technologies, special devices such as a data glove have to be used for the interaction, but it is difficult for anyone to manipulate objects on their own. If there is a helper who has direct access to objects in virtual space, we may ask them. The question, however, is how to communicate with the helper. The basic idea is to utilize speech and gesture recognition systems. We have already reported the above-mentioned result, although only the avatar can move a virtual object in the current system. The user cannot freely manipulate virtual objects. Therefore, in a new attempt, we constructed a communication channel between virtual space and the real world so that the virtual object could be manipulated. In order to develop the new system, we extended the existing system to an internet meeting system allowing users in different places to interact with each other by voice and by a pointing action with a finger. This work was presented in part and awarded as Young Author Award at the 13th International Symposium on Artificial Life and Robotics, Oita, Japan, January 31–February 2, 2008     

Image correction based on homography in stereoscopic projection display

Abstract— Stereoscopic and autostereoscopic projection‐display systems use projector arrays to present stereoscopic images, and each projector casts one parallax image of a stereoscopic scene. Because of the position shift of the projectors, the parallax images have geometric deformation, which influences the quality of the displayed stereoscopic images. In order to solve this problem, a method based on homography is proposed. The parallax images are pre‐transformed before they are projected, and then the stereoscopic images without geometric distortion can be obtained. An autostereoscopic projection‐display system is developed to present the images with and without calibration. Experimental results show that this method works effectively.