基于AR-Builder的增强现实应用探索

随着计算机计算能力的提高,增强现实技术已经逐步从实验室理论研究阶段开始转入行业应用阶段。介绍了增强现实的关键技术特点和应用领域,并基于AR-Builder实现了增强现实在主题公园人机交互的应用探索。     

一种基于视频流的增强现实关键技术研究与实现

当下,将真实世界和虚拟世界信息“无缝”集成的增强现实技术正逐步融入人们的生活,为我们带来颠覆式的视觉感受与体验.本文针对移动增强现实关键技术开展研究,提出了一种基于视频流的实时运动的估计方法.仿真分析和应用表明,该技术可有效提升增强现实中稳定、智能识别视频流的能力.后续将结合移动特有的海量数据资源,开展该技术在智慧地图、智慧旅游、AR手游等新兴场景的应用,提升通信行业在信息化服务领域的发展能力和核心竞争力.     

Fully convolutional network-based registration for augmented assembly systems

Image-based registration methods have been widely used in augmented assembly systems. However, many challenges remain to be solved, such as low robustness and poor timeliness during registration. This paper presents a deep learning approach for registration. To reduce the workload of data collection, an automatic picture generation method is offered for deep learning algorithm, and a dataset is built for detecting the keypoints of assembly objects. We propose a fully convolutional network (FCN) model for detecting keypoints from a single RGB image. The FCN model, which consists of 13 layers, can accurately detect the location of keypoints with a low computational resource consumption. The detected keypoints are used to solve the camera position and orientation for augmented assembly registration. The experiments demonstrate that our method is robust against different rotation angles of the assembly object and against background interference. The detection accuracy is high under different camera motion blurs; to be specific, the pixel error in different directions of 640 × 480 images was only 0.9 pixels. The FCN-based registration approach was shown to be fast during augmented assembly experiments, achieving up to 30 FPS.     

Newton-conjugate gradient （CG） augmented Lagrangian method for path constrained dynamic process optimization

In this paper, a Newton-conjugate gradient (CG) augmented Lagrangian method is proposed for solving the path constrained dynamic process optimization problems. The path constraints are simplified as a single final time constraint by using a novel constraint aggregation function. Then, a control vector parameterization (CVP) approach is applied to convert the constraints simplified dynamic optimization problem into a nonlinear programming (NLP) problem with inequality constraints. By constructing an augmented Lagrangian function, the inequality constraints are introduced into the augmented objective function, and a box constrained NLP problem is generated. Then, a linear search Newton-CG approach, also known as truncated Newton (TN) approach, is applied to solve the problem. By constructing the Hamiltonian functions of objective and constraint functions, two adjoint systems are generated to calculate the gradients which are needed in the process of NLP solution. Simulation examples demonstrate the effectiveness of the algorithm.     

Coupling dynamic simulation and interactive multiobjective optimization for complex problems: An APROS-NIMBUS case study

Dynamic process simulators for plant-wide process simulation and multiobjective optimization tools can be used by industries as a means to cut costs and enhance profitability. Specifically, dynamic process simulators are useful in the process plant design phase, as they provide several benefits such as savings in time and costs. On the other hand, multiobjective optimization tools are useful in obtaining the best possible process designs when multiple conflicting objectives are to be optimized simultaneously. Here we concentrate on interactive multiobjective optimization. When multiobjective optimization methods are used in process design, they need an access to dynamic process simulators, hence it is desirable for them to coexist on the same software platform. However, such a co-existence is not common. Hence, users need to couple multiobjective optimization software and simulators, which may not be trivial. In this paper, we consider APROS, a dynamic process simulator and couple it with IND-NIMBUS, an interactive multiobjective optimization software. Specifically, we: (a) study the coupling of interactive multiobjective optimization with a dynamic process simulator; (b) bring out the importance of utilizing interactive multiobjective optimization; (c) propose an augmented interactive multiobjective optimization algorithm; and (d) apply an APROS-NIMBUS coupling for solving a dynamic optimization problem in a two-stage separation process.     

Automatic generation and detection of highly reliable fiducial markers under occlusion

This paper presents a fiducial marker system specially appropriated for camera pose estimation in applications such as augmented reality and robot localization. Three main contributions are presented. First, we propose an algorithm for generating configurable marker dictionaries (in size and number of bits) following a criterion to maximize the inter-marker distance and the number of bit transitions. In the process, we derive the maximum theoretical inter-marker distance that dictionaries of square binary markers can have. Second, a method for automatically detecting the markers and correcting possible errors is proposed. Third, a solution to the occlusion problem in augmented reality applications is shown. To that aim, multiple markers are combined with an occlusion mask calculated by color segmentation. The experiments conducted show that our proposal obtains dictionaries with higher inter-marker distances and lower false negative rates than state-of-the-art systems, and provides an effective solution to the occlusion problem.     

Hand gesture guided robot-assisted surgery based on a direct augmented reality interface

Radiofrequency (RF) ablation is a good alternative to hepatic resection for treatment of liver tumors. However, accurate needle insertion requires precise hand-eye coordination and is also affected by the difficulty of RF needle navigation. This paper proposes a cooperative surgical robot system, guided by hand gestures and supported by an augmented reality (AR)-based surgical field, for robot-assisted percutaneous treatment. It establishes a robot-assisted natural AR guidance mechanism that incorporates the advantages of the following three aspects: AR visual guidance information, surgeon's experiences and accuracy of robotic surgery. A projector-based AR environment is directly overlaid on a patient to display preoperative and intraoperative information, while a mobile surgical robot system implements specified RF needle insertion plans. Natural hand gestures are used as an intuitive and robust method to interact with both the AR system and surgical robot. The proposed system was evaluated on a mannequin model. Experimental results demonstrated that hand gesture guidance was able to effectively guide the surgical robot, and the robot-assisted implementation was found to improve the accuracy of needle insertion. This human–robot cooperative mechanism is a promising approach for precise transcutaneous ablation therapy.     

Real-time advanced spinal surgery via visible patient model and augmented reality system

This paper presents an advanced augmented reality system for spinal surgery assistance, and develops entry-point guidance prior to vertebroplasty spinal surgery. Based on image-based marker detection and tracking, the proposed camera-projector system superimposes pre-operative 3-D images onto patients. The patients’ preoperative 3-D image model is registered by projecting it onto the patient such that the synthetic 3-D model merges with the real patient image, enabling the surgeon to see through the patients’ anatomy. The proposed method is much simpler than heavy and computationally challenging navigation systems, and also reduces radiation exposure. The system is experimentally tested on a preoperative 3D model, dummy patient model and animal cadaver model. The feasibility and accuracy of the proposed system is verified on three patients undergoing spinal surgery in the operating theater. The results of these clinical trials are extremely promising, with surgeons reporting favorably on the reduced time of finding a suitable entry point and reduced radiation dose to patients.     

Lessons learned: Evaluating visualizations for occluded objects in handheld augmented reality

Handheld devices like smartphones and tablets have emerged as one of the most promising platforms for Augmented Reality (AR). The increased usage of these portable handheld devices has enabled handheld AR applications to reach the end-users; hence, it is timely and important to seriously consider the user experience of such applications. AR visualizations for occluded objects enable an observer to look through objects. AR visualizations have been predominantly evaluated using Head-Worn Displays (HWDs), handheld devices have rarely been used. However, unless we gain a better understanding of the perceptual and cognitive effects of handheld AR systems, effective interfaces for handheld devices cannot be designed. Similarly, human perception of AR systems in outdoor environments, which provide a higher degree of variation than indoor environments, has only been insufficiently explored.In this paper, we present insights acquired from five experiments we performed using handheld devices in outdoor locations. We provide design recommendations for handheld AR systems equipped with visualizations for occluded objects. Our key conclusions are the following: (1) Use of visualizations for occluded objects improves the depth perception of occluded objects akin to non-occluded objects. (2) To support different scenarios, handheld AR systems should provide multiple visualizations for occluded objects to complement each other. (3) Visual clutter in AR visualizations reduces the visibility of occluded objects and deteriorates depth judgment; depth judgment can be improved by providing clear visibility of the occluded objects. (4) Similar to virtual reality interfaces, both egocentric and exocentric distances are underestimated in handheld AR. (5) Depth perception will improve if handheld AR systems can dynamically adapt their geometric field of view (GFOV) to match the display field of view (DFOV). (6) Large handheld displays are hard to carry and use; however, they enable users to better grasp the depth of multiple graphical objects that are presented simultaneously.     

ELECTROHYDRODYNAMIC ENHANCEMENT OF NUCLEATE BOILING HEAT TRANSFER IN HEAT EXCHANGERS

The effect of a radial d.c. electric field on nucleate boiling heat transfer was investigated experimentally using a single-tube shell/tube heat exchanger. A dielectric liquid (Freon R. 114) was used in the shell and the tube was heated by circulating water through it.

It was found that the application of a sufficiently intense electric field to the boiling heat transfer surface resulted both in the elimination of boiling hysteresis and the enhancement of heat transfer for the range of superheat studied. An application of approximately 20 kV was more than sufficient to eliminate the hysteresis.

It was also observed that there appeared to be a transition between two situations. At lower superheat there is an appreciable improvement in the heat transfer coefficient due to both initial (0 to 10 kV) and incremental (10 to 20 kV) voltage increases. At higher superheat, however, the greater improvement (about four times) is obtained with the initial voltage application