结合图像识别的移动增强现实系统设计与应用

目的 在移动互联网时代下,移动增强现实应用得到越来越快的发展。然而户外场景中存在许多相似结构的建筑,且手机的存储和计算能力有限,因此应用多集中于室内小范围环境,对于室外大规模复杂场景的适应性较弱。对此,建立一套基于云端图像识别的移动增强现实系统。方法 为解决相似特征的误匹配问题,算法中将重力信息加入到SURF和BRISK特征描述中去,构建Gravity-SURF和Gravity-BRISK特征描述。云端系统对增强信息进行有效管理,采用基于Gravity-SURF特征的VLAD方法对大规模图像进行识别;在智能终端上的应用中呈现识别图像的增强信息,并利用识别图像的Gravity-BRISK特征和光流结合的方法对相机进行跟踪,采用Unity3D渲染引擎实时绘制3维模型。结果 在包含重力信息的4 000幅户外图像的数据库中进行实验。采用结合重力信息的特征描述算法,能够增强具有相似特征的描述符的区分性,并提高匹配正确率。图像识别算法的识别率能达到88%以上,识别时间在420 ms左右;光流跟踪的RMS误差小于1.2像素,帧率能达到23 帧/s。结论 本文针对室外大规模复杂场景建立的基于图像识别的移动增强现实系统,能方便对不同应用的增强现实数据进行管理。系统被应用到谷歌眼镜和新闻领域上,不局限于单一的应用领域。结果表明,识别算法和跟踪注册算法能够满足系统的精度和实时性要求。     

Evaluating View Management for Situated Visualization in Web-based Handheld AR

As visualization makes the leap to mobile and situated settings, where data is increasingly integrated with the physical world using mixed reality, there is a corresponding need for effectively managing the immersed user's view of situated visualizations. In this paper we present an analysis of view management techniques for situated 3D visualizations in handheld augmented reality: a shadowbox, a world-in-miniature metaphor, and an interactive tour. We validate these view management solutions through a concrete implementation of all techniques within a situated visualization framework built using a web-based augmented reality visualization toolkit, and present results from a user study in augmented reality accessed using handheld mobile devices.     

OpenTracker: A flexible software design for three-dimensional interaction

Tracking is an indispensable part of any virtual reality and augmented reality application. While the need for quality of tracking, in particular for high performance and fidelity, has led to a large body of past and current research, little attention is typically paid to software engineering aspects of tracking software. To address this issue we describe a software design and implementation that applies the pipes-and-filter architectural pattern to provide a customizable and flexible way of dealing with tracking data and configurations. The contribution of this work cumulates in the development of a generic data flow network library called OpenTracker to deal specifically with tracking data. The flexibility of the data flow network approach is demonstrated in a set of development scenarios and prototype applications in the area of mobile augmented reality.     

基于移动增强现实的城乡自然科普游戏设计

近年来,随着信息技术的发展,移动增强现实作为一种新兴的体验方式,在目前的教学资源设计过程中得到了大量的应用。而在教育游戏开发领域,出现了许多融合移动增强现实技术与教育游戏特点与优势的增强现实教育游戏。本研究基于移动增强现实技术开发了一个自然科普游戏,通过创设相对真实的、具有交互性质的自然科学学习环境,使得许多曾经难以出现的学习材料以虚拟形象的方式出现在学习者面前,培养学习者的科学探究能力,拉近和提高城乡学生的生态保护知识水平并塑造正确对待自然的情感态度价值观,为城乡教育一体化提供了思路。     

A Geometry-Based Point Cloud Reduction Method for Mobile Augmented Reality System

In this paper, a geometry-based point cloud reduction method is proposed, and a real-time mobile augmented reality system is explored for applications in urban environments. We formulate a new objective function which combines the point reconstruction errors and constraints on spatial point distribution. Based on this formulation, a mixed integer programming scheme is utilized to solve the points reduction problem. The mobile augmented reality system explored in this paper is composed of the offline and online stages. At the offline stage, we build up the localization database using structure from motion and compress the point cloud by the proposed point cloud reduction method. While at the online stage, we compute the camera pose in real time by combining an image-based localization algorithm and a continuous pose tracking algorithm. Experimental results on benchmark and real data show that compared with the existing methods, this geometry-based point cloud reduction method selects a point cloud subset which helps the image-based localization method to achieve higher success rate. Also, the experiments conducted on a mobile platform show that the reduced point cloud not only reduces the time consuming for initialization and re-initialization, but also makes the memory footprint small, resulting a scalable and real-time mobile augmented reality system.     

基于OFern与PTAM算法的移动增强现实技术研究

当前,移动增强现实技术已成为虚拟现实领域的一个研究热点。由于移动设备内存较小且摄像头的采集范围有限,所以传统的增强现实算法难以满足移动用户的增强现实需求。为解决此类问题,本文提出一种基于OFern和PTAM算法相结合的增强现实技术,使其在较小的内存空间中就可以运行,且能实现较远距离的mark识别。实验证明,在满足较高的实时性和稳定性的同时,OFern算法得到的分类器尺寸减小到原始算法的1/8至1/10倍。融合PTAM算法后,在体验距离上较原始算法提高了13倍以上。     

A Flexible Software Architecture for Hybrid Tracking

Fusion of vision-based and inertial pose estimation has many high-potential applications in navigation, robotics, and augmented reality. Our research aims at the development of a fully mobile, completely self-contained tracking system, that is able to estimate sensor motion from known 3D scene structure. This requires a highly modular and scalable software architecture for algorithm design and testing. As the main contribution of this paper, we discuss the design of our hybrid tracker and emphasize important features: scalability, code reusability, and testing facilities. In addition, we present a mobile augmented reality application, and several first experiments with a fully mobile vision-inertial sensor head. Our hybrid tracking system is not only capable of real-time performance, but can also be used for offline analysis of tracker performance, comparison with ground truth, and evaluation of several pose estimation and information fusion algorithms. © 2004 Wiley Periodicals, Inc.     

Binocular mobile augmented reality based on stereo camera tracking

Mobile augmented reality (AR) applications have become feasible with the evolution of mobile hardware. For example, the advent of the smartphone allowed implementing real-time mobile AR, which triggered the release of various applications. Recently, rapid development of display technology, especially for stereoscopic displays, has encouraged researches to implement more immersive and realistic AR. In this paper, we present a framework of binocular augmented reality based on stereo camera tracking. Our framework was implemented on a smartphone and supports autostereoscopic display and video see-through display in which a smartphone can be docked. We modified edge-based 3-D object tracking in order to estimate poses of left and right cameras jointly; this guarantees consistent registration across left and right views. Then, virtual contents were overlaid onto camera images using estimated poses, and the augmented stereo images were distorted to be shown through a video see-through display. The feasibility of the proposed framework is shown by experiments and demonstrations.     

Agent-customized training for human learning performance enhancement

Training individuals from diverse backgrounds and in changing environments requires customized training approaches that align with the individual learning styles and ever-evolving organizational needs. Scaffolding is a well-established instructional approach that facilitates learning by incrementally removing training aids as the learner progresses. By combining multiple training aids (i.e. multimodal interfaces), a trainer, either human or virtual, must make real-time decisions about which aids to remove throughout the training scenario. A significant problem occurs in implementing scaffolding techniques since the speed and choice of removing training aids must be strongly correlated to the individual traits of a specific trainee. We detail an agent-based infrastructure that supports the customization of scaffolding routines as triggered by the performance of the trainee. The motivation for this agent-based approach is for integration into a training environment that leverages augmented reality (AR) technologies. Initial experiments using the simulated environment have compared the proposed adaptive approach with traditional static training routines. Results show that the proposed approach increases the trainees’ task familiarity and speed with negligible introduction of errors.     

A survey of mobile and wireless technologies for augmented reality systems

Recent advances in hardware and software for mobile computing have enabled a new breed of mobile augmented reality (AR) systems and applications. A new breed of computing called ‘augmented ubiquitous computing’ has resulted from the convergence of wearable computing, wireless networking, and mobile AR interfaces. In this paper, we provide a survey of different mobile and wireless technologies and how they have impact AR. Our goal is to place them into different categories so that it becomes easier to understand the state of art and to help identify new directions of research. Copyright © 2008 John Wiley & Sons, Ltd.     

Augmented reality technologies,systems and applications

This paper surveys the current state-of-the-art of technology, systems and applications in Augmented Reality. It describes work performed by many different research groups, the purpose behind each new Augmented Reality system, and the difficulties and problems encountered when building some Augmented Reality applications. It surveys mobile augmented reality systems challenges and requirements for successful mobile systems. This paper summarizes the current applications of Augmented Reality and speculates on future applications and where current research will lead Augmented Reality’s development. Challenges augmented reality is facing in each of these applications to go from the laboratories to the industry, as well as the future challenges we can forecast are also discussed in this paper. Section 1 gives an introduction to what Augmented Reality is and the motivations for developing this technology. Section 2 discusses Augmented Reality Technologies with computer vision methods, AR devices, interfaces and systems, and visualization tools. The mobile and wireless systems for Augmented Reality are discussed in Section 3. Four classes of current applications that have been explored are described in Section 4. These applications were chosen as they are the most famous type of applications encountered when researching AR apps. The future of augmented reality and the challenges they will be facing are discussed in Section 5.     

FMI：一种查找丢失物品的移动物联社区系统

为解决丢失的贵重物品快速查找问题,本文提出一种支持丢失物品移动查找的社区管理系统框架,在配置了RFID阅读器的手机和携带EPC标签物品的基础上,采用移动计算、增强现实技术,基于SOA架构实现灵活高效的移动交互式失物招领物联系统。本文描述该社区系统的设计理念及关键技术,并探讨提升系统性能、降低用户负担的方法,本系统是对物联网技术实际应用的有益探索,可有效地提升社区生活的舒适度。     

Interactive augmented reality system for enhancing library instruction in elementary schools

Due to limited budgets and manpower, most elementary schools in Taiwan do not plan or provide library instruction for students. Although students can use libraries, they typically lack the knowledge needed to use library resources effectively. Consequently, students have difficulty finding the books they need and can easily become overwhelmed by the massive amount of information in libraries. Computer-assisted instruction for teaching basic library skills to large numbers of students is an appealing method. Particularly, developing augmented reality (AR) technologies for learning have garnered considerable attention in education research. Many researchers and scholars believe that integrating teaching and AR enhances student learning performance and motivation. This work develops an educational AR system based on situated learning theory, and applies innovative augmented reality interactive technology to a library’s learning environment. Student library knowledge can be enhanced via the proposed augmented reality library instruction system (ARLIS). Experimental results demonstrate that student learning performance is improved significantly by using the proposed ARLIS. Moreover, this work demonstrates that using the proposed ARLIS for library instruction results in the same learning performance as conventional librarian instruction and there is no gender difference on learning performance between the proposed ARLIS and conventional librarian instruction. Moreover, the proposed library instruction system overcomes shortcomings of personal teaching skills of librarians that may adversely affect student learning performance by conveying the same learning content to all students. Additionally, the proposed system results in better learning performance for learners with the field-dependent cognitive style than learners with the field-independent cognitive style. Further, the proposed system provides more benefits in terms of library skills of application and comprehension than conventional librarian instruction. Moreover, the learning performance of students is not affected by their gaming skills. Therefore, student gaming skills do not need to be considered when adopting the proposed system in library instruction programs.     

A Direct Execution Approach to Simulating Mobile Agent Algorithms

Mobile agent technology has been applied to develop the solutions for various kinds of parallel and distributed computing problems. However, performance evaluation of mobile agent algorithms remains a difficult task, mainly due to the characteristics of mobile agents such as distributed and asynchronous execution, autonomy and mobility. This paper proposes a general approach based on direct execution simulation for evaluating the performance of mobile agent algorithms by collecting and analyzing the information about the agents during their execution. We describe the proposed generic simulation model, named MADES, the architecture of a software environment based on MADES, and a prototype implementation. A mobile agent-based distributed load balancing algorithm has been used for experiments with the prototype.     

Mapping quality requirements for pervasive mobile games

Games have not received the full attention of the requirements engineering community. This scenario is becoming more critical as we move towards newer forms of games, such as pervasive games. Pervasiveness (the quality that distinguishes pervasive games from traditional digital games) holds several meanings, including being ubiquitous, permeating something, or spreading something, somewhere, in a physical space. Pervasiveness can be recognized in by the boundaries of the game expanding every time it is played, from the virtual (or fictional) world to the real world. Pervasive games are a new form of digital entertainment that has evolved in different forms, such as alternate reality games, transmedia games, and crossmedia games. Sensor technologies, networking capabilities, augmented reality systems, computer vision technology, the internet, and, especially, mobile devices have been responsible for the rapid evolution of this new form of digital product. This paper is focused on “pervasive mobile games”, which we define as context-aware games that use mobile devices. We bear in mind that mobile devices are currently the main driver for fulfilling the promises of pervasive game playing. Our investigations and experiments on this class of games led us to study the quality requirements for pervasive mobile games. Using different information sources, we gathered a set of interrelated characteristics that are crucial to the success of these games. In this paper, we begin to clarify the definition and scope of pervasive mobile games, which are controversial issues in the literature. Using these fundamentals, we propose a two-level conceptual map of non-functional requirements that helps to realize pervasiveness in pervasive mobile games. These non-functional requirements are then associated with a set of questions that help the designers in verifying tasks and operationalizing the requirements of a game. We also propose a dependence matrix for pervasive game qualities that enhances the insight into pervasiveness and reveals important guidelines for the game designers.     

BRIDGING THE GAPS: HYBRID TRACKING FOR ADAPTIVE MOBILE AUGMENTED REALITY

Tracking accuracy in a location-aware mobile system can change dynamically as a function of the user's location and other variables specific to the tracking technologies used. This is especially problematic for mobile augmented reality systems, which ideally require extremely precise position tracking for the user's head, but which may not always be able to achieve that level of accuracy. While it is possible to ignore variable positional accuracy in an augmented reality user interface, this can make for a confusing system; for example, when accuracy is low, virtual objects that are nominally registered with real ones may be too far off to be of use.

To address this problem, we describe an experimental mobile augmented reality system that: (1) employs multiple position-tracking technologies, including ones that apply heuristics based on environmental knowledge; (2) coordinates these concurrently monitored tracking systems; and (3) automatically adapts the user interface to varying degrees of confidence in tracking accuracy. We share our experiences with managing these multiple tracking technologies, employing various techniques to facilitate smooth and reasonable “hand-offs” between the cooperating systems. We present these results in the context of an intelligent navigational guidance system that helps users to orient themselves in an unfamiliar environment, using path planning to guide them toward destinations they choose, and sometimes towards ones the system infers as equally relevant.     

Making Augmented Reality Practical on Mobile Phones, Part 1

Since 2003, the authors have been developing three generations of a software framework for handheld augmented reality (AR), targeting mobile phones. This article is the first part of a discussion on the experiences and design decisions that resulted in the current software framework, Studierstube ES (Embedded System). The authors describe the restrictions of the mobile phone platform and a software architecture that allows fast development of mobile phone AR applications.     

Execution coordination in mobile agent-based distributed job workflow execution

Mobile agent-based distributed job workflow execution requires the use of execution coordination techniques to ensure that an agent executing a subjob can locate its predecessors’ execution results. This paper describes the classification, implementation, and evaluation of execution coordination techniques in the mobile agent-based distributed job workflow execution system. First, a classification of the existing execution coordination techniques is developed for mobile agent systems. Second, to put the discussion into perspective, our framework for mobile agent-based distributed job workflow execution over the Grid (that is, MCCF: Mobile Code Collaboration Framework) is described. How the existing coordination techniques can be applied in the MCCF is also discussed. Finally, a performance study has been conducted to evaluate three coordination techniques using real and simulated job workflows. The results are presented and discussed in the paper.     

Gesture-based interactive augmented reality content authoring system using HMD

This paper proposes an augmented reality content authoring system that enables ordinary users who do not have programming capabilities to easily apply interactive features to virtual objects on a marker via gestures. The purpose of this system is to simplify augmented reality (AR) technology usage for ordinary users, especially parents and preschool children who are unfamiliar with AR technology. The system provides an immersive AR environment with a head-mounted display and recognizes users’ gestures via an RGB-D camera. Users can freely create the AR content that they will be using without any special programming ability simply by connecting virtual objects stored in a database to the system. Following recognition of the marker via the system’s RGB-D camera worn by the user, he/she can apply various interactive features to the marker-based AR content using simple gestures. Interactive features applied to AR content can enlarge, shrink, rotate, and transfer virtual objects with hand gestures. In addition to this gesture-interactive feature, the proposed system also allows for tangible interaction using markers. The AR content that the user edits is stored in a database, and is retrieved whenever the markers are recognized. The results of comparative experiments conducted indicate that the proposed system is easier to use and has a higher interaction satisfaction level than AR environments such as fixed-monitor and touch-based interaction on mobile screens.     

MobileTrust: a trust enhanced security architecture for mobile agent systems

While offering many practical benefits for distributed applications, mobile agent systems pose some fundamental security challenges. In this paper, we present a new approach to mobile agent security which helps to address some of these challenges. We present a new technique, which we refer to as trust enhanced security, and apply it to mobile agent-based systems; this new technique advocates a shift in security solutions from security-centric to trust-centric. This extends the traditional security mechanisms by enabling trust decisions through explicit specification and management of security-related trust relationships. The integration of the trust decisions into security decision-making process leads to our trust enhanced security performance. A formal trust model is proposed and is incorporated into the development of a novel trust management architecture—MobileTrust for mobile agent-based applications. We have conducted detailed practical investigations to evaluate and validate the emergent properties of the trust enhanced security technique. We present and discuss the key results in this paper.