A mixed methods assessment of students' flow experiences during a mobile augmented reality science game

Current studies have reported that secondary students are highly engaged while playing mobile augmented reality (AR) learning games. Some researchers have posited that players' engagement may indicate a flow experience, but no research results have confirmed this hypothesis with vision‐based AR learning games. This study investigated factors related to students' engagement – as characterized by flow theory – during a collaborative AR, forensic science mystery game using mobile devices. School Scene Investigators: The Case of the Stolen Score Sheets is a vision‐based AR game played inside the school environment with Quick Response codes. A mixed methods approach was employed with 68 urban middle school students. Data sources included pre‐ and post‐surveys, field observations and group interviews. Results showed that neither gender nor interest in science was an important predictor of variability in flow experience. Gaming attitude uniquely predicted 23% of the variance in flow experience. Student flow experience features included a flash of intensity, a sense of discovery and the desire for higher performance. The findings demonstrated a potential for mobile AR science games to increase science interest and help students learn collaboration skills. Implications for future research concerning mobile AR science games are discussed.     

Designing and implementing interactive and realistic augmented reality experiences

In this paper, we propose an approach for supporting the design and implementation of interactive and realistic Augmented Reality (AR). Despite the advances in AR technology, most software applications still fail to support AR experiences where virtual objects appear as merged into the real setting. To alleviate this situation, we propose to combine the use of model-based AR techniques with the advantages of current game engines to develop AR scenes in which the virtual objects collide, are occluded, project shadows and, in general, are integrated into the augmented environment more realistically. To evaluate the feasibility of the proposed approach, we extended an existing game platform named GREP to enhance it with AR capacities. The realism of the AR experiences produced with the software was assessed in an event in which more than 100 people played two AR games simultaneously.

     

Augmented reality visualization: A review of civil infrastructure system applications

In Civil Infrastructure System (CIS) applications, the requirement of blending synthetic and physical objects distinguishes Augmented Reality (AR) from other visualization technologies in three aspects: (1) it reinforces the connections between people and objects, and promotes engineers’ appreciation about their working context; (2) it allows engineers to perform field tasks with the awareness of both the physical and synthetic environment; and (3) it offsets the significant cost of 3D Model Engineering by including the real world background. This paper reviews critical problems in AR and investigates technical approaches to address the fundamental challenges that prevent the technology from being usefully deployed in CIS applications, such as the alignment of virtual objects with the real environment continuously across time and space; blending of virtual entities with their real background faithfully to create a sustained illusion of co-existence; and the integration of these methods to a scalable and extensible computing AR framework that is openly accessible to the teaching and research community. The research findings have been evaluated in several challenging CIS applications where the potential of having a significant economic and social impact is high. Examples of validation test beds implemented include an AR visual excavator-utility collision avoidance system that enables workers to “see” buried utilities hidden under the ground surface, thus helping prevent accidental utility strikes; an AR post-disaster reconnaissance framework that enables building inspectors to rapidly evaluate and quantify structural damage sustained by buildings in seismic events such as earthquakes or blasts; and a tabletop collaborative AR visualization framework that allows multiple users to observe and interact with visual simulations of engineering processes.     

Immersive Multiplayer Games With Tangible and Physical Interaction

In this paper, we present a new immersive multiplayer game system developed for two different environments, namely, virtual reality (VR) and augmented reality (AR). To evaluate our system, we developed three game applications-a first-person-shooter game (for VR and AR environments, respectively) and a sword game (for the AR environment). Our immersive system provides an intuitive way for users to interact with the VR or AR world by physically moving around the real world and aiming freely with tangible objects. This encourages physical interaction between players as they compete or collaborate with other players. Evaluation of our system consists of users' subjective opinions and their objective performances. Our design principles and evaluation results can be applied to similar immersive game applications based on AR/VR.     

Sense of Immersion in Computer Games Using Single and Stereoscopic Augmented Reality

As a novel interaction and presentation technology, Augmented Reality (AR) has been widely used in education, tourism, entertainment, psychology, and many other applications. In the game industry, this revolutionary technology has introduced a new form of interaction between players and the gaming environment. In this article, we aim to find if exploiting AR in computer games can enhance the sense of immersion, and consequently, satisfying the players. To this end, a mobile game called Ladybug is implemented using three different techniques including the regular virtual reality (VR) camera, single AR camera, and stereoscopic AR camera. The results of a comprehensive user study on these techniques showed that the single AR camera and stereoscopic AR camera can result in more sense of immersion in players in comparison to the VR camera. However, the hypothesis that the stereoscopic AR results in more sense of presence in comparison to the single AR camera is rejected. On the other hand, our findings show that the AR stereoscopic camera can lead to a better emotional involvement and enjoyment in comparison to the single AR camera and VR camera.     

Tangible authoring of 3D virtual scenes in dynamic augmented reality environment

This paper proposes tangible interfaces and interactions for authoring 3D virtual and immersive scenes easily and intuitively in tangible augmented reality (AR) environment. It provides tangible interfaces for manipulating virtual objects in a natural and intuitive manner and supports adaptive and accurate vision-based tracking in AR environments. In particular, RFID is used to directly integrate physical objects with virtual objects and to systematically support the tangible query of the relation between physical objects and virtual ones, which can provide more intuitive tangibility and a new way of virtual object manipulation. Moreover, the proposed approach offers an easy and intuitive switching mechanism between tangible environment and virtual environment. This paper also proposes a context-adaptive marker tracking method which can remove an inconsistent problem while embedding virtual objects into physical ones in tangible AR environments. The context-adaptive tracking method not only adjusts the locations of invisible markers by interpolating the locations of existing reference markers and those of previous ones, but also removes a jumping effect of movable virtual objects when their references are changed from one marker to another. Several case studies for generating tangible virtual scenes and comparison with previous work are given to show the effectiveness and novelty of the proposed approach.     

A 3D GIS-based interactive registration mechanism for outdoor augmented reality system

Registering virtual objects with reference to their corresponding real-world objects plays a key role in augmented reality (AR) system. Although there have been a lot of work on using vision-based method to perform registration for indoor AR system, it is very difficult to apply such registration method for outdoor AR system due to the inability to modify the objects in outdoor environment and the huge range of working area. 3D Geographic Information System (GIS) is capable of providing an outdoor virtual geographic environment where users are located at, which may provide users with a corresponding virtual object for the one in the physical world. In this study, a 3D GIS-based registration mechanism is proposed for outdoor AR system. Specifically, an easy-use interactive method for precise registration was developed to improve the performance of the registration. To implement the registration mechanism, an outdoor AR system built upon 3D GIS was developed, named Augmented Reality Geographical Information System (ARGIS). ARGIS has the capability of performing precise registration in outdoor environment without using traditional vision tracking method, which thus enables users to arbitrarily manipulate the system. A prototype was developed to conduct experiment on the campus of Peking University, Beijing, China to test the proposed registration mechanism. The experiment shows that the developed registration mechanism is feasible and efficient in the outdoor environment. The ARGIS is expected to enrich the applications of outdoor AR system, including but not limited to underground facility mapping, emergency rescue and urban planning.     

Developing a generic augmented-reality interface

We seek to design an AR interface that provides users with interactivity so rich it would merge the physical space in which we live and work with the virtual space in which we store and interact with digital information. In this single augmented space, computer-generated entities would become first-class citizens of the physical environment. We would use these entities just as we use physical objects, selecting and manipulating them with our hands instead of with a special-purpose device such as a mouse or joystick. Interaction would then be intuitive and seamless because we would use the same tools to work with digital and real objects. Tiles is an AR interface that moves one step closer to this vision. It allows effective spatial composition, layout, and arrangement of digital objects in the physical environment. The system facilitates seamless two-handed, three-dimensional interaction with both virtual and physical objects, without requiring any special-purpose input devices     

DPC：基于增强现实儿童绘本的多视角评估框架

 增强现实(AR)技术作为沟通现实与虚拟场景的桥梁,能够激发使用者的想象力、 空间感和创造力。现有 AR 儿童绘本在虚实结合方面不足、游戏形式和互动种类单一,缺少寓 教于乐的AR儿童绘本游戏作为内容。首先试用和研究了59款移动AR游戏并分类。使用Delphi 法选择了一些合适的游戏类型,并将其迁移到儿童绘本上。再让 6~12 岁儿童(n=6)试玩并进行 观察和访谈。同时,组织家长/老师(n=7)与开发者(n=5)形成焦点小组并针对绘本和应用进行讨 论。最终,提取了开发者(developer)、家长/老师(parents/teachers)和孩子(children)对于 AR 儿童 绘本不同的需求和对应指标,收获了对应 3 个维度的框架 DPC (developer, parents/teachers, children),且每个维度被细分。使用 DPC 框架对 AR 儿童绘本游戏进行了改进。     

A method to assess pervasive qualities in mobile games

Software designers are facing huge challenges imposed by a new generation of applications that mix real and digital worlds, such as pervasive games. This type of game has recently become a worldwide phenomenon, with thousands of people walking in the streets with smartphones to interact with the physical environment. In this paper, we propose a new method to assess pervasive qualities in pervasive mobile games, which can be customized and extended to other ubiquitous applications. This method generates a quality report, which consists of a quality spreadsheet (containing metric values and comments) and a quality vector (representing the game quality profile in the form of a bar chart). In addition, we can compare quality vectors using similarity criteria. In this paper, we apply the proposed method to commercial and academic prototype games to shed more light on their pervasive characteristics and identify ways to improve the overall quality that sets these games apart from traditional digital games—that is, pervasiveness.     

Deep learning-based smart task assistance in wearable augmented reality

Wearable augmented reality (AR) smart glasses have been utilized in various applications such as training, maintenance, and collaboration. However, most previous research on wearable AR technology did not effectively supported situation-aware task assistance because of AR marker-based static visualization and registration. In this study, a smart and user-centric task assistance method is proposed, which combines deep learning-based object detection and instance segmentation with wearable AR technology to provide more effective visual guidance with less cognitive load. In particular, instance segmentation using the Mask R-CNN and markerless AR are combined to overlay the 3D spatial mapping of an actual object onto its surrounding real environment. In addition, 3D spatial information with instance segmentation is used to provide 3D task guidance and navigation, which helps the user to more easily identify and understand physical objects while moving around in the physical environment. Furthermore, 2.5D or 3D replicas support the 3D annotation and collaboration between different workers without predefined 3D models. Therefore, the user can perform more realistic manufacturing tasks in dynamic environments. To verify the usability and usefulness of the proposed method, we performed quantitative and qualitative analyses by conducting two user studies: 1) matching a virtual object to a real object in a real environment, and 2) performing a realistic task, that is, the maintenance and inspection of a 3D printer. We also implemented several viable applications supporting task assistance using the proposed deep learning-based task assistance in wearable AR.     

Interactive 3D content insertion in images for multimedia applications

This article addresses the problem of creating interactive mixed reality applications where virtual objects interact in images of real world scenarios. This is relevant to create games and architectural or space planning applications that interact with visual elements in the images such as walls, floors and empty spaces. These scenarios are intended to be captured by the users with regular cameras or using previously taken photographs. Introducing virtual objects in photographs presents several challenges, such as pose estimation and the creation of a visually correct interaction between virtual objects and the boundaries of the scene. The two main research questions addressed in this article include, the study of the feasibility of creating interactive augmented reality (AR) applications where virtual objects interact in a real world scenario using the image detected high-level features and, also, verifying if untrained users are capable and motivated enough to perform AR initialization steps. The proposed system detects the scene automatically from an image with additional features obtained using basic annotations from the user. This operation is significantly simple to accommodate the needs of non-expert users. The system analyzes one or more photos captured by the user and detects high-level features such as vanishing points, floor and scene orientation. Using these features it will be possible to create mixed and augmented reality applications where the user interactively introduces virtual objects that blend with the picture in real time and respond to the physical environment. To validate the solution several system tests are described and compared using available external image datasets.     

Impact of monitor-based augmented reality for on-site industrial manual operations

This work evaluates the impact of augmented reality (AR) technology to support operators during manual industrial tasks. The work focuses specifically on monitor-based augmented reality as a solution to provide instructions to the operator. Instructions are superimposed directly onto a video representation of the physical workspace and are displayed on a standard monitor. In contrast to previous AR solutions proposed to support manual tasks, the current work is more industrially acceptable because it meets most of the industrial requirements and it is cost effective for a deployment in an industrial environment. The developed prototype is described and evaluated by means of a user study to compare the monitor-based augmented reality solution to provide instructions and the traditional method. The test shows that AR can be a valid substitute to support the operators during manual tasks because it allows them to be more time efficient and it reduces their mental workload compared to traditional instructional manuals.     

Playful training with augmented reality games: case studies towards reality-oriented system design

In this paper, we propose a reality-oriented augmentation approach to support training activities. The approach aims at adding new value and playful features to traditional training environments with keeping their original look-and-feel. For example, a game monitoring service enables to automatically record game events so that players can review a gaming process and strategy for soul-searching, or replay most impressive scenes to share the experience with others after the game finishes. Even several services are running on background, digital devices and services are seamlessly integrated to the game environment in unobtrusive way so that players can concentrate on training as usual. The concept can be applied to both traditional games (e.g., poker and the game of Go) and non-gaming activities (e.g., calligraphy and drumming). We developed four case studies on the concept: Augmented Reality Go, EmoPoker, Augmented Calligraphy and AR Drum Kit. We discuss design issues in the reality-oriented augmentation process based on user study results.     

EcoMOBILE: Integrating augmented reality and probeware with environmental education field trips

Positioned in the context of situated learning theory, the EcoMOBILE project combines an augmented reality (AR) experience with use of environmental probeware during a field trip to a local pond environment. Activities combining these two technologies were designed to address ecosystem science learning goals for middle school students, and aid in their understanding and interpretation of water quality measurements. The intervention was conducted with five classes of sixth graders from a northeastern school district as a pilot study for the larger EcoMOBILE project, and included pre-field trip training, a field trip to a local pond environment, and post-field trip discussions in the classroom.During the field experience, students used mobile wireless devices with FreshAiR™, an augmented reality application, to navigate the pond environment and to observe virtual media and information overlaid on the physical pond. This AR experience was combined with probeware, in that students collected water quality measurements at designated AR hotspots during the experience. We studied the characteristics of learning and instruction using measures of student attitudes, content learning gains, and opinions teachers provided via written and verbal feedback. We observed gains in student affective measures and content understanding following the intervention. Teachers reported that the combined technologies promoted student interaction with the pond and with classmates in a format that was student-centered rather than teacher-directed. Teachers also reported that students demonstrated deeper understanding of the principles of water quality measurement than was typical on prior field trips without these technologies and that students had expanded opportunities to engage in activities that resemble scientific practice. Overall, results of the students' surveys and teacher feedback suggest that there are multiple benefits to using this suite of technologies for teaching and for learning.     

A design and implementation of a framework for games in IoT

Interconnection of the sensing and actuating devices providing the ability to share information across platform through a unified framework for enabling innovative applications. This is achieved by seamless ubiquitous sensing, data analytics and information representation as the unifying framework. Extending the current internet with interconnected objects and devices and their virtual representation has been a growing trend in recent years. Internet of Things (IoT) services are becoming a popular services. This will be supported challenges in a large of aspects such as smart health, green energy, smart home and personalized applications. So, the IoT plays more and more important issue in lifestyle through entertainment such as Games. As of yet, there has not been much research done on IoT environment games as a service. In this paper, we propose schemes of the design and implantation of IT convergence framework for games as a service of IoT. First of all, we discussed what to consider when design and implementation of IT convergence framework for games through contents using user’s mobile devices and various sensors in IoT environment and suggest related techniques. Then, we showed the possibility of games in the IoT environment by creating games and measuring the interactions of users in the IoT environment.     

Mobile learning vs. traditional classroom lessons: a comparative study

Different methods can be used for learning, and they can be compared in several aspects, especially those related to learning outcomes. In this paper, we present a study in order to compare the learning effectiveness and satisfaction of children using an iPhone game for learning the water cycle vs. the traditional classroom lesson. The iPhone game includes multiple interaction forms and combined augmented reality (AR) mini‐games with non‐AR mini‐games. The traditional classroom lesson had the same learning content as the iPhone game. Thirty‐eight children participated in the study. The analyses showed that the children made significant learning gains about the water cycle, regardless of the method used. Even though the results showed that the iPhone method achieved higher knowledge results than the traditional classroom lesson, no statistically significant differences were found between the iPhone and the classroom lesson. When analysing the motivational outcomes, the results showed that the children found the iPhone game to be more satisfying than the classroom lessons. Since the iPhone game achieved similar learning results and a higher motivational effect than the classroom lesson, this suggests that games of this kind could be used as a tool in primary schools to reinforce students' lessons.     

Detecting students' perception style by using games

Knowing students' learning styles allows us to improve their experience in an educational environment. Particularly, the perception style is one of the most important dimensions of the learning styles since it describes the way students perceive the world as well as the kind of learning content they prefer. Several approaches to detect students' perception style according to Felder's model have been proposed. However, these approaches exhibit several limitations that make their implementation difficult. Thus, we propose a novel approach to detect the perception style of a student by analyzing his/her interaction with games, namely puzzle games. To carry out this detection, we track how students play a puzzle game and extract information about this interaction. Then, we train a Naive Bayes Classifier to infer the students' perception style by using the information extracted. We have evaluated our proposed approach with 47 Computer Engineering students. Experimental results showed that the perception style was successfully predicted through the use of games, with an accuracy of 85%. Finally, we conclude that games are a promising environment where the students' perception style can be detected.     

A Synthetic Actor model for long-term computer games

Virtual Reality and Artificial Intelligence provide suitable techniques to improve computer games quality. While the former offers mechanisms to model environment and characters' physical features, the latter provides models and tools for building characters, namely Synthetic Actors or Believable Agents, which can exhibit intelligent social behaviour and express personality and emotions. The current architecture proposals for Synthetic Actors do not fully meet the requirements for long-term games development, such as strategy and adventure ones, it is necessary to guarantee both personality stability and reactive emotional responses, which may be contradictory. In this work, we propose a new Synthetic Actor model that tightly connects emotions and social attitudes to personality, providing a long-term coherent behaviour. This model has been applied to two games presented here as case studies.     

Blowtooth: a provocative pervasive game for smuggling virtual drugs through real airport security

In this paper, we describe a pervasive game, Blowtooth, in which players use their mobile phones to hide virtual drugs on nearby airline passengers in real airport check-in queues. After passing through airport security, the player must find and recover their drugs from the innocent bystanders, without them ever realising they were involved in the game. The game explores the nature of pervasive game playing in environments that are not, generally, regarded as playful or “fun”. This paper describes the game’s design and implementation as well as an evaluation conducted with participants in real airports. It explores the players’ reactions to the game through questionnaire responses and in-game activity. The technologies used in Blowtooth are, intentionally, simple in order for the enjoyment of the game to be reliant more on the physical environment rather than the enabling technologies. We conclude that situating pervasive games in unexpected and challenging environments, such as international airports, may provide interesting and unique gaming experiences for players. In addition, we argue that pervasive games benefit most from using the specific features and nature of interesting real-world environments rather than focusing on the enabling technologies.