A context-aware ubiquitous learning environment for language listening and speaking

This paper reported the results of a study that aimed to construct a sensor and handheld augmented reality (AR)-supported ubiquitous learning (u-learning) environment called the Handheld English Language Learning Organization (HELLO), which is geared towards enhancing students' language learning. The HELLO integrates sensors, AR, ubiquitous computing and information technologies. It is composed of two subsystems: an English learning management system and a u-learning tool. In order to evaluate the effects of the proposed learning environment on the learning performance of students, a case study on English learning was conducted on a school campus. The participants included high school teachers and students. A learning course entitled 'My Campus' was conducted in the class; it included three activities, namely 'Campus Environment', 'Campus Life' and 'Campus Story'. The evaluation results showed that the proposed HELLO and the learning activities could improve the students' English listening and speaking skills.     

Construction safety training using immersive virtual reality

Construction workers’ ability to identify and assess risks is acquired through training and experience and is among the key factors that determine their behaviour and thus their safety. Yet researchers have questioned the effectiveness of conventional safety training. This research tested the hypotheses that safety training in a virtual reality (VR) construction site would be feasible and more effective, in terms of workers’ learning and recall in identifying and assessing construction safety risks, than would equivalent training using conventional methods. Sixty-six subjects were provided training in construction safety and their safety knowledge was tested prior to the training, immediately afterward, and one month later. Half of the subjects received traditional classroom training with visual aids; the other half were trained using a 3D immersive VR power-wall. Significant advantage was found for VR training for stone cladding work and for cast-in-situ concrete work, but not for general site safety. VR training was more effective in terms of maintaining trainees’ attention and concentration. Training with VR was more effective over time, especially in the context of cast-in-situ concrete works. Given the need for improved training and the advantages of training using VR, incorporation of VR in construction safety training is strongly recommended.     

Recent Advances and Opportunities of Active Materials for Haptic Technologies in Virtual and Augmented Reality

Virtual reality and augmented reality (VR/AR) are evolving. The market demands and innovation efforts call for a shift in the key VR/AR technologies and engaging people virtually. Tele-haptics with multimodal and bilateral interactions are emerging as the future of the VR/AR industry. By transmitting and receiving haptic sensations wirelessly, tele-haptics allow human-to-human interactions beyond the traditional VR/AR interactions. The core technologies for tele-haptics include multimodal tactile sensing and feedback based on highly advanced sensors and actuators. Recent developments of haptic innovations based on active materials show that active materials can significantly contribute to addressing the needs and challenges for the current and future VR/AR technologies. Thus, this paper intends to review the current status and opportunities of active material-based haptic technology with a focus on VR/AR applications. It first provides an overview of the current VR/AR applications of active materials for haptic sensing and actuation. It then highlights the state-of-the-art haptic interfaces that are relevant to the materials with an aim to provide perspectives on the role of active materials and their potential integration in haptic devices. This paper concludes with the perspective and outlook of immersive multimodal tele-haptic interaction technologies.     

Flexible touch sensor for evaluating geometric properties of virtual shapes through sound

This paper describes the design and implementation of a system for rendering virtual shape through vision, haptic and sound. The system consists of a haptic strip that physically renders virtual curves. A flexible capacitive touch sensor (FCTS) is integrated with the haptic strip, and allows the system to track the position of the user's fingers on the strip. According to the position, the system renders curve properties such as curve shape, inflexion points and curvature through sound metaphors. The goal of this sonification approach is to strengthen the user's understanding of the shape of a virtual prototype, and to inform the user about geometrical attributes that could otherwise remain unseen. Such unseen attributes may either be a result of limitations in the visual and haptic display hardware or a result of limitations in human perception.     

交互叙事视角下智慧博物馆沉浸式体验设计研究

目的 在交互叙事视角下,研究城市智慧博物馆的沉浸式体验设计方法及设计实践。方法 通过案例研究等方法分析智慧博物馆交互叙述的构成要素和特点,进一步探索交互叙事与智慧博物馆沉浸式体验的关系,从构建跨数字媒介的多感官沉浸体验、重构交互叙事要素与关联叙事内容、营造情境感知的叙事空间与互动式故事体验三个方面提出基于交互叙事的智慧博物馆沉浸式体验设计方法,并设计基于交互叙事的智慧博物馆沉浸式体验设计的流程,从实践良渚博物馆沉浸体验项目的角度验证方法的可行性和有效性。结论 设计者通过编码叙事空间和叙事内容,运用跨数字媒介助力多感官沉浸,提取智慧博物馆文化作为叙事主题,优化沉浸交互体验空间,使参观者超越时空限制,与所在城市的历史脉络和文化内涵进行深度互动,从而对城市文化产生更强烈的认同。     

一个沉浸式场景漫游系统的构建

虚拟现实技术,是一种可以创建和体验虚拟环境的计算机系统技术,在军事、游戏、室内设计等诸多领域都有着广泛的应用。一个沉浸式的虚拟现实系统,可以让用户完全沉浸在虚拟世界中,得到与真实世界近似相同的感受。在高精度定位跟踪系统的辅助下,利用visual studio开发环境和3ds max、virtools软件构建了一个沉浸式场景漫游系统,并给出了一个游戏应用示例和一个室内设计应用示例。     

基于心流理论的海派绘画数字文创产品设计

目的 致力于设计出能够让用户产生沉浸式文化体验的海派绘画数字文创产品。方法 在Csikszentmihalyi的心流理论基础上,把海派绘画数字文创产品的设计拆分为设定明确清晰的体验目标、提供准确有效的情感反馈、平衡体验所需技能与挑战这三个步骤。第一步对应数字文创的内容设计,包含清晰的构架体系、可视化界面设计两个步骤,引导用户进入愉悦沉浸的体验状态;第二步对应数字文创的情感设计,包含正面反馈优先、多感官反馈优先两个步骤,保持用户的沉浸式体验状态;第三步对应数字文创的文化体验设计,包含调控体验难度、交互情景化两个设计步骤,引导用户进入文化情境,探索文化背景,获得更高层级的文化体验。结论 心流理论改善了海派绘画数字文创产品的现有不足,完善了海派绘画数字文创产品的设计框架,较好地实现了用文创带动传统美术的理念。     

Image modification for immersive projection display based on pseudo‐projection models

This paper describes a practical method that enables actual images to be converted so that they can be projected onto an immersive projection display (IPD) screen. IPD screens are particularly unique in that their angle of view is extremely wide; therefore, the images projected onto them need to be taken on a special format. In practice, however, it is generally very difficult to shoot images that completely satisfy the specifications of the targeting IPD environment due to cost, technical problems or other reasons. To overcome these problems, we developed a method to modify the images by abandoning geometrical consistency. We were able to utilize this method by assuming that the given image was shot according to a special projection model. Because this model differed from the actual projection model with which the image was taken, we termed it the pseudo‐projection model. Since our method uses simple geometry, and can easily be expressed by a parametric function, the degree of modification or the time sequence for modification can readily be adjusted according to the features of each type of content. Copyright © 2003 John Wiley & Sons, Ltd.     

Unconsciously interactive Films in a cinema environment—a demonstrative case study

‘Many worlds’ is a short narrative live-action film written and directed so as to provide multiple linear routes through the plot to one of four endings, and designed for showing in a cinema environment. At two points during the film, decisions are made based on audience bio-signals as to which plot route to take. The use of bio-signals is to allow the audience to remain immersed in the film, rather than explicitly selecting plot direction. Four audience members have a bio-signal measured sensor for each person: ECG (heart rate), EMG (muscle tension), EEG (‘brain waves’) and Galvanic Skin Response (perspiration). The four are interpreted as a single average of emotional arousal. ‘Many worlds’ was the first live-action linear plotted film to be screened in a cinema to the general public utilizing multiple biosensor types. The film has been shown publically a number of times, and lessons learned from the technical and cinematic production are detailed in this paper.     

首钢赛区灯光展示沉浸体验设计与开发研究

本文是基于《首钢赛区智能化照明环境及灯光展示沉浸体验设计与开发》课题的研究编制,该课题受国家重点研发计划“科技冬奥专项”支持。课题是针对灯光展示沉浸体验设计方法和技术方法的开发研究。设计方面,针对工业遗址转型的问题和需求,对工业精神与体育竞技进行文化重构,设计开发具有冬奥内容、首钢特色的大尺度沉浸式活动体验空间和随境影像展现系统。技术方面,基于数字孪生和仿真模拟,研究智能化、低功耗、强交互的随境影像展现技术。指导建成在复兴城市公共空间、增加运营价值、突出科技生态方面,有引领示范意义的典型项目。