体感交互在大型触摸屏环境中的应用与研究

大型触摸屏环境带来高沉浸感的触控体验,然而当用户离大屏幕过近时体验效果欠佳。为改善其交互体验,引入体感交互技术,基于用户与屏幕的距离提出2种交互模式。通过设计并实现一套测试系统,将体感交互技术应用到大型触摸屏环境中。利用该系统进行用户测试,并对测试结果进行统计学分析。测试结果显示,体感交互技术能有效减少大型触摸屏环境体验效果欠佳的局限性。体感交互技术与大型触摸屏环境的结合有利于改善公共空间的交互体验。     

基于体感交互技术的太阳系三维模型演示系统的研究

体感交互技术能为学习者提供更加自然的人机交互方式,能够给课堂加入信息化的元素。将体感交互技术应用到高中地理教学的课堂上,能丰富教学内容,增强了学习的趣味性,改变了传统的教学模式。本文介绍了体感交互技术的应用现状,对基于体感交互技术的太阳系三维模型演示系统的研究进行了可行性分析,并阐述了体感交互技术应用于教育领域的意义及其前景展望。     

基于Kinect的拳击虚拟训练系统

沙袋击打训练是拳击训练的有效手段,应用体感交互技术和虚拟现实,设计并实现了虚拟的沙袋击打仿真系统。该系统主要包含虚拟场景搭建、体感交互和训练信息系统。拳击场景建立了拳台、沙袋、拳击手套和背景模型,并通过Direct 3D接口加载到虚拟系统中。交互系统通过Kinect体感传感器追踪并获取训练者骨骼关节点坐标,从中提取肘、腕关节建立空间向量并计算击打速度和击打力,控制虚拟拳头击打虚拟沙袋,通过体感交互的方式虚拟真实训练过程,并在信息系统中记录训练信息。实验和测试表明,此系统能够实现模拟训练的功能,为拳击游戏与教学提供了新的手段。     

体感交互技术在《虚拟现实技术》课程教学中的应用

根据虚拟现实技术课程知识综合性强、教学过程创新性需求大和实验工程应用性高的特点,笔者引入流行的体感交互技术,基于微软公司的Kinect镜头,围绕体感数据的捕捉与处理、人体动作的识别和虚拟交互设计三大知识模块,给出了课程中体感交互教学实验项目的原理、技术和开发方法。通过教学实践,本文设计的教学实验项目提高了学生学习的主动性,培养了学生的体感交互技术项目研发能力,增强了学生的创新思维能力和职业素养。     

三维虚拟校园体感交互系统设计与优化研究

三维虚拟校园系统的研究和创建对数字校园的建设有着重要的现实意义。本文在系统分析三维场景构建及虚拟交互技术的基础上,阐述了基于体感交互的三维校园系统开发流程,基于以上研究实现了3dsmax、Unity3D与Kinect平台的三维虚拟校园体感交互系统,该系统具有较高的交互能力、仿真质量和运行效率,最后针对虚拟校园系统的优化与规范提出了设计规范与优化原则。     

基于触觉交互界面的虚拟现实警务训练系统

虚拟现实凭借着其高沉浸感的体验,在警务仿真培训中有着广泛应用,由于商用虚拟现实系统多关注视觉反馈,难以使用标准的手控器为警务培训提供逼真的触觉交互体验。触觉交互界面基于多功能的离散单点触控交互系统,可实现虚拟现实中的道具操作直接交互,针对虚拟训练中的警务场景,该系统能够识别用户所选道具,并响应用户的交互动作,以提高虚拟现实系统的交互真实性。该系统使用HTC Vive 追踪器对用户进行运动捕捉,以生成人体骨架驱动虚拟代理。同时,使用单片机获取并处理用户的道具选择信息及用户与道具的交互信息,再通过蓝牙通信传输至上位机。硬件设计采用 STM32F103 系列中的 C8T6 最小系统板,相关功能通过软件 Keil5 编程实现。实验验证了系统的正确性和可行性,能够提供多样的物理触觉代理,增强了虚拟环境中的交互体验。该系统为虚拟现实交互技术提供了新的思路和方向,具有很好的应用前景。     

VR技术在变电站仿真培训系统中的应用研究

为研究虚拟现实(VR)技术在变电站仿真培训系统中的优势,并分析应用前景,创新性地设计一种结合多种技术的330 kV变电站沉浸式仿真培训系统。该系统包括三维VR引擎技术、多通道视频输出技术、虚拟人驱动技术、Kinect体感交互和漫游技术。在分析系统总体架构的基础上,重点介绍多通道视频输出技术和Kinect体感的交互虚拟技术。330 kV变电站沉浸式仿真培训系统能够显示三维立体逼真效果图,同时能模拟人物的转向和漫游。330 kV变电站沉浸式仿真培训系统引入VR技术后,互动性和真实性更强。Kinect体态识别技术的最佳捕捉范围为2.0～3.5 mm。200次抽样数据中,识别准确率最高和最低的动作分别为左手朝胸部挥动作和手握拳动作,相应的样本数为32和18,识别准确率分别为97%和80%。试验收集到的真实数据能够为今后虚拟式仿真培训系统的完善提供支持。     

教育领域应用体感交互技术的探索

随着科技的高速发展,新的科学技术在不断地冲击着我们生活的各个领域.体感交互技术作为人机互动的新技术,必将在教育领域掀起新一轮革命 多媒体交互式教学系统中应用体感技术,能使教学系统解决现时人机系统的多种问题,具有更强大的交互功能以及更简单的控制,还能更直观地演示教学内容,从而让学生体验到体感技术在多媒体教学中全新的教学感受,使教学中的抽象内容更生动化、形象化和具体化.     

虚拟环境下基于语义的三维交互技术

自然、高效的三维交互技术是虚拟现实系统成功应用的关键.现有的交互技术主要是从几何层次上考虑如何有效实现交互任务,而对面向高层应用的交互任务的支持还不够.借鉴人类在真实世界中的认知原理,虚拟环境中的交互对象不仅具有外观意义上的几何属性,而且包含了与交互有关的规则、约束和供给等语义属性,这些虚拟对象称为语义对象.在系统导航、对象选择/操作等交互任务的执行中,通过语义对象可以实现高层交互语义的封装和解析.从应用角度提高交互技术的效率和可用性,为用户提供"直接操纵"之上的面向高层语义的交互隐喻,屏蔽交互技术的底层实现细节,使用户专注于应用领域相关的高层交互控制.     

基于混合现实交互界面的下颌运动仿真系统

针对传统医学可视化仿真系统中交互界面和交互技术的缺点,基于视觉跟踪技术采用实物模型设计和实现了一个基于混合现实界面的下颌运动仿真系统。通过对物理界面元素、虚拟界面元素以及交互隐喻的分析,为混合现实环境下的交互技术建立了一个具有通用性的分类体系。物理与虚拟的界面元素之间通过语义映射将用户的操作从物理世界映射到虚拟空间中,从而为用户提供一种更为直观、自然的交互方式。     

Gesture recognition and sensorimotor learning‐by‐doing of motor skills in manual professions: A case study in the wheel‐throwing art of pottery

This paper presents a methodological framework for the use of gesture recognition technologies in the learning/mastery of the gestural skills required in wheel‐throwing pottery. In the case of self‐instruction or training, learners face difficulties due to the absence of the teacher/expert and the consequent lack of guidance. Motion capture technologies, machine learning, and gesture recognition may provide a way of overcoming such issues. The proposed methodology is used to record and model expert gestures and then to compare this model in real time with the gestures performed by the learner. Differences in kinematic aspects such as hand distances are detected, and optical/sonic sensorimotor feedback is provided to the learner by the system, alerting him/her when errors occur and guiding him/her to achieve better results. In the case described here, the system was evaluated with 11 learners. With the use of our system, the gestural performance of learners during self‐training has been improved in comparison to cases of self‐training without computer assistance.     

严肃游戏中虚拟角色行为建模综述

严肃游戏是计算机游戏一个新的发展方向,可以提供形象互动的模拟教学环境,已经广泛应用于科学教育、康复医疗、应急管理、军事训练等领域。虚拟角色是严肃游戏中模拟具有生命特征的图形实体,行为可信的虚拟角色能够提升用户使用严肃游戏的体验感。严肃游戏中的图形渲染技术已经逐步成熟,而虚拟角色行为建模的研究尚在初级阶段。可信的虚拟角色必须能够具有感知、情绪和行为能力。本文分别从游戏剧情与行为、行为建模方法、行为学习和行为建模评价等4个方面来分析虚拟角色行为建模研究。分析了有限状态机和行为树的特点,讨论了虚拟角色的行为学习方法。指出了强化学习的关键要素,探讨了深度强化学习的应用途径。综合已有研究,归纳了虚拟角色行为框架,该框架主要包括感觉输入、知觉分析、行为决策和动作4大模块。从情感计算的融入、游戏剧情和场景设计、智能手机平台和多通道交互4个角度讨论需要进一步研究的问题。虚拟角色的行为建模需要综合地考虑游戏剧情、机器学习和人机交互技术,构建具有自主感知、情绪、行为、学习能力、多通道交互的虚拟角色能够极大地提升严肃游戏的感染力,更好地体现寓教于乐。     

Leveraging a personalized system to improve self-directed learning in online educational environments

Many students who participate in online courses experience frustration and failure because they are not prepared for the demanding and isolated learning experience. A traditional learning theory known as self-directed learning (SDL) is a foundation that can help establish features of a personalized system that helps students improve their abilities to manage their overall learning activities and monitor their own performance. Additionally, the system enables collaboration, interaction, feedback, and the much-needed support from the instructor and students' peers. A Web 2.0 social-technology application, MediaWiki, was adopted as the platform from which incremental features were developed to utilize the fundamental concepts of SDL. Students were able to customize content by setting specific learning goals, reflecting on their learning experiences, self-monitoring activities and performances, and collaborating with others in the class. SDL skills exist to some degree in all learners, this study finds that students' SDL abilities can improve when a course adopts a personalized and collaborative learning system that enables the students to be more proactive in planning, organizing, and monitoring their course activities.     

基于Kinect的虚拟展馆游览系统的设计与实现

针对现有三维虚拟展览馆游览系统交互方式用户体验不理想的现状,提出一种基于Kinect的虚拟展馆游览系统。该系统重点研究了利用Kinect骨骼追踪技术实现对体验者身体动作的解析和利用四元数插值对骨骼动画的控制。在功能上,实现了站在Kinect前的体验者通过不同的姿势,对虚拟展馆三维场景中虚拟人物的动作、姿态、位置、朝向的控制。实验结果表明,体验者能够通过体感交互技术较好地实现对虚拟人物姿态的控制并方便地进行虚拟场景游览。     

Iterative augmentation of a medical training simulator: Effects of affective metacognitive scaffolding

Experiential training simulators are gaining increasing popularity for job-related training due to their potential to engage and motivate adult learners. They are designed to provide learning experiences that are directly connected to users' work environments and support self-regulated learning. Nevertheless, learners often fail to transfer the knowledge gained in the simulated environment to real-world contexts. The EU-funded ImREAL project aimed to bridge that gap by developing a suite of intelligent services designed to enhance existing training simulators. This paper presents work that was a subset of this research project, reporting the iterative development and evaluation of a scaffolding service, which was integrated into a simulator for training medical students to perform diagnostic interviews. The study comprises three evaluation phases, comparing the pure simulator to a first version with metacognitive scaffolding and then to a final version with affective metacognitive scaffolding and enriched user modelling. The scaffolding service provides the learner with metacognitive prompts; affective elements are realized by an integrated affect reporting tool and affective prompts. Using a mixed-method approach by analysing questionnaires (N = 106) and log-data (N = 426), the effects of the services were investigated with respect to real-world relevance, self-regulated learning support, learning experience, and integration. Despite some limitations, the outcomes of this study demonstrate the usefulness of affective metacognitive scaffolding in the context of experiential training simulators; significant post-simulation increases in perceived relevance of the simulator, reflective note-taking, overall motivation, and feeling of success could be identified. Perceived usability and flow of the simulation increased, whereas overall workload and frustration decreased. However, low response rates to specific functions of the simulation point to a need to further investigate how to raise users' awareness and understanding of the provided tools, to encourage interaction with the services, and to better convey the benefits of using them. Thus, future challenges concern not so much technological developments for personalizing learning experiences, but rather new ways to change user attitudes towards an open approach to learning systems that enables them to benefit from all offered features.     

Perceived usefulness of,engagement with,and effectiveness of virtual reality environments in learning industrial operations: the moderating role of openness to experience

The development of virtual reality (VR) in enhancing the effectiveness of the learning process, with its interactive, immersive, and intuitive pedagogical environment, has become a necessity for corporations with increasingly complex operations. However, VR users’ perceptions, openness and learning effectiveness are seldom comprehensively evaluated, particularly in learning complex industrial operations. In this study, grounded in the technology acceptance model, a moderated mediation model of perceived usefulness, ease of use, openness to experience, and engagement in VR-based learning was developed. The model was empirically validated using responses collected from 321 users who were trained on aircraft and cargo terminal operations powered by a novel VR-based learning platform. A survey to measure openness to experience and a pre-training performance test were carried out, followed by a post-training survey of learners’ intrinsic factors, including the influence of perceived usefulness, openness to experience, and attitude towards learning. The study revealed that learners with an open attitude towards experiencing new technology tend to perceive VR technology as a useful platform for training. In addition, the learners with more positive views of VR technology-supported training were more engaged in learning.

     

Age related differences in learning to use a text-editing system

The increased use of computer technology in most occupations means that many middle-aged and older workers who lack computer experience will need to acquire skills to interact with this technology. This study examined age-group differences in learning to use a text-editing system. One hundred and thirty-five females ranging in age from 25 to 70 participated in the study. They received training under one of three training conditions: instructor-based, online or manual-based. Age differences were assessed by comparing performance on criterion tasks. Results indicated significant differences among the groups in learning success, younger learners being the most successful. There were no significant age by training interactions. The findings indicate a need to develop more effective training strategies to teach older learners to use computer technologies. They also indicate the need for changes in system design because computer applications are difficult for novice users to learn.     

Automatic Schaeffer's gestures recognition system

Schaeffer's sign language consists of a reduced set of gestures designed to help children with autism or cognitive learning disabilities to develop adequate communication skills. Our automatic recognition system for Schaeffer's gesture language uses the information provided by an RGB‐D camera to capture body motion and recognize gestures using dynamic time warping combined with k‐nearest neighbors methods. The learning process is reinforced by the interaction with the proposed system that accelerates learning itself thus helping both children and educators. To demonstrate the validity of the system, a set of qualitative experiments with children were carried out. As a result, a system which is able to recognize a subset of 11 gestures of Schaeffer's sign language online was achieved.     

Using augmented reality to support a software editing course for college students

This study aimed to explore whether integrating augmented reality (AR) techniques could support a software editing course and to examine the different learning effects for students using online‐based and AR‐based blended learning strategies. The researcher adopted a comparative research approach with a total of 103 college students participating in the study. The experimental group (E.G.) learned with the AR‐based contents, while the control group (C.G.) learned with the online‐based support. The findings demonstrated the potential of AR techniques for supporting students' learning motivation and peer learning interaction, and the AR‐based contents could be used as scaffolding to better support blended learning strategies. The AR‐based learning interaction could also be a trigger arousing learners' interest in becoming active learners and the students presented great learning involvement after the AR‐based supports were removed, while the learners in the C.G. were passive once the supports had been removed. Moreover, it was found that (1) their lack of experience with AR interaction and applications, (2) the slow speed of the Internet in the school, (3) the affordances of each learner's mobile learning devices, (4) the screen size of the learning interface and (5) the overloading of the learning information from the AR contents and teacher lectures might be the reasons why the learners were still more used to the online‐based support. It was therefore concluded that when integrating AR applications into a course, technology educational researchers should take into careful consideration the target learning content design, the amount of information displayed on the mobile screen and the affordances of the learning equipment and classroom environment so as to achieve a suitable learning scenario.     

Journeys up the mountain: Different paths to learning object-oriented programming

Among the challenges facing companies transitioning from structured to object-oriented (OO) programming is how (and whether) to retrain existing procedural programmers. Common wisdom has it that old-time programmers can be retrained in object technology only with great difficulty, but new programmers lack experience building large systems and the knowledge of the business. This paper describes a study of students learning OO programming where the participants ranged in experience from a single semester of programming to over 10 years of professional programming. The purpose of this study was to explore how students learn OO programming by observing them between their first exposure to OO programming and the time they finally “get it.” We identified five categories of learners who each took a different path to learning OO programming, encountered different obstacles, and adopted different learning strategies. We describe some factors that may play a part in helping and/or hindering a student's progress and that may be used to predict a student's learning category. We conclude with suggestions for alternative training program strategies that may be appropriate for each category and with directions for future research.