Exploring educational immersive videogames: an empirical study with a 3D multimodal interaction prototype

ABSTRACT

Gestural interaction devices emerged and originated various studies on multimodal human–computer interaction to improve user experience (UX). However, there is a knowledge gap regarding the use of these devices to enhance learning. We present an exploratory study which analysed the UX with a multimodal immersive videogame prototype, based on a Portuguese historical/cultural episode. Evaluation tests took place in high school environments and public videogaming events. Two users would be present simultaneously in the same virtual reality (VR) environment: one as the helmsman aboard Vasco da Gama’s fifteenth-century Portuguese ship and the other as the mythical Adamastor stone giant at the Cape of Good Hope. The helmsman player wore a VR headset to explore the environment, whereas the giant player used body motion to control the giant, and observed results on a screen, with no headset. This allowed a preliminary characterisation of UX, identifying challenges and potential use of these devices in multi-user virtual learning contexts. We also discuss the combined use of such devices, towards future development of similar systems, and its implications on learning improvement through multimodal human–computer interaction.     

Automatic Schaeffer's gestures recognition system

Schaeffer's sign language consists of a reduced set of gestures designed to help children with autism or cognitive learning disabilities to develop adequate communication skills. Our automatic recognition system for Schaeffer's gesture language uses the information provided by an RGB‐D camera to capture body motion and recognize gestures using dynamic time warping combined with k‐nearest neighbors methods. The learning process is reinforced by the interaction with the proposed system that accelerates learning itself thus helping both children and educators. To demonstrate the validity of the system, a set of qualitative experiments with children were carried out. As a result, a system which is able to recognize a subset of 11 gestures of Schaeffer's sign language online was achieved.     

On improving the efficiency of tensor voting

This paper proposes two alternative formulations to reduce the high computational complexity of tensor voting, a robust perceptual grouping technique used to extract salient information from noisy data. The first scheme consists of numerical approximations of the votes, which have been derived from an in-depth analysis of the plate and ball voting processes. The second scheme simplifies the formulation while keeping the same perceptual meaning of the original tensor voting: The stick tensor voting and the stick component of the plate tensor voting must reinforce surfaceness, the plate components of both the plate and ball tensor voting must boost curveness, whereas junctionness must be strengthened by the ball component of the ball tensor voting. Two new parameters have been proposed for the second formulation in order to control the potentially conflictive influence of the stick component of the plate vote and the ball component of the ball vote. Results show that the proposed formulations can be used in applications where efficiency is an issue since they have a complexity of order O(1). Moreover, the second proposed formulation has been shown to be more appropriate than the original tensor voting for estimating saliencies by appropriately setting the two new parameters.     

Randomization can be a healer: consensus with dynamic omission failures

Wireless ad-hoc networks are being increasingly used in diverse contexts, ranging from casual meetings to disaster recovery operations. A promising approach is to model these networks as distributed systems prone to dynamic communication failures. This captures transitory disconnections in communication due to phenomena like interference and collisions, and permits an efficient use of the wireless broadcasting medium. This model, however, is bound by the impossibility result of Santoro and Widmayer, which states that, even with strong synchrony assumptions, there is no deterministic solution to any non-trivial form of agreement if n ? 1 or more messages can be lost per communication round in a system with n processes. In this paper we propose a novel way to circumvent this impossibility result by employing randomization. We present a consensus protocol that ensures safety in the presence of an unrestricted number of omission faults, and guarantees progress in rounds where such faults are bounded by ${f \,{\leq}\,\lceil \frac{n}{2} \rceil (n\,{-}\,k)\,{+}\,k\,{-}\,2}$ , where k is the number of processes required to decide, eventually assuring termination with probability 1.     

Visualization of level-of-detail meshes on the GPU

Extensive research has been carried out in multiresolution models for many decades. The tendency in recent years has been to harness the potential of GPUs to perform the level-of-detail extraction on graphics hardware. The aim of this work is to present a new level-of-detail scheme based on triangles which is both simple and efficient. In this approach, the extraction process updates vertices instead of indices, thus providing a perfect framework for adapting the algorithms to work completely on GPU shaders. One of the key aspects of our proposal is the need for just a single rendering pass in order to obtain the desired geometry. Moreover, coherence among the different approximations is maximized by means of a symmetric extraction algorithm, which performs the same process when refining and coarsening the mesh. Lastly, we also introduce different uses of the scheme to offer continuous and view-dependent resolution.     

Applying intelligent agents and semantic web services in eGovernment environments

The increasing volume of eGovernment‐related services is demanding new approaches for service integration and interoperability in this domain. Semantic web (SW) technologies and applications can leverage the potential of eGovernment service integration and discovery, thus tackling the problems of semantic heterogeneity characterizing eGovernment information sources and the different levels of interoperability. eGovernment services will therefore be semantically described in the foreseeable future. In an environment with semantically annotated services, software agents are essential as the entities responsible for exploiting the semantic content in order to automate some tasks, and so enhance the user's experience. In this paper, we present a framework that provides a seamless integration of semantic web services and intelligent agents technologies by making use of ontologies to facilitate their interoperation. The proposed framework can assist in the development of powerful and flexible distributed systems in complex, dynamic, heterogeneous, unpredictable and open environments. Our approach is backed up by a proof‐of‐concept implementation, where the breakthrough of integrating disparate eGovernment services has been tested.     

A hierarchical, agent-based service oriented architecture for smart environments

Service Oriented Architectures, which allow for the integration of different subsystems and of applications running on different devices and platforms, may be very suitable to solve the problem of service personalization in large smart environments like cities, where the number of potential users and potentially available services is rapidly growing. Taking this into account, we have designed an agent-based service oriented architecture for smart spaces. Two of the greatest challenges in the design of such a solution are providing effective device, service and context federation and composition mechanisms and handling user mobility. For the first challenge, we have designed a hierarchical architecture and developed a set of inheritance, aggregation and access mechanisms for devices, services and context. To handle user mobility, we have followed three different strategies, using stationary, mobile or nomadic agents. In this paper, we describe the main aspects of our architecture and perform an experimental evaluation to determine the advantages and drawbacks of the different strategies.     

Gaussian mean-shift is an EM algorithm

The mean-shift algorithm, based on ideas proposed by Fukunaga and Hosteller, is a hill-climbing algorithm on the density defined by a finite mixture or a kernel density estimate. Mean-shift can be used as a nonparametric clustering method and has attracted recent attention in computer vision applications such as image segmentation or tracking. We show that, when the kernel is Gaussian, mean-shift is an expectation-maximization (EM) algorithm and, when the kernel is non-Gaussian, mean-shift is a generalized EM algorithm. This implies that mean-shift converges from almost any starting point and that, in general, its convergence is of linear order. For Gaussian mean-shift, we show: 1) the rate of linear convergence approaches 0 (superlinear convergence) for very narrow or very wide kernels, but is often close to 1 (thus, extremely slow) for intermediate widths and exactly 1 (sublinear convergence) for widths at which modes merge, 2) the iterates approach the mode along the local principal component of the data points from the inside of the convex hull of the data points, and 3) the convergence domains are nonconvex and can be disconnected and show fractal behavior. We suggest ways of accelerating mean-shift based on the EM interpretation