Exploring educational immersive videogames: an empirical study with a 3D multimodal interaction prototype

ABSTRACT

Gestural interaction devices emerged and originated various studies on multimodal human–computer interaction to improve user experience (UX). However, there is a knowledge gap regarding the use of these devices to enhance learning. We present an exploratory study which analysed the UX with a multimodal immersive videogame prototype, based on a Portuguese historical/cultural episode. Evaluation tests took place in high school environments and public videogaming events. Two users would be present simultaneously in the same virtual reality (VR) environment: one as the helmsman aboard Vasco da Gama’s fifteenth-century Portuguese ship and the other as the mythical Adamastor stone giant at the Cape of Good Hope. The helmsman player wore a VR headset to explore the environment, whereas the giant player used body motion to control the giant, and observed results on a screen, with no headset. This allowed a preliminary characterisation of UX, identifying challenges and potential use of these devices in multi-user virtual learning contexts. We also discuss the combined use of such devices, towards future development of similar systems, and its implications on learning improvement through multimodal human–computer interaction.     

Wafer-level fabrication process for fully encapsulated micro-supercapacitors with high specific energy

In this paper a wafer-level process is proposed to fully integrate carbon-based micro-supercapacitor onto silicon substrate. This process relies on the deposition of a paste containing carbon, PVDF and acetone into cavities etched in silicon. After electrolyte deposition in a controlled atmosphere, a wafer-level encapsulation is realized. Cyclic voltammetry performed on non-encapsulated micro-components showed specific energy of 257 mJ cm⁻² for 336 μm deep cavities. The specific encapsulation process developed was tested separately and proved to be efficient in terms of resistance to organic electrolytes and mechanical strength.     

Robot task plan representation by Petri nets: modelling, identification, analysis and execution

In this paper we introduce a framework to represent robot task plans based on Petri nets. Our approach enables modelling a robot task, analysing its qualitative and quantitative properties and using the Petri net representation for actual plan execution. The overall model is obtained from the composition of simple models, leading to a modular approach. Analysis is applied to a closed loop between the robot controller and the environment Petri net models. We focus here on the quantitative properties, captured by stochastic Petri net models. Furthermore, we introduce a method to identify the environment and action layer parameters of the stochastic Petri net models from real data, improving the significance of the model. The framework building blocks and a single-robot task model are detailed. Results of a case study with simulated soccer robots show the ability of the framework to provide a systematic modelling tool, and of determining, through well-known analysis methods for stochastic Petri nets, relevant properties of the task plan applied to a particular environment.     

PhotoGeo: a photo digital library with spatial-temporal support and self-annotation

The recent popularity of digital cameras has posed a new problem: how to efficiently store and retrieve the very large number of digital photos captured and chaotically stored in multiple locations without any annotation. This paper proposes an infrastructure, called PhotoGeo, which aims at helping users with the people photo annotation, event photo annotation, storage and retrieval of personal digital photos. To achieve the desired objective, PhotoGeo uses new algorithms that make it possible to annotate photos with the key metadata to facilitate their retrieval, such as: the people who were shown in the photo (who); where it was captured (where); the date and time of capture (when); and the event that was captured. The paper concludes with a detailed evaluation of these algorithms.     

Iris recognition: Analysis of the error rates regarding the accuracy of the segmentation stage

Iris recognition has been widely used in several scenarios with very satisfactory results. As it is one of the earliest stages, the image segmentation is in the basis of the process and plays a crucial role in the success of the recognition task. In this paper we analyze the relationship between the accuracy of the iris segmentation process and the error rates of three typical iris recognition methods. We selected 5000 images of the UBIRIS, CASIA and ICE databases that the used segmentation algorithm can accurately segment and artificially simulated four types of segmentation inaccuracies. The obtained results allowed us to conclude about a strong relationship between translational segmentation inaccuracies – that lead to errors in phase – and the recognition error rates.     

A web-based platform for biosignal visualization and annotation

With the advent of wearable sensing and mobile technologies, biosignals have seen an increasingly growing number of application areas, leading to the collection of large volumes of data. One of the difficulties in dealing with these data sets, and in the development of automated machine learning systems which use them as input, is the lack of reliable ground truth information. In this paper we present a new web-based platform for visualization, retrieval and annotation of biosignals by non-technical users, aimed at improving the process of ground truth collection for biomedical applications. Moreover, a novel extendable and scalable data representation model and persistency framework is presented. The results of the experimental evaluation with possible users has further confirmed the potential of the presented framework.     

Efficient Binary Tomographic Reconstruction

Tomographic reconstruction of a binary image from few projections is considered. A novel heuristic algorithm is proposed, the central element of which is a nonlinear transformation ψ(p)=log(p/(1?p)) of the probability p that a pixel of the sought image be 1-valued. It consists of backprojections based on ψ(p) and iterative corrections. Application of this algorithm to a series of artificial test cases leads to exact binary reconstructions, (i.e., recovery of the binary image for each single pixel) from the knowledge of projection data over a few directions. Images up to 10⁶ pixels are reconstructed in a few seconds. A series of test cases is performed for comparison with previous methods, showing a better efficiency and reduced computation times.     

The ChaLearn gesture dataset (CGD 2011)

This paper describes the data used in the ChaLearn gesture challenges that took place in 2011/2012, whose results were discussed at the CVPR 2012 and ICPR 2012 conferences. The task can be described as: user-dependent, small vocabulary, fixed camera, one-shot-learning. The data include 54,000 hand and arm gestures recorded with an RGB-D \(\hbox {Kinect}^\mathrm{TM}\) camera. The data are organized into batches of 100 gestures pertaining to a small gesture vocabulary of 8–12 gestures, recorded by the same user. Short continuous sequences of 1–5 randomly selected gestures are recorded. We provide man-made annotations (temporal segmentation into individual gestures, alignment of RGB and depth images, and body part location) and a library of function to preprocess and automatically annotate data. We also provide a subset of batches in which the user’s horizontal position is randomly shifted or scaled. We report on the results of the challenge and distribute sample code to facilitate developing new solutions. The data, datacollection software and the gesture vocabularies are downloadable from http://gesture.chalearn.org. We set up a forum for researchers working on these data http://groups.google.com/group/gesturechallenge.     

The CAM-Brain Machine (CBM): An FPGA Based Tool for Evolving a 75 Million Neuron Artificial Brain to Control a Lifesized Kitten Robot

This article introduces the CAM-Brain Machine (CBM), an FPGA based piece of hardware which implements a genetic algorithm (GA) to evolve a cellular automata (CA) based neural network circuit module, of approximately 1,000 neurons, in about a second, i.e., a complete run of a GA, with 10,000s of circuit growths and performance evaluations. Up to 65,000 of these modules, each of which is evolved with a humanly specified function, can be downloaded into a large RAM space, and interconnected according to humanly specified artificial brain architectures. This RAM, containing an artificial brain with up to 75 million neurons, is then updated by the CBM at a rate of 130 billion CA cells per second. Such speeds should enable real time control of robots and hopefully the birth of a new research field that we call brain building. The first such artificial brain, to be built in 2000 and beyond, will be used to control the behaviors of a life sized robotkitten called Robokitty.     

Modeling multi-rate DSP specification semantics for formal transformational design in HOL

The CATHEDRAL Silicon Compilers synthesize hardware for DSP algorithms specified in Silage, a high level applicative language. In order to optimize the results of the silicon compilation in terms of chip-area and/or throughput, the user often massages the specification applying transformations to the Silage code. To guarantee that the transformations preserve the behavior of the specified algorithm, the formal semantics of the specification language had to be defined. The semantics has been used to prove in HOL the correctness of the transformations and to prove properties of the specification. We are currently building a system where a menu of useful andcorrectness preserving transformations will be available to the user. In this system the user could choose appropriate transformations from the menu taking advantage of his creativity and expertise to interactively guide the silicon compiler, without the risk of introducing inconsistencies. This article describes the formalmulti-rate semantics of a substantial subset of Silage and illustrates some formally verified transformations.