数字皮影的交互仿真和动画绘制

目的 从保护和传承传统皮影文化的角度,使用Kinect设备和带有颜色标识的皮影签子,提出一种类似于传统皮影的交互方式来控制数字皮影。方法 首先,利用Kinect获得的深度图以及彩色图并配合训练已得的颜色标识,计算出签子头部的位置,且为每个签子设置卡尔曼滤波器的控制,为皮影动画提供较为稳定的控制信息。其次,将数字皮影组织成3根签子控制特有的关节层次模型。在当前模型下,结合输入的控制信息分析皮影的动作原理,通过OpenGL绘制技术,最终呈现出皮影动画。结果 实验中,用户手执皮影签子面对Kinect,操纵绘制在计算机上的皮影人物,随着签子的移动,皮影可以在构造出的悟空摘桃的皮影场景里进行相应的任意移动、挥舞胳膊、腿部下跪弯曲等动作。用户调查表明该操控方式与传统皮影类似,但该方式更简单,皮影动画也美观有趣味性。结论 仿真传统皮影交互的新的数字皮影的交互方式,对于想体验传统皮影戏表演的非专业人士来说,操作更加容易,且相比于其他数字皮影的交互更有继承传统皮影交互的力度。     

Learning at the digital frontier: a review of digital literacies in theory and practice

This paper describes a literature review, institutional audit and analysis of practice in the area of digital literacy provision, based on research across the UK Higher Education sector. It concludes that institutions need to place greater value on ‘literacies of the digital’, and better prepare their students and their own organizational processes to thrive in an age of digital knowledge practices. It extends the debate about individual entitlement and provision to ask whether digital literacy offers an opportunity for the academy to redefine its relationship to knowledge in society.     

Mobile phone-based mixed reality: the Snap2Play game

The ubiquity of camera phones provides a convenient platform to develop immersive mixed-reality games. In this paper we introduce such a game which is loosely based on the popular card game “Memory”, where players are asked to match a pair of identical cards among a set of overturned cards by revealing only two cards at a time. In our game, the players are asked to match a “digital card”, which corresponds to a scene in a virtual world, to a “physical card”, which is an image of a scene in the real world. The objective is to convey a mixed-reality sensation. Cards are matched with a scene identification engine which consists of multiple classifiers trained on previously collected images. We present our comprehensive overall game design, as well as implementation details and results. We also describe how we constructed our scene identification engine and its performance. Finally, we present an analysis of player surveys to gauge the potential market acceptance.

Social cooperation and competition in the mixed reality space eXperience Induction Machine XIM

Although the architecture of mixed reality spaces is becoming increasingly more complex, our understanding of human behavior in such spaces is still limited. Despite the sophisticated methods deployed in ethology and behavioral biology to track and analyze the actions and movements of animals, we rarely find studies that focus on the understanding of human behavior using such instruments. Here, we address this issue by analyzing social behavior and physical actions of multiple humans who are engaging in a game. As a paradigm of social interaction, we constructed a mixed reality football game in which two teams of two players have to cooperate and compete in order to win. This paradigm was deployed in the, so-called, eXperience Induction Machine (XIM), a human accessible, fully instrumented space that supports full body interaction in mixed reality without the need for body-mounted sensors. Our results show that winning and losing strategies can be discerned by specific behavioral patterns and proxemics. This demonstrates that mixed reality systems such as XIM provide new paradigms for the investigation of human social behavior.     

Music,dimensions and play: composing for autonomous laptop musicians and improvising humans

Abstract

This article discusses three compositions for autonomous laptops performing with improvising musicians. The laptops were programmed with software environments that make narrow-band ‘decisions’ regarding performative soundscapes. They also made ‘decisions’ about visual materials that were presented as onscreen scores for the human improviser to consider. The unfolding aural and visual streams were both functional and creative, and highlighted the here-and-now of indeterminate composition, whilst carefully embedding the aesthetics of the composer within the core of the programming. As such, these laptops were considered as collaborating performers and collaborating composers. The opening section highlights the connecting principles within this collection of works, before discussing each in isolation. This is placed within the context of late twentieth-century and early twenty-first- century composition, whilst offering a framework, for the reading of these, based on Deleuze and Guattari's rhizome philosophy. Three Last Letters is then used as a case study of the concept of wholeness within the computer-generated soundscapes and the algorithmic programming. Section 3 draws together several theories surrounding the mental seeing evoked by sound into dimensionality: a playground for inventive HCI-music improvisation. The values and principles with which the humans and computers interact is developed using creative play theory, followed by some personal responses of the musicians. Finally, an evaluation is offered on the effect on the audience and their perceptual modes when experiencing live and fluid compositions of this nature. The article is concluded with a poetics of experience from the composer's perspective.     

A method of creating a transparent space based on mixed reality

In this article, we propose a novel idea to create a transparent space. Accidents often happen when a person, a motorcycle, a car, etc., comes out suddenly from a blind corner. In such a place, if the shielding objects become transparent, hidden objects reveal themselves. This may contribute to preventing a lot of accidents. The proposed method creates a transparent space by the employment of a mixed-reality technique and a simple three-dimensional (3-D) recovery technique. The 3-D virtual space is created by a single camera image in real-time, and it is superposed onto a real space employing see-through vision. The observer can watch a dynamic scene of the invisible space through the object in real-time. In the experiment performed, an observer could watch persons walking around in a space hidden by an opaque partition. Future application of this technique may include its installation at various corners in buildings or at intersections on roads with bad visibility. This would surely contribute to reducing traffic accidents in future. This work was presented in part at the 11th International Symposium on Artificial Life and Robotics, Oita, Japan, January 23–25, 2006     

Social media,design and creative citizenship: an introduction

ABSTRACT

This article serves as an introduction to a special issue of the journal Digital Creativity, which addresses the challenges, potentials and meaning of different creative forms of citizenship enacted through social media. The articles in this special issue paint a complex picture of the expressions and meaning and citizenship. This introduction aims to provide a wider context for the contributions to this special issue by reviewing some of the claims made for the creative civic potential of social media and identifying a number of issues and questions that might inform analyses of the subject.     

Who but not where: The effect of social play on immersion in digital games

The majority of digital games available today offer a variety of multi-player settings including co-located and mediated play between opponents. Immersion, the sense of being “in the game,” is one of the key components of the gaming experience but existing literature suggests that social play provides more fun but less immersion. There is however little empirical support for this. This paper therefore addresses the question: how does playing digital games in a social situation alter the sense of immersion felt by the individuals playing? This paper presents three experiments that test the relationship between social setting and immersion. The three experiments aim to manipulate the social setting in which players play, be it against a computer, against a person online or against a co-located person. Overall the three experiments show that players are more immersed when playing against another person rather than playing against a computer but there is no significant difference in immersion whether the other person is online or in the same room. This refutes previous claims about social play reducing immersion and indeed that social play enhances the sense of being in the game where interaction is through the game.     

Teaching training in a mixed-reality integrated learning environment

In this mixed-method study, we examined the design and potential impact of a mixed-reality integrated learning environment (MILE) in providing the simulated and immersive teaching practice for university teaching assistants. A virtual-reality-based learning platform integrating a Kinect-enabled sensorimotor interface was developed and used by twenty three university teaching assistants. Qualitative and quantitative data on the participants' participation behaviors, engagement, and perceptions were collected via video/screen recording, interview, surveys on teaching self-efficacy and sense of presence, and eye tracking. Results indicated that the MILE reinforced sense of presence and supported the performance of an ample range of virtual teaching tasks/actions with avatar-embodied live gesturing. The environmental fidelity in the mixed-reality learning spaces, the design and arrangement of virtual agents and avatars, and the affordance of embodied gesturing and walking are salient MILE design features that affected participants' sense of presence and their virtual teaching performance.     

Reflections on interdisciplinary team development for the design of a platform for computer supported collaborative play

The Constructed Narratives project has been designed for use in public spaces where there is the opportunity for individuals and groups of people, who are not acquainted with each other, to encounter the game and subsequently each other. The goal is to provide a platform that supports discourse in environments where ‘keeping comfortable distance’ between oneself and others is the norm. The system framework developed for this project can be applied for use in computer supported collaborative learning (CSCL), and collaborative design activities in the tradition of computer supported collaborative work (CSCW). The current domain explored in the Constructed Narratives project is computer systems designed to enable shared experience through play, or computer supported collaborative play (CSCP). This paper examines the learner-centred design methodologies used for the development of the physical artifacts and underlying software and hardware system architecture for the Constructed Narratives project. The design methodology demonstrates how an interdisciplinary team of artists, designers, and technologists can exploit the opportunities inherent in this ‘symmetry of ignorance’ to find solutions for multiple wicked design problems that can arise during the development of an integrated software and hardware system.     

Augmented reality in language learning: A state-of-the-art review of 2014–2019

This systematic review study synthesizes research findings pertaining to the use of augmented reality (AR) in language learning. Published research from 2014 to 2019 has been explored and specific inclusion and exclusion criteria have been applied resulting in 54 relevant publications. Our findings determined: (a) devices and software employed for mastering AR; languages and contexts in which AR had been applied; theoretical perspectives adopted for guiding the use of AR; the number of participants in AR activities and benefits from using AR as an educational tool in the language classroom; (b) alignment of the affordances of Augmented Reality with the KSAVE (Knowledge, Skills, Attitudes, Values, Ethics) 21st-century skills framework; (c) future directions in AR research and practice. The main findings from this review demonstrate the popularity of mobile-based AR for supporting vocabulary (23.9%), reading (12.7%), speaking (9.9%) writing (8.5%) or generic language skills (9.9%). Our findings also uncovered areas that merit future attention in the application of AR in language learning – for instance learning theories were not often considered in the implementation of AR. The study concludes with suggestions for future research especially in the areas of instructional design and user experience.     

Exploring educational immersive videogames: an empirical study with a 3D multimodal interaction prototype

ABSTRACT

Gestural interaction devices emerged and originated various studies on multimodal human–computer interaction to improve user experience (UX). However, there is a knowledge gap regarding the use of these devices to enhance learning. We present an exploratory study which analysed the UX with a multimodal immersive videogame prototype, based on a Portuguese historical/cultural episode. Evaluation tests took place in high school environments and public videogaming events. Two users would be present simultaneously in the same virtual reality (VR) environment: one as the helmsman aboard Vasco da Gama’s fifteenth-century Portuguese ship and the other as the mythical Adamastor stone giant at the Cape of Good Hope. The helmsman player wore a VR headset to explore the environment, whereas the giant player used body motion to control the giant, and observed results on a screen, with no headset. This allowed a preliminary characterisation of UX, identifying challenges and potential use of these devices in multi-user virtual learning contexts. We also discuss the combined use of such devices, towards future development of similar systems, and its implications on learning improvement through multimodal human–computer interaction.     

Virtual Bounds: a teleoperated mixed reality

This paper introduces a mixed reality workspace that allows users to combine physical and computer-generated artifacts, and to control and simulate them within one fused world. All interactions are captured, monitored, modeled and represented with pseudo-real world physics. The objective of the presented research is to create a novel system in which the virtual and physical world would have a symbiotic relationship. In this type of system, virtual objects can impose forces on the physical world and physical world objects can impose forces on the virtual world. Virtual Bounds is an exploratory study allowing a physical probe to navigate a virtual world while observing constraints, forces, and interactions from both worlds. This scenario provides the user with the ability to create a virtual environment and to learn to operate real-life probes through its virtual terrain.     

Creativity,method and process in digital fabric printing: a 21st century paintbrush

Abstract

This paper analyses how the arrival and widespread use of digital printing affects the artist as maker in the postmodern world of instant reproduction. In response to the question of using digital printing as an artistic tool, a 21^st century paintbrush, the marks and imagery of paintings and drawings of the author's own work produced over the past few years were analysed. The digitally printed fabric outcomes, large scale painterly prints, will be presented as examples of how digitisation becomes a powerfultoolto the textile artist with the creative textile maker manipulating technology and not being controlled by it. The conclusion will show how digitisation and new technologies should not be seen as creating isolation but can be integrated into the textile artist's tool box to inspire and enhance creativity.     

The pataphysics of creativity: developing a tool for creative search

We introduce the idea of a new kind of web search tool that uses the literary and philosophical idea of pataphysics as a conceptual framework in order to return creative results. Pataphysics, the science of exceptions and imaginary solutions, can be directly linked to creativity and is therefore very suitable to guide the transformation from relevant into creative search results. To enable pataphysical algorithms within our system we propose the need for a new type of system architecture. We discuss a component-based software architecture that would allow the flexible integration of the new algorithms at any stage or location and the need for an index suitable to handle patadata, data which have been transformed pataphysically. This tool aims to generate surprising, novel and provocative search results rather than relevant ones, in order to inspire a more creative interaction that has applications in both creative work and learning contexts.     

Role‐play virtual worlds for teaching object‐oriented design: the ViRPlay development experience

The use of game technology for building virtual learning environments is intended to improve the motivation and engagement of the student, borrowing such properties from their entertaining counterparts. Nevertheless, wrapping pedagogical contents in a virtual environment is no simple achievement, and requires a rigorous process of designing and validating the metaphors and mechanics included in the virtual learning system. In this paper we describe such design process exemplified in the construction of consecutive versions of ViRPlay, a 3D role play virtual environment for teaching object‐oriented design. We show how main mechanics were transferred from experiments in the real world and how such mechanics were evolved based on empirical evaluations. Copyright © 2011 John Wiley & Sons, Ltd.     

Creative Control: digital labour,superimposition, datafication,and the image of uncertainty

ABSTRACT

In this paper I will peruse a specific cinematic example of an uncertain-image, i.e. that of the film Creative Control (Dickinson B, dir. 2015. USA: Ghost Robot | Greencard Pictures | Mathematic | Magnolia Pictures | Amazon Studios). My interest in this film lies not, or at least not per se, or not solely, in its representationalism, but rather in its capturing of various kinds of uncertainty, to which I will here attend by situating the film against the backdrop of three different, yet interrelated, problematics related to the ubiquitous presence of digital imaging technologies: i.e., first, the concerns over digital or immaterial labour and the loss of eros; second, the use of contemporary cinematics (and the superimposition effect in particular) to address these and other issues related to living in ‘information-intensive mixed-reality environments;’ and third, the film's own suggestive counter-image, which is that of the characters’ partly defection, and, arguably, that of the image's own withdrawal from the world.     

The Ambient Horn: designing a novel audio-based learning experience

The Ambient Horn is a novel handheld device designed to support children learning about habitat distributions and interdependencies in an outdoor woodland environment. The horn was designed to emit non-speech audio sounds representing ecological processes. Both symbolic and arbitrary mappings were used to represent the processes. The sounds are triggered in response to the childrens location in certain parts of the woodland. A main objective was to provoke children into interpreting and reflecting upon the significance of the sounds in the context in which they occur. Our study of the horn being used showed the sounds to be provocative, generating much discussion about what they signified in relation to what the children saw in the woodland. In addition, the children appropriated the horn in creative ways, trying to scoop up new sounds as they walked in different parts of the woodland.     

Designing creative spaces: an experimental examination of the effect of a nature poster on divergent thinking

Abstract

This paper reports the results of an independent samples experiment designed to examine the effects of the presence of a large poster depicting a natural woodland scene on individual performance on two Divergent Thinking tasks. In comparison to the no-poster control condition, the presentation of a large poster depicting a nature scene was found to lead to greater levels of creativity as rated by judges who were blind to the experimental design. The effects of the large poster on Divergent Thinking were found to hold when controlling for Openness-to-Experience and Mood. Exploratory analyses of participant ratings of room characteristics indicated that the mechanism underlying the posters’ effect related to elevated stimulation.

Practitioner summary: This study compared the effects of presenting a large poster depicting a natural woodland scene (experimental condition) versus no poster (control condition) on individual creative thinking. Three judges, who were unaware of the design of the study, did not know the participant responses were from two different conditions and who did not facilitate the experiment rated the responses of the participants who were exposed to the large poster as significantly more creative.