Physiology-based affect recognition for computer-assisted intervention of children with Autism Spectrum Disorder

Generally, an experienced therapist continuously monitors the affective cues of the children with Autism Spectrum Disorders (ASD) and adjusts the course of the intervention accordingly. In this work, we address the problem of how to make the computer-based ASD intervention tools affect-sensitive by designing therapist-like affective models of the children with ASD based on their physiological responses. Two computer-based cognitive tasks are designed to elicit the affective states of liking, anxiety, and engagement that are considered important in autism intervention. A large set of physiological indices are investigated that may correlate with the above affective states of children with ASD. In order to have reliable reference points to link the physiological data to the affective states, the subjective reports of the affective states from a therapist, a parent, and the child himself/herself were collected and analyzed. A support vector machines (SVM)-based affective model yields reliable prediction with approximately 82.9% success when using the therapist's reports. This is the first time, to our knowledge, that the affective states of children with ASD have been experimentally detected via physiology-based affect recognition technique.     

Enhancing social communication of children with high-functioning autism through a co-located interface

This study evaluated the effectiveness of a 3-week intervention in which a co-located cooperation enforcing interface, called StoryTable, was used to facilitate collaboration and positive social interaction for six children, aged 8–10 years, with autistic spectrum disorder (ASD). The intervention focused on exposing pairs of children to an enforced collaboration paradigm while they narrated a story. Pre- and post-intervention tasks included a “low technology” version of the storytelling device and a non storytelling play situation using a free construction game. The outcome measure was a structured observation scale of social interaction. Results demonstrated progress in three areas of social behaviors. First, the participants were more likely to initiate positive social interaction with peers after the intervention. Second, the level of shared play of the children increased from the pre-test to the post-test and they all increased the level of collaboration following the intervention. Third, the children with ASD demonstrated lower frequencies of autistic behaviors while using the StoryTable in comparison to the free construction game activity. The implications of these findings are discussed in terms of the effectiveness of this intervention for higher functioning children with ASD.     

Child–robot interaction across cultures: How does playing a game with a social robot compare to playing a game alone or with a friend?

The present study investigates how children from two different cultural backgrounds (Pakistani, Dutch) and two different age groups (8 and 12 year olds) experience interacting with a social robot (iCat) during collaborative game play. We propose a new method to evaluate children’s interaction with such a robot, by asking whether playing a game with a state-of-the-art social robot like the iCat is more similar to playing this game alone or with a friend. A combination of self-report scores, perception test results and behavioral analyses indicate that child–robot interaction in game playing situations is highly appreciated by children, although more by Pakistani and younger children than by Dutch and older children. Results also suggest that children enjoyed playing with the robot more than playing alone, but enjoyed playing with a friend even more. In a similar vein, we found that children were more expressive in their non-verbal behavior when playing with the robot than when they were playing alone, but less expressive than when playing with a friend. Our results not only stress the importance of using new benchmarks for evaluating child–robot interaction but also highlight the significance of cultural differences for the design of social robots.     

NAO机器人在自闭症干预中的应用

描述了自闭症谱系障碍（ASD）患儿基于人形机器人NAO人机互动情况下和在普通课堂环境下的行为反应,并给出其在干预治疗及普通课堂环境下的评估结果。大量实验结果表明,基于NAO机器人的人机互动干预过程中患儿自闭症状行为较普通课堂大大减少,其症状得到有效控制。因此可得出结论：人形机器人NAO可以作为帮助自闭症谱系障碍患儿干预治疗的平台。     

Expressing and interpreting emotional movements in social games with robots

This paper provides a framework for recording, analyzing and modeling of 3 dimensional emotional movements for embodied game applications. To foster embodied interaction, we need interfaces that can develop a complex, meaningful understanding of intention—both kinesthetic and emotional—as it emerges through natural human movement. The movements are emulated on robots or other devices with sensory-motor features as a part of games that aim improving the social interaction skills of children. The design of an example game platform that is used for training of children with autism is described since the type of the emotional behaviors depends on the embodiment of the robot and the context of the game. The results show that quantitative movement parameters can be matched to emotional state of the embodied agent (human or robot) using the Laban movement analysis. Emotional movements that were emulated on robots using this principle were tested with children in the age group 7–9. The tests show reliable recognition on most of the behaviors.     

Serious Game for Children with Autism to Learn Vocabulary: An Experimental Evaluation

The use of serious games (SGs) to provide intervention for various skills to children with autism spectrum disorder (ASD) has increased in the recent years. However, the potential of these games has not been utilized to provide learning of vocabulary to children with ASD. In designing games for children with ASD, there is a need to consider components specifically from the perspectives of ASD and the generic components that can support from the theoretical, content, and game design aspects. In the absence of any existing framework to design games for children with ASD, this article first introduces a framework named serious game design framework (SGDF), constructed to provide support throughout the design process. This framework was constructed based on the extensive review of the literature on ASD and existing SGDFs. The article then focuses on the design and development of a SG prototype based on SGDF. An experimental evaluation prototype was conducted to examine the effectiveness using the withdrawal design of single-subject research design (SSRD) in improving the receptive identification of vocabulary items among children with autism before and after using the prototype. The receptive identification of vocabulary items is measured in terms of the number of correct responses given and the number of attempts made to identify the correct response. The pre- and post-evaluations of the SG prototype show that learning of vocabulary items among children with ASD improved after using the game and they retained items at the end of weeks 1 and 2 following the withdrawal of intervention.     

Communication using robots: a Perception-action scenario in moderate ASD

ASD children are characterised by a lack of intentionality. We analysed nonverbal and verbal information, associated with heart rate and emotional feeling, respectively, in ASD and neurotypical children. Analogies of heart rate between ASD and neurotypical children were expressed when the human was ‘passive’ actor and the robot was ‘active’ actor; disanalogies were released when the human was ‘active’ actor. Only ASD children reported better emotional feeling ‘after’ than ‘before’ the interaction with the robot. These results suggest that ASD children might be more reliable to low-level intentionality represented by robots, than to high-level intentionality associated with humans.     

An inclusive design approach for developing video games for children with Autism Spectrum Disorder

The efficacy of therapeutic treatments for Autism Spectrum Disorder is mainly associated with the treatment's intensity in terms of weekly hours. This has led mental health professionals to explore the use of video games to complement traditional treatments. However, major weaknesses have been found due to poor game design, which has failed to fulfill therapeutic objectives or to properly engage children. These weaknesses are typically characterized by the poor integration of knowledge from mental health experts, children's interests and designers' expertise. Starting from this necessity, we propose an inclusive design approach to develop therapeutic games. The method presents strategies to integrate the expertise of clinicians, contributions of children and experience of designers through a set of elicitation and merging techniques. The goal of this method is to design games that are effective in terms of therapeutic objectives and that are enjoyable for children. To describe this method, we present its application in the design and development of a Kinect-based game for high-functioning children with ASD called “Pico's Adventures”. This game aims at promoting social initiation in young children with autism. Findings from a first exploratory study with 10 children with ASD showed the effectiveness of the game in eliciting social initiation behaviors. This provides a first validation of the method. An essential aspect of the game's success was the use of elements and mechanics that were appealing for the children. As a result, we have identified effective design concepts and paths for further research on games for children with ASD.     

Designing computer-based rewards with and for children with Autism Spectrum Disorder and/or Intellectual Disability

Children with Autism Spectrum Disorder (ASD) tend to have an affinity for digital technologies, often preferring computer-assisted learning to human-assisted learning. Many children with ASD are also diagnosed with Intellectual Disabilities (ID), yet design studies involving children with ASD and ID are scarce. Rewards can have a positive impact on children's learning and motivation, but little is known about the nature and impact of rewards for children with ASD, and/or ID. Digital technologies are well placed to provide task-based rewards, and in combination with a better understanding of the reward preferences of children with ASD and/or ID this has significant potential to enhance learning. This paper presents two robust participatory design (PD) studies involving children with: i) ASD; ii) ID; and iii) both ASD and ID. The studies aimed to identify: i) the reward preferences of children with ASD and/or ID (RQ1) and ii) how rewards might develop throughout a task as the child progresses (RQ2). Results revealed a number of reward categories that were common to all children, as well as children's preferences for how rewards could develop as they progress through computer-based tasks, for the first time. Original implications for designing computer-based rewards embedded within digital intervention/educational technologies for children with ASD and/or ID, are discussed.     

Outcomes for design and learning when teenagers with autism codesign a serious game: A pilot study

Digital educational games research tends to lack ecological validity by not adequately taking into account the views and perspectives of children and young people with autism spectrum disorders (ASD). This paper is a pilot study that explores and analyses an academic‐based educational game that was co‐designed with and for young people with ASD. The serious game aims to help the players learn Geography‐specific knowledge and integrates several strategic features so that users can collaborate together against the computer or compete against each other. The educational game was evaluated over 5 sessions by 3 peer teams from 2 different special educational institutions, involving a total of 6 students with ASD. The participants were positive about their enjoyment, motivation, and social engagement. The results showed that the players' level of competitiveness not only influenced the experience within the game but also the interaction within the peer teams. The game mechanisms did help the participants with ASD increase their knowledge in Geography content. The main conclusion is that there are considerable benefits of including children with ASD in the design process and future research should explore more fully on how their involvement can enhance curriculum‐based learning as well as social engagement within the classroom.     

Impact of mainstream classroom setting on attention of children with autism spectrum disorder: an eye-tracking study

It has long been reported that children with autism spectrum disorder (ASD) exhibit attention difficulties while learning. They tend to focus on irrelevant information and can easily be distracted. As a result, they are often confined to a one-to-one teaching environment, with fewer distractions and social interactions than would be present in a mainstream educational setting. In recent years, inclusive mainstream schools have been growing in popularity due to government policies on equality rights. Therefore, it is crucial to investigate attentional patterns of children with ASD in mainstream schools. This study aims to explore the attentional behaviors of children with ASD in a virtual reality simulated classroom. We analyzed four eye-gaze behaviors and performance scores of 45 children: children with ASD (ASD n = 20) and typically developing children (TD n = 25) when performing attention tasks. The gaze behaviors included time to first fixate (TTFF), first fixation duration (FFD), average fixation duration (AFD) and the sum of fixation count (SFC) on fourteen areas of interest (AOIs) in the classroom. Our results showed that children with ASD exhibit similar gaze behaviors to TD children, but with significantly lower performance scores and SFC on the target AOI. These findings showed that classroom settings can influence attentional patterns and the academic performance of children with ASD. Further studies are needed on different modalities for supporting the attention of children with ASD in a mainstream setting.

     

A toy or a friend? Children's anthropomorphic beliefs about robots and how these relate to second-language word learning

This study investigates the degree to which children anthropomorphize a robot tutor and whether this anthropomorphism relates to their vocabulary learning in a second-language (L2) tutoring intervention. With this aim, an anthropomorphism questionnaire was administered to 5-year-old children (N = 104) twice: prior to and following a seven-session L2 vocabulary training with a humanoid robot. On average, children tended to anthropomorphize the robot prior to and after the lessons to a similar degree, but many children changed their attributed anthropomorphic features. Boys anthropomorphized the robot less after the lessons than girls. Moreover, there was a weak but significant positive correlation between anthropomorphism as measured before the lessons and scores on a word-knowledge post-test administered the day after the last lesson. There was also a weak but significant positive correlation between the change in anthropomorphism over time and scores on a word-knowledge post-test administered approximately 2 weeks after the last lesson. Our results underscore the need to manage children's expectations in robot-assisted education. Also, future research could explore adaptations to individual children's expectations in child-robot interactions.     

Entering Aladdin's cave: Developing an app for children with Down syndrome

Tablets have much to offer children with learning difficulties, but evidence of their effectiveness to teach academic skills is limited and cannot be easily separated from the quality of the software. This paper analyses data from 3 iterative cycles of designing an app for children with Down syndrome to support their awareness of quantity through an inclusive game. Research with neurotypical children suggests that representation of quantity (or magnitude) is an area with considerable potential in supporting the foundations for children's mathematical learning. It has received little attention as an aspect for intervention for children with Down syndrome. Data collected in this study illustrate the need to carefully align the game mechanic to the target skills, strengthen levels of access, and introduce gradations of attentional demand. They also signal the interrelationship between children's cognitive and affective responses to the game, making it essential to find the optimal level of challenge. Children's strategies in response to mistakes indicate the importance of creating an agile responsive system. The data also suggest that developers routinely extend the number of features that are optional, enabling a greater level of personalization and a more inclusive game.     

A Two-Month Field Trial in an Elementary School for Long-Term Human–Robot Interaction

Interactive robots participating in our daily lives should have the fundamental ability to socially communicate with humans. In this paper, we propose a mechanism for two social communication abilities: forming long-term relationships and estimating friendly relationships among people. The mechanism for long-term relationships is based on three principles of behavior design. The robot we developed, Robovie, is able to interact with children in the same way as children do. Moreover, the mechanism is designed for long-term interaction along the following three design principles: (1) it calls children by name using radio frequency identification tags; (2) it adapts its interactive behaviors for each child based on a pseudo development mechanism; and (3) it confides its personal matters to the children who have interacted with the robot for an extended period of time. Regarding the estimation of friendly relationships, the robot assumes that people who spontaneously behave as a group together are friends. Then, by identifying each person in the interacting group around the robot, it estimates the relationships between them. We conducted a two-month field trial at an elementary school. An interactive humanoid robot, Robovie, was placed in a classroom at the school. The results of the field trial revealed that the robot successfully continued interacting with many children for two months, and seemed to have established friendly relationships with them. In addition, it demonstrated reasonable performance in identifying friendships among children. We believe that these results demonstrate the potential of current interactive robots to establish social relationships with humans in our daily lives.     

Adaptive Social Robot for Sustaining Social Engagement during Long-Term Children–Robot Interaction

One of the known challenges in Children–Robot Interaction (cHRI) is to sustain children’s engagement during long-term interactions with robots. Researchers have hypothesized that robots that can adapt to children’s affective states and can also learn from the environment can result in sustaining engagement during cHRI. Recently, researchers have conducted a range of studies where robots portray different social capabilities and have shown that it has positively influenced children’s engagement. However, despite an immense body of research on implementation of different adaptive social robots, a pivotal question remains unanswered: Which adaptations portrayed by a robot can result in maintaining long-term social engagement during cHRI? In other words, what are the appropriate and effective adaptations portrayed by a robot that will sustain social engagement for an extended number of interactions? In this article, we report on a study conducted with three groups of children who played a snakes and ladders game with the NAO robot to address the aforementioned question. The NAO performed 1) game-based adaptations, 2) emotion-based adaptations, and 3) memory-based adaptation. Our results showed that emotion-based adaptations were found out to be most effective, followed by memory-based adaptations. Game adaptation didn’t result in sustaining long-term social engagement.     

Modeling and evaluating empathy in embodied companion agents

Affective reasoning plays an increasingly important role in cognitive accounts of social interaction. Humans continuously assess one another's situational context, modify their own affective state accordingly, and then respond to these outcomes by expressing empathetic behaviors. Synthetic agents serving as companions should respond similarly. However, empathetic reasoning is riddled with the complexities stemming from the myriad factors bearing upon situational assessment. A key challenge posed by affective reasoning in synthetic agents is devising empirically informed models of empathy that accurately respond in social situations. This paper presents Care, a data-driven affective architecture and methodology for learning models of empathy by observing human–human social interactions. First, in Care training sessions, one trainer directs synthetic agents to perform a sequence of tasks while another trainer manipulates companion agents’ affective states to produce empathetic behaviors (spoken language, gesture, and posture). Care tracks situational data including locational, intentional, and temporal information to induce a model of empathy. At runtime, Care uses the model of empathy to drive situation-appropriate empathetic behaviors. Care has been used in a virtual environment testbed. Two complementary studies investigating the predictive accuracy and perceived accuracy of Care-induced models of empathy suggest that the Care paradigm can provide the basis for effective empathetic behavior control in embodied companion agents.     

Video games as virtual teachers: Prosocial video game use by children and adolescents from different socioeconomic groups is associated with increased empathy and prosocial behaviour

ObjectiveThe main aim of this study was to determine if there was a positive relationship between prosocial video game use and prosocial behaviour in children and adolescents.MethodThis study had a cross-sectional correlational design. Data were collected from 538 9–15 year old children and adolescents between March and December 2014. Participants completed measures of empathy, prosocial behaviour and video game habits. Teachers rated the prosocial behaviour of participants. The socioeconomic status of participants was also gathered.ResultsMultiple linear regressions were conducted on these data. Prosocial video game use was positively associated with the tendency to maintain positive affective relationships, cooperation and sharing as well as empathy. This association remained significant after controlling for gender, age, school type (disadvantaged/non-disadvantaged), socioeconomic status, weekly game play and violent video game use.ConclusionsThese findings provide evidence that prosocial video game use could develop empathic concern and improve affective relationships in a diverse population of young people.     

Beat gesture recognition and finger motion control of a piano playing robot for affective interaction of the elderly

This paper introduces a piano playing robot in views of smart house and assistive robot technology to care the affective states of the elderly. We address the current issues in this research area and propose a piano playing robot as a solution. For affective interaction based on music, we first present a beat gesture recognition method to synchronize the tempo of a robot playing a piano with the desired tempo of the user. To estimate the period of an unstructured beat gesture expressed by any part of a body or an object, we apply an optical flow method, and use the trajectories of the center of gravity and normalized central moments of moving objects in images. In addition, we also apply a motion control method by which robotic fingers are trained to follow a set of trajectories. Since the ability to track the trajectories influences the sound a piano generates, we adopt an iterative learning control method to reduce the tracking error.     

Rehabilitation Robotics: Performance-Based Progressive Robot-Assisted Therapy

In this paper we describe the novel concept of performance-based progressive robot therapy that uses speed, time, or EMG thresholds to initiate robot assistance. We pioneered the clinical application of robot-assisted therapy focusing on stroke—the largest cause of disability in the US. We have completed several clinical studies involving well over 200 stroke patients. Research to date has shown that repetitive task-specific, goal-directed, robot-assisted therapy is effective in reducing motor impairments in the affected arm after stroke. One research goal is to determine the optimal therapy tailored to each stroke patient that will maximize his/her recovery. A proposed method to achieve this goal is a novel performance-based impedance control algorithm, which is triggered via speed, time, or EMG. While it is too early to determine the effectiveness of the algorithm, therapists have already noted one very strong benefit, a significant reduction in arm tone.     

Audio-augmented paper for therapy and educational intervention for children with autistic spectrum disorder

Autism affects children׳s learning and social development. Commonly used rehabilitative treatments are aimed at stimulating the social skills of children with autism. In this article, we present a prototype and a pilot study on an audio-augmented paper to support the therapy of children with autism spectrum disorder (ASD). The prototype supports audio recording with standard sheets of paper by using tangible tools that can be shared between the therapist and the child. The prototype is a tool for the therapist to engage the child in a storytelling activity. We use a progressive design method based on a dynamic process that merges concept generation, technology benchmarking and activity design into continuously enriching actions. The paper highlights the qualities and benefits of using tangible audio-augmented artefacts for therapy and educational intervention for children with ASD. The work describes three main qualities of our prototype: from building cooperation to attention control, flow control, and using the children׳s own voices to foster attention.