Using emotion recognition technologies to teach children with autism spectrum disorder how to identify and express emotions

Autism spectrum disorder (ASD), which since 2013 is considered as an umbrella term for several disorders such as autistic syndrome, Asperger’s disorder and pervasive developmental disorder, is characterized, among other aspects, by deficits in social-emotion reciprocity. This deficit manifests itself as a reduced sharing of emotions and an increased difficulty in interpreting emotions other people are feeling, which in the end leads to more impairments in social communication. Since it is possible to help a person with ASD (especially children) to improve their ability to understand and detect emotions, we have developed a proposal which integrates emotion recognition technologies, often used in the field of HCI, to try to overcome this difficulty. In this paper, we present a novel software application developed as a serious game to teach children with autism spectrum disorder (ASD) to identify and express emotions. The system incorporates cutting-edge technology to support novel interaction mechanisms based on tangible user interfaces (TUIs) and emotion recognition from facial expressions. In this way, children interact with the system in a natural way by simply grasping objects with their hands and using their faces. The system has been assessed on the premises of an association with children with ASD. The outcomes of the evaluation are very positive and support the validity of the proposal.

     

基于小波相干性算法的孤独症儿童脑电评估

孤独症是一种先天的大脑发育障碍性疾病, 孤独症儿童的早期评估诊断尤为重要. 脑电图(Electroencephalography, EEG)是大脑神经细胞电生理活动在大脑皮层或头皮表面的总体反映. EEG信号中包含了大量的生理与疾病信息, 可为某些脑疾病提供诊断依据. 本文按照国际10-20系统标准电极分布将全脑划分为5个脑区, 采用小波相干性算法对孤独症(Autistic spectrum disorder, ASD)儿童和正常(Typical development, TD)儿童任意两通道之间在不同节律下的连接性进行计算, 按脑区进行划分, 得到脑区内和跨脑区功能连接结果, 随后应用独立样本t检验分析和FDR (False discovery rate)多重校正方法后给出脑区内和跨脑区在不同节律下的组间差异.结果表明, ASD组相对于TD组跨脑区连接和脑区内连接普遍较弱, 除delta频段外其他频段均差异显著, 尤其是额叶与其他脑区连接. 多重校正后通道间长程连接中额叶与枕叶、中央区与枕叶在四个频段差异显著较明显, 通道间短程连接额叶在theta和alpha频段较显著, 其他频段其他脑区对比不显著.     

Dynamic screening of autistic children in various mental states using pattern of connectivity between brain regions

In this study, a dynamic screening strategy is proposed to discriminate subjects with autistic spectrum disorder (ASD) from healthy controls. The ASD is defined as a neurodevelopmental disorder that disrupts normal patterns of connectivity between the brain regions. Therefore, the potential use of such abnormality for autism screening is investigated. The connectivity patterns are estimated from electroencephalogram (EEG) data collected from 8 brain regions under various mental states. The EEG data of 12 healthy controls and 6 autistic children (age matched in 7–10) were collected during eyes-open and eyes-close resting states as well as when subjects were exposed to affective faces (happy, sad and calm). Subsequently, the subjects were classified as autistic or healthy groups based on their brain connectivity patterns using pattern recognition techniques. Performance of the proposed system in each mental state is separately evaluated. The results present higher recognition rates using functional connectivity features when compared against other existing feature extraction methods.     

Impact of mainstream classroom setting on attention of children with autism spectrum disorder: an eye-tracking study

It has long been reported that children with autism spectrum disorder (ASD) exhibit attention difficulties while learning. They tend to focus on irrelevant information and can easily be distracted. As a result, they are often confined to a one-to-one teaching environment, with fewer distractions and social interactions than would be present in a mainstream educational setting. In recent years, inclusive mainstream schools have been growing in popularity due to government policies on equality rights. Therefore, it is crucial to investigate attentional patterns of children with ASD in mainstream schools. This study aims to explore the attentional behaviors of children with ASD in a virtual reality simulated classroom. We analyzed four eye-gaze behaviors and performance scores of 45 children: children with ASD (ASD n = 20) and typically developing children (TD n = 25) when performing attention tasks. The gaze behaviors included time to first fixate (TTFF), first fixation duration (FFD), average fixation duration (AFD) and the sum of fixation count (SFC) on fourteen areas of interest (AOIs) in the classroom. Our results showed that children with ASD exhibit similar gaze behaviors to TD children, but with significantly lower performance scores and SFC on the target AOI. These findings showed that classroom settings can influence attentional patterns and the academic performance of children with ASD. Further studies are needed on different modalities for supporting the attention of children with ASD in a mainstream setting.

     

Enhancing social communication of children with high-functioning autism through a co-located interface

This study evaluated the effectiveness of a 3-week intervention in which a co-located cooperation enforcing interface, called StoryTable, was used to facilitate collaboration and positive social interaction for six children, aged 8–10 years, with autistic spectrum disorder (ASD). The intervention focused on exposing pairs of children to an enforced collaboration paradigm while they narrated a story. Pre- and post-intervention tasks included a “low technology” version of the storytelling device and a non storytelling play situation using a free construction game. The outcome measure was a structured observation scale of social interaction. Results demonstrated progress in three areas of social behaviors. First, the participants were more likely to initiate positive social interaction with peers after the intervention. Second, the level of shared play of the children increased from the pre-test to the post-test and they all increased the level of collaboration following the intervention. Third, the children with ASD demonstrated lower frequencies of autistic behaviors while using the StoryTable in comparison to the free construction game activity. The implications of these findings are discussed in terms of the effectiveness of this intervention for higher functioning children with ASD.     

基于自然情绪感知下异常眼动轨迹的孤独症预测模型

孤独症谱系障碍(autism spectrum disorder,ASD)是一类以社会交流、刻板行为和狭隘兴趣为主要特征的神经发育障碍性疾病,致残率较高,严重影响着儿童的健康成长。ASD 主观临床诊断存在耗时长、主观性强等问题。因此,迫切需要一种快速、经济、有效的客观筛查方法。研究发现,ASD 儿童具有非典型的情绪视觉感知模式,有望将眼动追踪技术用于 ASD 的辅助诊断。该文提出一个在自然场景下,ASD 非典型情绪视觉感知模式结合机器学习的自动筛查 ASD 患者的模型。该模型可提取自然场景下感知情绪的眼动轨迹特征,通过机器学习模型进行建模,以实现根据眼动轨迹自动识别 ASD 患儿。实验结果表明,该方法的准确率为 79.71%,有望成为一种 ASD 儿童早期筛查的辅助工具。     

基于多重多尺度熵的孤独症静息态脑电信号分析

面向孤独症儿童脑功能状态评估问题, 提出一种多重多尺度熵脑电特征提取算法.算法针对传统多尺度熵信息丢失问题, 在移动均值粗粒化基础上, 采用延搁取值法构建多个尺度上的多重脑电信号序列, 再进一步计算各个尺度的样本熵.算法不仅克服了传统多尺度熵的信息丢失问题, 还能充分挖掘脑电信号的细节信息, 同时减小了尺度间的波动.基于该算法分析了16名孤独症儿童和16名正常儿童的19个通道的脑电信号.结果表明:正常儿童F7、F8、T4、P3通道的多重多尺度熵和复杂度均高于孤独症儿童, 且存在显著性差异(P < 0.05).表明前颞叶(F7、F8)可以作为孤独症儿童脑功能状态评估的敏感脑区, T4、P3可以作为辅助干预的敏感通道.     

Image-based synthesis of Chinese landscape painting

This paper describes a new framework for synthesizing chinese landscape painting using an image-ased approach.The framework involves two stages:a preprocessing phase,in which a few brush stroke texture primitivities(BSTP) are collected from samples of hand-made Chinese paintings,and the control picture is constructed to provide color IDs of mountains,and the on-line phases,in which the fog image is synthesized and mountains are “drawn“ by mapping multiple layers of BSTP guided by the control picture.When more complex shading is needed,the shading picture is constructed and used during the BSTP mapping phase.Finally,the synthesized Chinese landscape paintings of a variety of styles are given and they look more close to the hand-made work than those produced with previous modeling methods.     

基于脑功能网络分析的孤独症儿童辅助干预效果研究

脑功能网络是分析复杂网络之间连接关系的一种有效方法, 对脑功能障碍分析具有重要意义. 本文基于频域Granger因果分析的定向传递函数(Direction-transfer function, DTF), 构建了各频段的脑功能网络. 采用图论方法分析最佳阈值下经颅直流电刺激(Transcranial direct current stimulation, tDCS)干预前后孤独症(Autism spectrum disorder, ASD)儿童脑网络的平均度、全局效率和平均局部效率等特征, 并对比了经颅直流电刺激对孤独症儿童脑功能状态辅助干预效果. 结果发现刺激前组在各频段的图论特征均低于刺激后组($P<0.05$), 其中Theta频段和低-beta频段的局部效率统计性差异显著, 表明在一定程度上tDCS干预是ASD儿童治疗的有效手段.     

Tojisha-Kenkyu of autism spectrum disorders

Accumulating evidence suggests behavioral and genetic fractionation of the autistic traits. Above it, ethnographic researches that document how the sociality of children with autism varies across different situational conditions, depict autistic sociality not as an oxymoron but, rather, as a reality. Given the discussion above, it is important to do case-based studies on the assumption of heterogeneity of sociality and of autism. In addition, subjective experience is an important outcome variable whenever considering support. As a candidate of the research methods that meet the above conditions, we would like to introduce Tojisha-Kenkyu. In a nutshell, ‘Tojisha-Kenkyu’ is studying oneself through communication with others who share similar experiences. We introduce Ayaya’s Tojisha-Kenkyu and Necco Tojisha-Kenkyu meeting as examples of Tojisha-Kenkyu on autism spectrum disorders (ASD), which illustrate that Tojisha-Kenkyu has not only recuperative effects but also academic potentials changing the concept of ASD.     

Analyzing the performance of fuzzy cognitive maps with non-linear hebbian learning algorithm in predicting autistic disorder

The soft computing technique of fuzzy cognitive maps (FCM) for modeling and predicting autistic spectrum disorder has been proposed. The FCM models the behavior of a complex system and is used to develop new knowledge based system applications. FCM combines the robust properties of fuzzy logic and neural networks. To overwhelm the limitations and to improve the efficiency of FCM, a good learning method of unsupervised training could be applied. A decision system based on human knowledge and experience with a FCM trained using unsupervised non-linear hebbian learning algorithm is proposed here. Through this work the hebbian algorithm on non-linear units is used for training FCMs for the autistic disorder prediction problem. The investigated approach serves as a guide in determining the prognosis and in planning the appropriate therapies to special children.     

A community of practice approach to the development of non-traditional learners through networked learning

Abstract This paper analyses a sample of online discussions to evaluate the development of adult learners as reflective practitioners within a networked learning community. The context for our study is a blended learning course offering post-experience professional training to non-traditional university students. These students are parents and carers of people with autistic spectrum disorder (ASD). We use Lave and Wenger's 'communities of practice' as a theoretical framework for establishing how students develop a learning community based upon mutual engagement, joint enterprise and shared repertoires. Those three aspects are analysed according to two measures. The first focuses on learner appropriation of the professional discourse, values and goals of the ASD carer through the network. The second relates to changes in the quality of collaborative activity over time. Our analysis demonstrates that students belong to an overarching community of practice, with different subsets who work at sharing and co-constructing common understandings. This shared discourse and common notions of what constitutes good practice help create a safe interaction space for the students. Once group identity is consolidated, more challenging questions emerge and the group are able to define further common values, understandings and goals through processes of resolution.     

Online Affect Detection and Robot Behavior Adaptation for Intervention of Children With Autism

Investigation into robot-assisted intervention for children with autism spectrum disorder (ASD) has gained momentum in recent years. Therapists involved in interventions must overcome the communication impairments generally exhibited by children with ASD by adeptly inferring the affective cues of the children to adjust the intervention accordingly. Similarly, a robot must also be able to understand the affective needs of these children—an ability that the current robot-assisted ASD intervention systems lack—to achieve effective interaction that addresses the role of affective states in human–robot interaction and intervention practice. In this paper, we present a physiology-based affect-inference mechanism for robot-assisted intervention where the robot can detect the affective states of a child with ASD as discerned by a therapist and adapt its behaviors accordingly. This paper is the first step toward developing “understanding” robots for use in future ASD intervention. Experimental results with six children with ASD from a proof-of-concept experiment (i.e., a robot-based basketball game) are presented. The robot learned the individual liking level of each child with regard to the game configuration and selected appropriate behaviors to present the task at his/her preferred liking level. Results show that the robot automatically predicted individual liking level in real time with 81.1% accuracy. This is the first time, to our knowledge, that the affective states of children with ASD have been detected via a physiology-based affect recognition technique in real time. This is also the first time that the impact of affect-sensitive closed-loop interaction between a robot and a child with ASD has been demonstrated experimentally.      

基于颜色特征的画家艺术风格提取方法

针对现有全局与局部特征提取方法所提取的颜色特征无法有效描述画家艺术风格的问题，提出了一种基于关键区域的油画描述法。首先，通过融合主色调占比与颜色丰富度定义了油画区域信息丰富度，以检测并选取油画的关键区域。其次，提取所选油画关键区域的54维特征来描述油画，并利用Fisher Score对这些特征进行评估，选取重要的特征描述油画关键区域，从而高度体现画家艺术风格。最后，为了验证上述方法的有效性，利用朴素贝叶斯分类器实现油画的分类。建立两个数据库进行仿真实验，数据库1包含3位画家的120幅作品，数据库2包含9位画家3种流派的513幅作品。数据库1上的实验结果表明，结合Fisher Score特征描述的分类正确率高于普通分类的正确率，所提方法依据画家与流派的油画分类正确率分别达到了90%与90.20%。数据库2上的实验结果表明，所提方法根据画家的油画分类正确率达到了90%，相较Features-FS的正确率提高了6.67个百分点；依据流派分类的结果正确率达到了90.20%，与Features-FS的正确率相当。所提的基于关键区域的油画描述法所提取的特征能够有效描述画家的艺术风格。     

Audio-augmented paper for therapy and educational intervention for children with autistic spectrum disorder

Autism affects children׳s learning and social development. Commonly used rehabilitative treatments are aimed at stimulating the social skills of children with autism. In this article, we present a prototype and a pilot study on an audio-augmented paper to support the therapy of children with autism spectrum disorder (ASD). The prototype supports audio recording with standard sheets of paper by using tangible tools that can be shared between the therapist and the child. The prototype is a tool for the therapist to engage the child in a storytelling activity. We use a progressive design method based on a dynamic process that merges concept generation, technology benchmarking and activity design into continuously enriching actions. The paper highlights the qualities and benefits of using tangible audio-augmented artefacts for therapy and educational intervention for children with ASD. The work describes three main qualities of our prototype: from building cooperation to attention control, flow control, and using the children׳s own voices to foster attention.     

基于 PCCS 色彩体系的 ASD 儿童干预 APP 配色研究

为提升自闭症谱系障碍(ASD)儿童干预 APP 的康复效果,为设计师提供合理有效的参考,提出 一种基于 PCCS 色彩体系的 ASD 儿童干预 APP 界面配色评价方法。首先通过文献分析和焦点小组法,构建基 于 PCCS 色彩体系的 ASD 儿童干预 APP 界面配色评价体系;其次运用 AHP 法获取各项评价指标的权重;再 次筛选得到 8 个基于 PCCS 色彩体系的界面配色设计方案,邀请专家对其打分,得到初始评判矩阵;然后通过 TOPSIS 法获得各方案的排序结果;最后对结果进行解析。研究表明,ASD 儿童对绿色、蓝色等冷色调及明度、 纯度较高的色彩具有较高的认可度;在 PCCS 色调图中,ASD 儿童更喜爱轻柔色调、浅色调、邻近色与互补色 的搭配;在界面配色设计中应更趋于明快、清晰、和谐、稳重的色彩搭配。将该评价方法应用于干预 APP 界 面配色实验方案评价中,可以提高设计评价过程中的客观性、科学性,为设计师进行决策提供参考,更好地促 进 ASD 儿童干预 APP 的发展。     

Aging and Reverse-aging Traditional Chinese Painting Images Based on Web-Mining

Traditional Chinese painting has a long history more than thousands of year. No matter when these paintings are drawn, all suffer deterioration caused both by aging and external forces. Restoring damaged artworks involves very professional issues beyond the scope of computer science. However, retouching their images with aging/reverse-aging effects throws down a very interesting challenge to us. Unfortunately, it is very difficult to physically simulate the optical properties of aging pigments and surface material. In addition, the conventional image processing approaches fail to consider the above-mentioned factors, as well as historical trends in color selection and utilization. This paper proposes a knowledge-based and progressive refinement approach to simulate and remove the aging effects in the images of traditional Chinese paintings. Web mining is an important technique which supports this approach by a large number of the source images. A database of information related to degrees of color fade over time is constructed by collecting, categorizing, and analyzing the images of paintings from different eras. In addition, relationships among color distribution histograms, the different subjects being depicted, the year or dynasty the painting was created, and the properties of different pigments are also tracked. To separate the aging coloration influence of pigment from surface, the untouched areas, which are intentionally left blank as a common motif in traditional Chinese paintings, are utilized to provide a useful clue. The acquired data are then modified and feedbacked into our system to progressively refine the results. At the final rendering stage, several image processing techniques are applied to further enhance image quality. By using our prototype system, both aging and reverse-aging processes can be simulated. Initial study results show that this approach has great potential in simulating aging effects on images of traditional Chinese paintings.     

Physiology-based affect recognition for computer-assisted intervention of children with Autism Spectrum Disorder

Generally, an experienced therapist continuously monitors the affective cues of the children with Autism Spectrum Disorders (ASD) and adjusts the course of the intervention accordingly. In this work, we address the problem of how to make the computer-based ASD intervention tools affect-sensitive by designing therapist-like affective models of the children with ASD based on their physiological responses. Two computer-based cognitive tasks are designed to elicit the affective states of liking, anxiety, and engagement that are considered important in autism intervention. A large set of physiological indices are investigated that may correlate with the above affective states of children with ASD. In order to have reliable reference points to link the physiological data to the affective states, the subjective reports of the affective states from a therapist, a parent, and the child himself/herself were collected and analyzed. A support vector machines (SVM)-based affective model yields reliable prediction with approximately 82.9% success when using the therapist's reports. This is the first time, to our knowledge, that the affective states of children with ASD have been experimentally detected via physiology-based affect recognition technique.