基于BM神经网络编码的生理信号情感识别

处理生理信号情感识别问题,用连续限制的BM神经网络对采集的生理信号特征进行编码,可以有效地消除噪声与实验个体特征间的差异,结合传统支持向量机分类方法,在进行情感二分类以及泛化分类问题上可以取得较其它分类方法更普适化的效果.构建了生理情感识别方法的框架,采用基于BM神经网络编码的情感识别方法,设计实验并进行了数据采集特征提取以及算法验证.基于实验所采集数据以及实验结果表明了该方法的有效性和可行性.     

基于深度情感唤醒网络的多模态情感分析与情绪识别

随着网络平台上各类图像、视频数据的快速增长,多模态情感分析与情绪识别已成为一个日益热门的研究领域.相比于单模态情感分析,多模态情感分析中的模态融合是一个亟待解决的关键问题.受到认知科学中情感唤起模型的启发,提出一种能够模拟人类处理多通道输入信息机制的深度情感唤醒网络(DEAN),该网络可实现多模态信息的有机融合,既能处理情绪的连贯性,又能避免融合机制的选择不当而带来的问题.DEAN网络主要由以下3部分组成:跨模态Transformer模块,用以模拟人类知觉分析系统的功能;多模态BiLSTM系统,用以模拟认知比较器;多模态门控模块,用以模拟情感唤起模型中的激活结构.在多模态情感分析与情绪识别的3个经典数据集上进行的比较实验结果表明,DEAN模型在各数据集上的性能均超越了目前最先进的情感分析模型.     

基于交互注意力机制的多模态情感识别算法

在多模态语音情感识别中,现有的研究通过提取大量特征来识别情感,但过多的特征会导致关键特征被淹没在相对不重要特征里,造成关键信息遗漏.为此提出了一种模型融合方法,通过两种注意力机制来寻找可能被遗漏的关键特征.本方法在IEMOCAP数据集上的四类情感识别准确率相比现有文献有明显提升;在注意力机制可视化下,两种注意力机制分别找到了互补且对人类情感识别重要的关键信息,从而证明了所提方法相比传统方法的优越性.     

基于视听觉感知系统的多模态情感识别

情绪识别作为人机交互的热门领域,其技术已经被应用于医学、教育、安全驾驶、电子商务等领域.情绪主要由面部表情、声音、话语等进行表达,不同情绪表达时的面部肌肉、语气、语调等特征也不相同,使用单一模态特征确定的情绪的不准确性偏高,考虑到情绪表达主要通过视觉和听觉进行感知,本文提出了一种基于视听觉感知系统的多模态表情识别算法,分别从语音和图像模态出发,提取两种模态的情感特征,并设计多个分类器为单特征进行情绪分类实验,得到多个基于单特征的表情识别模型.在语音和图像的多模态实验中,提出了晚期融合策略进行特征融合,考虑到不同模型间的弱依赖性,采用加权投票法进行模型融合,得到基于多个单特征模型的融合表情识别模型.本文使用AFEW数据集进行实验,通过对比融合表情识别模型与单特征的表情识别模型的识别结果,验证了基于视听觉感知系统的多模态情感识别效果要优于基于单模态的识别效果.     

多模态情感识别研究进展*

情感识别研究热点正从单模态转移到多模态。针对多模态情感特征提取与融合的技术难点,本文列举了目前应用较广的多模态情感识别数据库,介绍了面部表情和语音情感这两个模态的特征提取技术,重点阐述了多模态情感融合识别技术,主要对多模态情感特征融合策略和融合方法进行了综述,对不同算法下的识别效果进行了对比。最后,对多模态情感识别研究中存在的问题进行了探讨,并对未来的研究方向进行了展望。     

基于视觉识别的机器人音频多模态情感识别系统设计

情感识别系统是人机交互的核心功能,但是常规的机器人音频情感识别系统在提取情感特征时易产生误差,导致情感识别准确度通常较低,因此基于视觉识别设计机器人音频多模态情感识别系统.在硬件设计上,设计嵌入式视觉采集系统,设计机器人音频识别电路.在软件设计上,构建情感分类语料库,提取基音频率、短时能量、过零率、频率倒谱系数等音频多...     

基于异步芯片的多模态神经生理信号采集技术

心理生理计算目前主要基于同步芯片的神经生理信号采集技术进行分析研究,而人体神经生理活动表征具有异步属性,同步采集技术无法精准、实时和高效地刻画人体神经生理信号活动规律。如何低功耗、低冗余、实时精准地采集异步多模态神经生理信号成为心理生理计算首要解决的难题。针对这一难题,本文以研究微观神经生理活动规律和宏观心理生理活动为目的,解决异步多模态生理信息采集方案和相匹配的被动生理信号传感技术的设计难点,设计研发了首款异步生理信号处理芯片,该芯片具备低功耗、高精度时序、高性能计算和抗干扰的特点。最后展望了该芯片在脑科学和类脑计算领域的应用前景。     

基于图神经网络多模态融合的语音情感识别模型

目前,基于多模态融合的语音情感识别模型普遍存在无法充分利用多模态特征之间的共性和互补性、无法借助样本特征间的拓扑结构特性对样本特征进行有效地优化和聚合,以及模型复杂度过高的问题。为此,引入图神经网络,一方面在特征优化阶段,将经过图神经网络优化后的文本特征作为共享表示重构基于声学特征的邻接矩阵,使得在声学特征的拓扑结构特性中包含文本信息,达到多模态特征的融合效果;另一方面在标签预测阶段,借助图神经网络充分聚合当前节点的邻接节点所包含的相似性信息对当前节点特征进行全局优化,以提升情感识别准确率。同时为防止图神经网络训练过程中可能出现的过平滑问题,在图神经网络训练前先进行图增强处理。在公开数据集IEMOCAP 和RAVDESS上的实验结果表明,所提出的模型取得了比基线模型更高的识别准确率和更低的模型复杂度,并且模型各个组成部分均对模型性能提升有所贡献。     

基于Android的多模态情感识别APP的设计与开发

在团队自建的中文多模态情感识别语料库的基础上,训练了多模态情感识别深度神经网络模型,能综合视频中图片、音频和文本三个模态信息进行情感识别。基于该情感识别模型,设计并开发了一款基于Android平台的移动应用程序,捕获视频中人物的话语文本、声音和面部表情,识别出人物的情感状态,以支撑后续共情聊天机器人的研发。     

基于BPSO的四种生理信号的情感状态识别

通过生理信号来识别人的情感状态越来越引起人们的关注.如何提取有效的生理信号特征进行情感状态的分类,是情感识别的关键.本文采用离散二进制粒子群优化算法(BPS0)进行特征选择,以提高情感状态分类的效果.通过四种生理信号来识别四种情感状态,用最近邻法进行分类,总体识别率达到85%.仿真实验结果表明,将BPSO方法用于生理信号的特征选择是可行的.     

用分层循环遗传算法去识别生理信号情感状态

针对生理信号的情感识别问题,采用自适应分层式遗传算法方法对生理信号进行特征选择,能有效地解决特征组合优化问题。实验仿真表明,可以得到有效地特征组合来进行生理信号情感状态识别。     

Emotion recognition based on physiological signals using brain asymmetry index and echo state network

Neural Computing and Applications - This paper proposes a method to evaluate the degree of emotion being motivated in continuous music videos based on asymmetry index (AsI). By collecting two...     

用自适应蚁群算法的生理信号情感状态识别

针对生理信号的情感识别问题,将蚁群优化算法用于情感生理信号特征选择,并采用自适应的适应度参数值、变异策略和临近位置交换策略对算法进行改进,使用K近邻法进行情感分类,以获得较高的识别率和有效特征组合.通过四种生理信号(EMG、SC、ECG、RSP)来识别四种情感(joy、anger、sadness、pleasure),实验仿真结果表明,将蚁群优化算法引入情感识别的研究是可行的.     

Emotion recognition based on physiological changes in music listening

This paper investigates the potential of physiological signals as reliable channels for emotion recognition. All essential stages of an automatic recognition system are discussed, from the recording of a physiological dataset to a feature-based multiclass classification. In order to collect a physiological dataset from multiple subjects over many weeks, we used a musical induction method which spontaneously leads subjects to real emotional states, without any deliberate lab setting. Four-channel biosensors were used to measure electromyogram, electrocardiogram, skin conductivity and respiration changes. A wide range of physiological features from various analysis domains, including time/frequency, entropy, geometric analysis, subband spectra, multiscale entropy, etc., is proposed in order to find the best emotion-relevant features and to correlate them with emotional states. The best features extracted are specified in detail and their effectiveness is proven by classification results. Classification of four musical emotions (positive/high arousal, negative/high arousal, negative/low arousal, positive/low arousal) is performed by using an extended linear discriminant analysis (pLDA). Furthermore, by exploiting a dichotomic property of the 2D emotion model, we develop a novel scheme of emotion-specific multilevel dichotomous classification (EMDC) and compare its performance with direct multiclass classification using the pLDA. Improved recognition accuracy of 95\% and 70\% for subject-dependent and subject-independent classification, respectively, is achieved by using the EMDC scheme.     

Emotion recognition from physiological signals using wireless sensors for presence technologies

In this article we describe a new approach to enhance presence technologies. First, we discuss the strong relationship between cognitive processes and emotions and how human physiology is uniquely affected when experiencing each emotion. Secondly, we introduce our prototype multimodal affective user interface. In the remainder of the paper we describe the emotion elicitation experiment we designed and conducted and the algorithms we implemented to analyse the physiological signals associated with emotions. These algorithms can then be used to recognise the affective states of users from physiological data collected via non-invasive technologies. The affective intelligent user interfaces we plan to create will adapt to user affect dynamically in the current context, thus providing enhanced social presence.     

Negative emotion recognition using multimodal physiological signals for advanced driver assistance systems

Artificial Life and Robotics - Recent advanced driver assistance systems’ (ADASs) control cars to avoid accidents, but few of them consider driver’s comfort. To realize comfortable...     

生理信号情感识别的遗传算法研究

将情感识别看成一个组合模式优化问题,从生理信号ECG,EMG,SC,RSP中抽取情感特征,遗传算法和最近邻算法相结合尝试找出最能“代表”某一情感状态joy,anger,sadness,pleasure的最优情感特征组合模式,仿真实验表明,该方法是可行并且有效的。     

Detecting driving stress in physiological signals based on multimodal feature analysis and kernel classifiers

Monitoring driving status has great potential in helping us decline the occurrence probability of traffic accidents and the aim of this research is to develop a novel system for driving stress detection based on multimodal feature analysis and kernel-based classifiers. Physiological signals such as electrocardiogram, galvanic skin response and respiration were record from fourteen drives executed in a prescribed route at real drive environments. Features were widely extracted from time, spectral and wavelet multi-domains. In order to search for the optimal feature sets, Sparse Bayesian Learning (SBL) and Principal Component Analysis (PCA) were combined and adopted. Kernel-based classifiers were employed to improve the accuracy of stress detection task. Analysis I used features from 10 s intervals of data which were recorded during well-defined rest, highway and city driving conditions to discriminate three levels of diving stress achieving an averaging accuracy over 99% at per-drive level and 89% in cross-drive validation. Analysis II made continuous stress evaluation throughout a complete driving test attaining a high coincidence with the true road situation especially at the switching interval of traffic conditions. Experimental results reveal that different levels of driving stress can be characterized by specific set of physiological measures. These physiological measures could be applied to in-vehicle intelligent systems in various approaches to help the drivers better manage their negative driving status. Our design scheme for driving stress detection could also facilitate the development of similar in-vehicle expert systems, such as driver's emotion management, driver's sleeping onset monitoring, and human-computer interaction (HCI).     

多模态数据的行为识别综述

行为识别是当前计算机视觉方向中视频理解领域的重要研究课题。从视频中准确提取人体动作的特征并识别动作,能为医疗、安防等领域提供重要的信息,是一个十分具有前景的方向。本文从数据驱动的角度出发,全面介绍了行为识别技术的研究发展,对具有代表性的行为识别方法或模型进行了系统阐述。行为识别的数据分为RGB模态数据、深度模态数据、骨骼模态数据以及融合模态数据。首先介绍了行为识别的主要过程和人类行为识别领域不同数据模态的公开数据集;然后根据数据模态分类,回顾了RGB模态、深度模态和骨骼模态下基于传统手工特征和深度学习的行为识别方法,以及多模态融合分类下RGB模态与深度模态融合的方法和其他模态融合的方法。传统手工特征法包括基于时空体积和时空兴趣点的方法(RGB模态)、基于运动变化和外观的方法(深度模态)以及基于骨骼特征的方法(骨骼模态)等;深度学习方法主要涉及卷积网络、图卷积网络和混合网络,重点介绍了其改进点、特点以及模型的创新点。基于不同模态的数据集分类进行不同行为识别技术的对比分析。通过类别内部和类别之间两个角度对比分析后,得出不同模态的优缺点与适用场景、手工特征法与深度学习法的区别和融合多模态的优...