基于卷积联合适应网络的脑电信号情感识别

提出一种基于深度卷积联合适应网络（Convolutional neural network-joint adaptation network,CNN-JAN）的脑电信号（Electroencephalogram, EEG）情感识别模型。该模型将迁移学习中联合适应的思想融合到深度卷积网络中,首先采用长方形卷积核提取数据的空间特征,捕捉脑电数据通道间的深层情感相关信息,再将提取的空间特征输入含有联合分布的多核最大均值差异算法（Multi-kernel joint maximum mean discrepancy,MK-JMMD）的适配层进行迁移学习,使用MK-JMMD度量算法解决源域和目标域分布不同的问题。所提方法在SEED数据集上使用微分熵特征和微分尾端性特征分别进行情感分类实验,其中使用微分熵特征被试内跨试验准确率达到84.01%,与对比实验和目前流行的迁移学习方法相比,准确率进一步提高,跨被试实验精度也取得较好的性能,验证了该模型用于EEG信号情感识别任务的有效性。     

采用多路图注意力网络的情绪脑电信号识别方法

情绪是一种大脑产生的主观认知的概括。脑信号解码技术可以以一种较客观的方式来有效地研究人的情绪及其相关认知行为。本文提出了一种基于图注意力网络的脑电情绪识别方法(multi-path graph attention networks, MPGAT),该方法通过对脑电信号通道建图,利用卷积层提取脑电信号的时域特征以及各频带的特征,使用图注意力网络进一步捕捉情绪脑电信号的局部特征以及各脑区之间的内在功能关系,进而构建出更好的脑电信号表征。MPGAT在SEED和SEED-IV数据集的跨被试情绪识别平均准确率分别为86.03%、72.71%,在DREAMER数据集的效价(valence)和唤醒(arousal)维度的跨被试平均准确率分别为76.35%和75.46%,达到并部分超过了目前最先进脑电情绪识别方法的性能。本文所提出的脑电信号处理方法有望为情绪认知科学研究与情绪脑机接口系统提供新的技术手段。     

基于PCA的离散小波自回归情感识别

针对情感识别进行研究,提出基于主成分分析法（PCA）过滤小波变换结合自回归模型提取的信号特征方法,并基于梯度提升分类树以实现情感分类.将特征提取的重点放在脑电信号变化情况以及小波分量变化情况作为脑电信号特征.采用Koelstra等提出的分析人类情绪状态的多模态标准数据库DEAP,提取8种正负情绪代表各个脑区的14个通道脑电数据.结果表明,算法对8种情感两两分类识别平均准确率为95.76%,最高准确率为98.75%,可为情感识别提供帮助.     

基于位置信息重建与时频域信息融合的脑电信号情感识别

脑电信号由中枢神经系统产生,具有很高的真实性,但存在数据量少和数据复杂等问题.为提高脑电信号情感识别准确率,在脑电信号功率谱密度的基础上提出一种脑电位置信息重建的方法,使神经网络模型可以直接获取脑电信号中不易学习的位置信息.运用融合网络从原始的脑电信号中分别抽取时域特征和频域特征,根据频域信息重建脑电信号的位置信息,将时频域信息及位置信息进行融合,以获得更高的脑电信号情感分类准确率.在公开数据集DEAP上的实验结果表明,Valence和Arousal的二分类准确率分别达到86.31％和85.57％,与传统脑电信号情感识别方法相比,该方法分类准确率得到有效提高.     

基于位置信息重建与时频域信息融合的脑电信号情感识别

脑电信号由中枢神经系统产生,具有很高的真实性,但存在数据量少和数据复杂等问题.为提高脑电信号情感识别准确率,在脑电信号功率谱密度的基础上提出一种脑电位置信息重建的方法,使神经网络模型可以直接获取脑电信号中不易学习的位置信息.运用融合网络从原始的脑电信号中分别抽取时域特征和频域特征,根据频域信息重建脑电信号的位置信息,将时频域信息及位置信息进行融合,以获得更高的脑电信号情感分类准确率.在公开数据集DEAP上的实验结果表明,Valence和Arousal的二分类准确率分别达到86.31％和85.57％,与传统脑电信号情感识别方法相比,该方法分类准确率得到有效提高.     

基于无阈值递归图和深度残差网络的脑电信号情感识别

提出基于无阈值递归图和深度残差网络相结合的脑电信号情感识别方法。基于非线性动力学理论,将脑电信号转化为无阈值递归图,克服了传统递归图分析中阈值选取的问题,同时脑电信号非线性特征被映射到二维平面。通过深度残差网络实现特征图非线性特征的自动提取,建立情感脑电分类模型,实现了单导联脑电信号情感识别。为进一步提高识别精度,联合四个单导联识别结果,采用“投票法”完成多导联脑电信号情感状态的联合识别。仿真结果表明,对Fp1、Fp2、F3、F4单导联脑电信号情感识别,平均准确率分别为93.82%、93.62%、94.54%、92.92%;多导联平均准确率为94.95%,提高了识别的准确率,具有很大的实用价值。     

基于KIV模型的脑电识别方法

脑电信号具有动态、非线性和数值高度随机的特点, 针对传统的人工神经网络模型识别脑电信号时在特征提取和识别精度方面表现出的局限性, 本研究提出了一种新的识别方法, 使用KIV模型对脑电信号进行识别. 首先, 通过仿真实验, 分析了KIV模型不同的刺激下表现出的动力学特性. 接着, 使用KIV模型分别对癫痫脑电信号和情感脑电信号进行识别, 在实验过程中不进行特征提取, 直接将多通道原始脑电信号输入到KIV模型中, 在BONN和GAMEEMO数据集上分别获得了99.50%和90.83%的识别准确率. 研究结果表明, 与现有的模型相比, KIV模型具有较好的识别脑电信号的能力, 可为脑电识别提供帮助.     

基于便携式脑电设备的快乐和悲伤情绪分类

驾驶员情绪状态的识别对车辆主动安全技术的研究具有重要的应用价值.本研究通过情绪视频诱发的方法采集17位被试前额双通道脑电信号,提取不同情绪的脑电特征,并对数据进行降维处理后采用多种分类器进行情绪分类.结果显示,与单核分类器和集成学习分类器相比,基于梯度提升决策树(GBDT)算法得到快乐和悲伤的识别准确率最高.本研究为驾驶员情绪状态的实时监测和识别提供新方法,为提高行车的安全性提供了理论保障.     

基于集成深度学习方法的跨被试EEG特征情感识别

该研究使用脑电（EEG）信号经过处理得到的数据集DEAP和HCI,利用微分熵作为特征提取的工具,基于传统机器学习算法,集成学习中的梯度提升树、Xgboost、Adaboost、随机森林算法,以及深度神经网络、卷积神经网络与GoogLeNet实现跨被试的EEG特征情感识别任务,并比较各方法应用于EEG情感分析时的结果差异。比较平均准确率,结果表明深度学习方法取得了不错的成绩,三个深度模型对两个数据集的valence平均准确率达到0.5956和0.6307之间,arousal达到0.6062和0.6774之间,显著优于机器学习算法与集成学习模型取得的结果。     

基于传递熵关键因果连接的情感识别方法

人脑在情绪活动中呈现的信息流是复杂多变的,因此理解脑区间的动态交互过程至关重要,但是基于原始脑电信号构建的情绪网络包含了许多与情绪无关的冗余信息.针对此问题,提出一种在不丢失关键因果信息的前提下去除情绪无关网络连接的方法,并验证其在情感识别过程中的有效性.首先,基于传递熵因果分析方法对积极、中性和消极情绪构建归一化传递熵矩阵,再从积极、消极情绪矩阵中减去中性情绪矩阵,最后基于简化后的矩阵构建因效性脑网络并利用图论分析不同情绪的网络连通性.通过在DEAP数据集上的验证发现,该方法有效地提高了情感识别准确率.     

Real-time EEG-based emotion monitoring using stable features

In human–computer interaction (HCI), electroencephalogram (EEG) signals can be added as an additional input to computer. An integration of real-time EEG-based human emotion recognition algorithms in human–computer interfaces can make the users experience more complete, more engaging, less emotionally stressful or more stressful depending on the target of the applications. Currently, the most accurate EEG-based emotion recognition algorithms are subject-dependent, and a training session is needed for the user each time right before running the application. In this paper, we propose a novel real-time subject-dependent algorithm with the most stable features that gives a better accuracy than other available algorithms when it is crucial to have only one training session for the user and no re-training is allowed subsequently. The proposed algorithm is tested on an affective EEG database that contains five subjects. For each subject, four emotions (pleasant, happy, frightened and angry) are induced, and the affective EEG is recorded for two sessions per day in eight consecutive days. Testing results show that the novel algorithm can be used in real-time emotion recognition applications without re-training with the adequate accuracy. The proposed algorithm is integrated with real-time applications “Emotional Avatar” and “Twin Girls” to monitor the users emotions in real time.     

基于人脸图像和脑电的连续情绪识别方法

基于多模态生理数据的连续情绪识别技术在多个领域有重要用途,但碍于被试数据的缺乏和情绪的主观性,情绪识别模型的训练仍需更多的生理模态数据,且依赖于同源被试数据.本文基于人脸图像和脑电提出了多种连续情绪识别方法.在人脸图像模态,为解决人脸图像数据集少而造成的过拟合问题,本文提出了利用迁移学习技术训练的多任务卷积神经网络模型...     

Inter-subject transfer learning for EEG-based mental fatigue recognition

Mental fatigue is one of the major factors leading to human errors. To avoid failures caused by mental fatigue, researchers are working on ways to detect/monitor fatigue using different types of signals. Electroencephalography (EEG) signal is one of the most popular methods to recognize mental fatigue since it directly measures the neurophysiological activities in the brain. Current EEG-based fatigue recognition algorithms are usually subject-specific, which means a classifier needs to be trained per subject. However, as fatigue may need a relatively long period to induce, collecting training data from each new user could be time-consuming and troublesome. Calibration-free methods are desired but also challenging since significant variability of physiological signals exists among different subjects. In this paper, we proposed algorithms using inter-subject transfer learning for EEG-based mental fatigue recognition, which did not need a calibration. To explore the influence of the number of EEG channels on the algorithms’ accuracy, we also compared the cases of using one channel only and multiple channels. Random forest was applied to choose the channel that has the most distinguishable features. A public EEG fatigue dataset recorded during driving was used to validate the algorithms. EEG data from 11 subjects were selected from the dataset and leave-one-subject-out cross-validation was employed. The channel from the occipital lobe is selected when only one channel is desired. The proposed transfer learning-based algorithms using Maximum Independence Domain Adaptation (MIDA) achieved an accuracy of 73.01% with all thirty channels, and using Transfer Component Analysis (TCA) achieved 68.00% with the one selected channel.     

适于数据流组合分类的直推学习方法

在进行组合决策时,已有的组合分类方法需要对多个组合分类器均有效的公共已知标签训练样本。为了解决在没有已知标签样本的情况下数据流组合分类决策问题,提出一种基于约束学习的数据流组合分类器的融合策略。在判定测试样本上的决策时,根据直推学习理论设计满足每一个局部分类器约束度量的方法,保证了约束的可行性,解决了分布式分类聚集时最大熵的直推扩展问题。测试数据集上的实验证明,与已有的直推学习方法相比,此方法可以获得更好的决策精度,可以应用于数据流组合分类的融合。     

改进DBA算法的眼动模式分析

随着眼动追踪技术的进步和设备成本的降低,眼动追踪技术已广泛应用于智能教育领域,分析眼动数据以评估学习状态成为智能教育中一个十分重要的环节。眼动扫描路径可以直接或间接地反映思维模式及心理状态的变化,通过分析扫描路径探索学习者眼动行为的共性和差异性,为改善视觉内容和给出指导性意见提供重要参考。首先研究在同一任务情况下学习者扫描路径的时间序列表示和聚类,通过聚类结果评估专注、走神及信息迷航等三种学习状态。进而对重心平均动态时间规整（DTW?barycenter averaging,DBA）算法进行改进,并用于提取群体眼动模式,结合动态时间规整（dynamic time warping,DTW）算法计算扫描路径的相似度和确定聚类种子,采用距离密度聚类（distance density clustering,DDC）算法进行聚类。实验表明,基于时间序列的眼动模式挖掘能够识别群体观看行为。而聚类揭示了不同的阅读策略,并提供了评估学习状态的能力。     

一种异构直推式迁移学习算法

目标领域已有类别标注的数据较少时会影响学习性能,而与之相关的其他源领域中存在一些已标注数据.迁移学习针对这一情况,提出将与目标领域不同但相关的源领域上学习到的知识应用到目标领域.在实际应用中,例如文本-图像、跨语言迁移学习等,源领域和目标领域的特征空间是不相同的,这就是异构迁移学习.关注的重点是利用源领域中已标注的数据来提高目标领域中未标注数据的学习性能,这种情况是异构直推式迁移学习.因为源领域和目标领域的特征空间不同,异构迁移学习的一个关键问题是学习从源领域到目标领域的映射函数.提出采用无监督匹配源领域和目标领域的特征空间的方法来学习映射函数.学到的映射函数可以把源领域中的数据在目标领域中重新表示.这样,重表示之后的已标注源领域数据可以被迁移到目标领域中.因此,可以采用标准的机器学习方法(例如支持向量机方法)来训练分类器,以对目标领域中未标注的数据进行类别预测.给出一个概率解释以说明其对数据中的一些噪声是具有鲁棒性的.同时还推导了一个样本复杂度的边界,也就是寻找映射函数时需要的样本数.在4个实际的数据库上的实验结果,展示了该方法的有效性.     

Improving BCI-based emotion recognition by combining EEG feature selection and kernel classifiers

Current emotion recognition computational techniques have been successful on associating the emotional changes with the EEG signals, and so they can be identified and classified from EEG signals if appropriate stimuli are applied. However, automatic recognition is usually restricted to a small number of emotions classes mainly due to signal’s features and noise, EEG constraints and subject-dependent issues. In order to address these issues, in this paper a novel feature-based emotion recognition model is proposed for EEG-based Brain–Computer Interfaces. Unlike other approaches, our method explores a wider set of emotion types and incorporates additional features which are relevant for signal pre-processing and recognition classification tasks, based on a dimensional model of emotions: Valenceand Arousal. It aims to improve the accuracy of the emotion classification task by combining mutual information based feature selection methods and kernel classifiers. Experiments using our approach for emotion classification which combines efficient feature selection methods and efficient kernel-based classifiers on standard EEG datasets show the promise of the approach when compared with state-of-the-art computational methods.     

Subject adaptation network for EEG data analysis

Biosignals tend to display manifest intra- and cross-subject variance, which generates numerous challenges for electroencephalograph (EEG) data analysis. For instance, in the context of classification, the discrepancy between EEG data can make the trained model generalising poorly for new test subjects. In this paper, a subject adaptation network (SAN) inspired by the generative adversarial network (GAN) to mitigate different variances is proposed for analysing EEG data. First the challenges faced by traditional approaches employed for EEG signal processing are emphasised. Then the problem is formulated from mathematical perspective to highlight the key points in resolving such discrepancies. Third, the motivation behind and design principle of the SAN are described in an intuitive manner to reflect its suitability for analysing EEG data. Then after depicting the overall architecture of the SAN, several experiments are used to justify the practicality and efficiency of using the proposed model from different perspectives. For instance, an EEG dataset captured during a stereotypical neurophysiological experiment called the VEP oddball task is utilised to demonstrate the performance improvement achieved by running the SAN.     

Deep learning-based construction equipment operators’ mental fatigue classification using wearable EEG sensor data

Operator attention failure due to mental fatigue during extended equipment operations is a common cause of equipment-related accidents that result in catastrophic injuries and fatalities. As a result, tracking operators' mental fatigue is critical to reducing equipment-related accidents on construction sites. Previously, several strategies aimed at recognizing mental fatigue with adequate accuracy, such as machine learning utilizing EEG-based wearable sensing systems, have been proposed. However, the ability to track operators’ mental fatigue for its implementation on an actual construction site is still an issue. For instance, the mobility and systemic instability of EEG sensors necessitate their application in laboratory settings rather than on actual construction sites. Furthermore, while the machine learning classifiers achieved acceptable accuracy, their input is limited to manually developed EEG features, which may compromise the models’ performance on real construction sites. Accordingly, the current research proposes the viability of a construction site strategy that uses flexible headband-based sensors for acquiring raw EEG data and deep learning networks to recognize operators' mental fatigue. To serve this purpose, a one-hour excavator operation by fifteen operators was conducted on a construction site. The NASA-TLX score was used as the ground truth of mental fatigue, and brain activity patterns were recorded using a wearable EEG sensor. The raw EEG data was then used to develop deep learning-based classification models. Finally, the performance of deep learning models, i.e., long short-term memory, bidirectional LSTM, and one-dimensional convolutional networks, was investigated using accuracy, precision, recall, specificity, and an F1-score. The findings indicate that the Bi-LSTM model outperforms the other deep learning models with a high accuracy of 99.941% and F1-score between 99.917% and 99.993%. These findings demonstrate the feasibility of applying the Bi-LSTM model and contribute to wearable sensor-based mental fatigue recognition and classification, thus enhancing on-site health and safety operations.     

Prediction of product design decision Making: An investigation of eye movements and EEG features

Design decision making is happened in every design node and iteration, and the expert decision-making bias and personal preference will ultimately affect the success or failure of the product reaching the market. In this paper, we try to predict the design decision making by investigating the relations between design decision making and subjects’ eye movements and Electroencephalogram(EEG) response. Four different methods were applied and compared to classify the different EEG features and two methods were used for EEG feature selection to correspond the design decision making results. In this study, the authors applied a multimodal fusion strategy for design decision making recognition where the authors used eye tracking and EEG response data as input dataset. According to the experiment results, the performance of the fusion strategy combined with EEG signals and eye movement characteristics is well in fitting the expert decision making results. The multimodal fusion combining eye tracking data and EEG has a strong potential to be a new design decision method to guide the design practice and provide supportive and objective data to reduce the effects of subjectivity, one-sidedness and superficiality in decision making. These results show that it is possible to create a classifier based on features extracted from eye movements and EEG response for the design decision making behaviour.