Spatiotemporal emotion recognition based on 3D time-frequency domain feature matrix

The research of emotion recognition based on electroencephalogram (EEG) signals often ignores the relatedinformation between the brain electrode channels and the contextual emotional information existing in EEG signals,which may contain important characteristics related to emotional states. Aiming at the above defects, aspatiotemporal emotion recognition method based on a 3-dimensional (3D) time-frequency domain feature matrixwas proposed. Specifically, the extracted time-frequency domain EEG features are first expressed as a 3D matrixformat according to the actual position of the cerebral cortex. Then, the input 3D matrix is processed successivelyby multivariate convolutional neural network (MVCNN) and long short-term memory (LSTM) to classify theemotional state. Spatiotemporal emotion recognition method is evaluated on the DEAP data set, and achievedaccuracy of 87.58% and 88.50% on arousal and valence dimensions respectively in binary classification tasks, aswell as obtained accuracy of 84.58% in four class classification tasks. The experimental results show that 3D matrixrepresentation can represent emotional information more reasonably than two-dimensional (2D). In addition,MVCNN and LSTM can utilize the spatial information of the electrode channels and the temporal context information of the EEG signal respectively.     

抑郁症正性情绪加工脑电样本熵异常

本文通过研究抑郁症患者与正常人在处理不同情绪刺激时脑电信号样本熵的差异,探索抑郁症患者情绪加工异常的电生理机制。我们招募了16名抑郁症患者和14名健康对照组参与面部表情空间搜索任务,同时采集了他们完成任务时的头皮脑电信号。我们首先选用希尔伯特-黄变换获取脑电的各频段活动;然后通过比较抑郁症患者与健康对照组脑电的样本熵来研究两组受试者不同情绪加工的电生理差异;最后选取β频段样本熵作为特征,采用不同分类器和不同提取方式进行分类研究。结果反映,抑郁症患者在情绪加工上,尤其是正性情绪的认知加工上存在异常。同时也表明样本熵在一定程度上可以反映不同条件情绪加工脑电的特异性,可作为一种区分正常人与抑郁症患者的潜在的特征指标,用于抑郁症患者的辅助分类识别,为医生诊断抑郁症患者提供一种辅助方案。      

Differences of EEG between Eyes-Open and Eyes-Closed States Based on Autoregressive Method

Autoregressive （AR） power spectral density estimate method was used to analyze the electroencephalogram （EEG） signals in eyes-open and eyes-closed states. From the topographical distributions of delta, theta, alpha, and beta power spectrum, these two states can be clearly discriminated. In these two states, frontal areas were activated in delta power, both frontal and occipital areas were activated in theta band, and occipital areas were activated in alpha and beta bands. These four bands had significantly higher power in frontal, parietal, and occipital areas when eyes were close. The results also implied that the optimum order of AR model could be more suitable for estimating EEG power spectrum of different states.     

Increased Arousal Levels and Decreased Sleep by Brain Music in Rats

More and more studies have been reported on whether music and other types of auditory stimulation would improve the quality of sleep. Many of these studies have found significant results, but others argue that music is not significantly better than the tones or control conditions in improving sleep. For further understanding the relationship between music and sleep or music and arousal, the present study therefore examines the effects of brain music on sleep and arousal by means of biofeedback. The music is from the transformation of rapid eye movement (REM) sleep electroencephalogram (EEG) of rats using an algorithm in the Chengdu Brain Music (CBM) system. When the brain music was played back to rats, EEG data were recorded to assess the efficacy of music to induce or improve sleep, or increase arousal levels by sleep staging, etc. Our results demonstrate that exposure to the brain music increases arousal levels and decreases sleep in rats, and the underlying mechanism of decreased non-rapid eye movement (NREM) and REM sleep may be different.     

Increased Arousal Levels and Decreased Sleep by Brain Music in Rats

More and more studies have been reported on whether music and other types of auditory stimulation would improve the quality of sleep. Many of these studies have found significant results, but others argue that music is not significantly better than the tones or control conditions in improving sleep. For further understanding the relationship between music and sleep or music and arousal, the present study therefore examines the effects of brain music on sleep and arousal by means of biofeedback. The music is from the transformation of rapid eye movement （REM） sleep electroencephalogram （EEG） of rats using an algorithm in the Chengdu Brain Music （CBM） system. When the brain music was played back to rats, EEG data were recorded to assess the efficacy of music to induce or improve sleep, or increase arousal levels by sleep staging, etc. Our results demonstrate that exposure to the brain music increases arousal levels and decreases sleep in rats, and the underlying mechanism of decreased non-rapid eye movement （NREM） and REM sleep may be different.     

Emotional effect of cinematic VR compared with traditional 2D film

Virtual reality (VR) has been increasingly applied in filmmaking in recent years. The integration of film and VR is becoming an important breakthrough for traditional films. This new types of film, named cinematic virtual reality (CVR), provides immersive VR experience where individual users can immerse themselves in synthetic world experience in 360° (Mateer, 2017).However, the characteristics and influence of this new digital media on human emotion is far from clear. In this study, we conducted an experiment to investigate the different emotional effects of CVR by comparing with a traditional two-dimensional (2D) film from two critical aspects: subjective emotional experience and real-time objective physiological reaction. Our results first revealed that the subjective experience and the physiological reaction showed significantly stronger emotional effect in the CVR condition than in the 2D condition. Upon further observation, we found that four emotions (excitement, nervousness, hostility, and jitteriness) are more closely correlated with the CVR than with the traditional 2D film. Real-time analysis of skin temperature shows a faster and steadier decline in the CVR group than in the 2D group. Finally, we summarized the causality of the effect by analyzing the characteristics of CVR. Our study suggests that CVR is an effective medium that induces stronger emotional experience and physiological reaction than traditional 2D film can. The design of the VR environment in CVR, as a key feature in the narrative of a film story, has an important influence on the emotional processing of the audience.     

基于时域多分辨法微带线串扰分析

时域多分辨分析法作为一种时域计算方法,其吸收边界直接影响到计算的准确度。采用具有紧支撑性和对称性的CDF(2,6)尺度函数作为基函数得到了三维各向异性完全匹配层吸收边界;将时域多分辨分析法应用于微带线串扰分析中,给出了适用于任意尺度函数的集总电阻和阻抗电压源模拟方法,并用该方法分析了某印刷电路板上两根平行微带线的串扰问题。仿真结果表明:与传统的时域有限差分算法相比,以CDF(2,6)尺度函数为基函数的时域多分辨分析法只需要其一半的网格数,计算速度提高三倍,同时具有内存使用少、利用率高等特点。     

实现感兴趣区域编码的通用部分位平面偏移法

提出一种通用的部分位平面偏移方法(GPBShift),可克服JPEG2000中定义的两种标准感兴趣区域(ROI)编码方法的局限性。与标准方法中将全部位平面用统一的偏移值进行移位不同,该方法将ROI系数和背景(BG)系数的位平面分别划分成两部分,进行不同的位平面偏移,以控制ROI和BG区的相对重要性。GPBShift方法兼容Maxshift、GBbBShift和PSBShift3种方法,并提供比上述3法更大的灵活性。它不仅能够在不传输任何形状信息的情况下,对任意形状的ROI进行编码,而且通过选择偏移值,能灵活调整ROI和BG区的相对压缩质量。此外,它能够根据不同的优先级,编码多个ROI区域。实验结果显示：该方法在低码率时,能提供比Maxshift方法更好的视觉质量,且比标准中的一般偏移方法(general scaling based method)具有更高的编码效率。     

基于有记忆递归神经网络的脑电特征情感识别研究

为了提高脑电信号多分类的情感识别率,文中选用上海交通大学提供的SEED脑电信号数据集,对其进行分频带特征提取。将脑电数据的微分熵特征、微分不对称性特征和有理不对称性特征通过线性动力系统平滑特征后,与功率谱密度特征进行分类效果比较,再利用有记忆递归神经网络的方法进行分类,发现提取的微分熵特征经过分类的效果好。在对3种情感进行分类的过程中,采用长短时记忆神经网络分类相比于其他机器学习方法识别率有所提高,情感识别的平均准确率可达到95.045 9%。     

基于同步性脑网络的支持张量机情绪分类研究

一直以来,情绪是心理学、教育学、信息科学等多个学科的研究热点,脑电信号(EEG)因其客观、不易伪装的特点,在情绪识别领域受到广泛关注。由于人类情绪是大脑多个脑区相互作用产生的,该文提出一种基于同步性脑网络的支持张量机情绪分类算法(SBN-STM),该算法采用相位锁定值(PLV)构建了同步性脑网络,分析多导联脑电信号之间的同步性和相关性,并生成2阶张量序列作为训练集,运用支持张量机(STM)模型实现正负情绪的二分类。该文基于DEAP脑电情绪数据库,详细分析了同步性脑网络张量序列的选取方法,最佳张量序列窗口的大小和位置,解决了传统情绪分类算法特征冗余的问题,提高了模型训练速度。仿真实验表明,基于支持张量机的同步性脑网络分类方法的情绪准确率优于支持向量机、C4.5决策树、人工神经网络、K近邻等以向量为特征的情绪分类模型。     

Conditional generative adversarial network for EEG-based emotion fine-grained estimation and visualization

In the field of affective computing (AC), coarse-grained AC has been developed and widely applied in many fields. Electroencephalogram (EEG) signals contain abundant emotional information. However, it is difficult to develop fine-grained AC due to the lack of fine-grained labeling data and suitable visualization methods for EEG data with fine labels. To achieve a fine mapping of EEG data directly to facial images, we propose a conditional generative adversarial network (cGAN) to establish the relationship between EEG data associated with emotions, a coarse label, and a facial expression image in this study. In addition, a corresponding training strategy is also proposed to realize the fine-grained estimation and visualization of EEG-based emotion. The experiments prove the reasonableness of the proposed method for the generation of fine-grained facial expressions. The image entropy of the generated image indicates that the proposed method can provide a satisfactory visualization of fine-grained facial expressions.     

MRTD算法的APML实现及其在光波导仿真中的应用

实现了基于Daubechies紧支集尺度函数的时域多分辨分析(MRTD)算法的各向异性理想匹配层(APML)吸收边界条件,并将其应用到平面光波导的仿真和分析中。验证结果表明,APML吸收层性能主要由其层数和计算空间步长所决定。与传统的时域有限差分(FDTD)法相比,基于高阶消失矩Daubechies尺度函数的MRTD法可以提高吸收层性能。     

采用原子表示模型的维吾尔语语音情感识别

针对现有的基于表示学习的语音情感计算算法中存在着限制条件单一的问题,且没有证明它们的有效性,提出了一种采用原子表示模型的语音情感识别算法。通过引入一个新的条件,称为原子分类条件。在这种条件下,对正确识别新的测试情感样本有较好的效果。现有的基于表示的分类算法以单一的稀疏表示方法为主,而提出的算法可以结合稀疏表示模型和其他的表示模型。该算法能够放宽适用条件的范围,使得原子表示模型适应更多分类任务。采集并建立了维吾尔语语音情感数据库。在该情感数据库上,分析维吾尔语情感语音的基本声学特征。通过对情感特征空间进行原子表示的映射变换,可以有效表示情感特征空间。经实验结果证明所提出的方法优于传统的方法,在维吾尔语情感语音数据库上达到了64.17%识别率。      

Speech emotion recognition using modified quadratic discrimination function

Quadratic Discrimination Function （QDF） is commonly used in speech emotion recognition, which proceeds on the premise that the input data is normal distribution. In this paper, we propose a transformation to normalize the emotional features, emotion recognition. Features based on prosody then derivate a Modified QDF （MQDF） to speech and voice quality are extracted and Principal Component Analysis Neural Network （PCANN） is used to reduce dimension of the feature vectors. The results show that voice quality features are effective supplement for recognition, and the method in this paper could improve the recognition ratio effectively.     

User emotion analysis in conflicting versus non-conflicting regions using online social networks

Society plays a vital role in maintaining the emotional health of an individual. People living in conflicting zones are emotionally degraded and often hold more negativity than the people living in serene areas. Nowadays, analyzing the emotions shared on social networking websites is an ongoing topic of research. In this paper, we presented the potential of user data available on the Facebook website in distinguishing the emotions of netizens in conflicting versus non-conflicting areas. We collected the Facebook posts of the users living in Kashmir (conflicting region) and Delhi (non-conflicting) with the help of two source accounts. Plutchik’s eight basic emotions, namely, fear, anger, sadness, joy, surprise, disgust, trust and anticipation have been used to determine the emotion state of a user. Based on two well-known lexicons, namely, EmoLex and Empath, a new dictionary called MoodBook is designed to determine the user emotions from their posts.After analyzing the data, we found that violence in the conflicting region has badly affected the psychology of the citizens as most of the people in Kashmir fall under three negative emotion categories, namely, fear, anger, and sadness, whereas the joy mood has been found more in the posts of Delhi-based users. Furthermore, a mood-vector is created for each user and used as an input to k-means clustering where it has been found that the citizens of two regions form separate groups based on their psychological state. The study of the difference between emotions expressed online by the citizens of conflicting and non-conflicting has not been seen in the literature till date.     

Violence in Music Videos: Examining the Prevalence and Context of Physical Aggression

The purpose of this study was to examine violence in music video programming. Using a representative sample of television content, we assessed whether the amount and context of physical aggression varied across different music video channels (BET, MTV, VH-1) and genres (adult contemporary, heavy metal, rap rhythm and blues, and rock). The results reveal that 15% of music videos feature violence, and most of that aggression is sanitized, not chastised, and presented in realistic contexts. Significant differences emerged in the prevalence and nature of violence by channel and genre, however. The findings are discussed in terms of the risk that exposure to violence in each channel and genre may be posing to viewers' learning of aggression, fear, and emotional desensitization.     

MEF融合HFF的戏剧视频关键情节自动提取

为了更好地从戏剧视频提取关键情节,提出了一种基于音乐情感特征(MEF)融合人脸特征(HFF)的自动提取方法.首先,利用基于音频指纹技术的二级音乐情感识别方法进行音频识别,并利用人脸特征进行视频识别;然后,利用音频和视频识别得到的各元素获取关键情节值,从而提取关键情节;最后,提出了一种量化评估方法评估关键情节提取方法的一致性.在四个戏剧视频上的评估实验验证了该方法的有效性及可靠性,相比其他几种较新的提取模型,该方法提取效果更好.     

Fourier-Based Feature Extraction for Classification of EEG Signals Using EEG Rhythms

In this paper, we propose a method for the analysis and classification of electroencephalogram (EEG) signals using EEG rhythms. The EEG rhythms capture the nonlinear complex dynamic behavior of the brain system and the nonstationary nature of the EEG signals. This method analyzes common frequency components in multichannel EEG recordings, using the filter bank signal processing. The mean frequency (MF) and RMS bandwidth of the signal are estimated by applying Fourier-transform-based filter bank processing on the EEG rhythms, which we refer intrinsic band functions, inherently present in the EEG signals. The MF and RMS bandwidth estimates, for the different classes (e.g., ictal and seizure-free, open eyes and closed eyes, inter-ictal and ictal, healthy volunteers and epileptic patients, inter-ictal epileptogenic and opposite to epileptogenic zone) of EEG recordings, are statistically different and hence used to distinguish and classify the two classes of signals using a least-squares support vector machine classifier. Experimental results, with 100 % classification accuracy, on a real-world EEG signals database analysis illustrate the effectiveness of the proposed method for EEG signal classification.     

Listening to the Brain in Real Time-- The Chengdu Brainwave Music

Electroencephalogram （EEG） provides a window for the activity of the human brain. As a novel form of the brain-computer interface （BCI）, the online/offline EEG data may be interpreted through its auditory representation which can be considered as a specific tool in EEG monitoring and analysis. In this work, after a comprehensive comparison of the various designs of brainwave music generations, a waveform event mapping system for music display in real time-- the Chengdu Brainwave Music （CBM） is proposed, which is a special on-line BCI system. In CBM, the user datagram protocol （UDP） is adopted to transport EEG data from the recorder to a music generator. The CBM could possibly be used as an audio feedback tool in BCI, or a monitoring tool in clinic EEG, and a subject specified music therapy method.     

Listen to the Brain in Real Time-The Chengdu Brainwave Music

Electroencephalogram (EEG) provides a window for the activity of the human brain. As a novel form of the brain-computer interface (BCI), the online/offline EEG data may be interpreted through its auditory representation which can be considered as a specific tool in EEG monitoring and analysis. In this work, after a comprehensive comparison of the various designs of brainwave music generations, a waveform event mapping system for music display in real time-the Chengdu Brainwave Music (CBM) is proposed, which is a special on-line BCI system. In CBM, the user datagram protocol (UDP) is adopted to transport EEG data from the recorder to a music generator. The CBM could possibly be used as an audio feedback tool in BCI, or a monitoring tool in clinic EEG, and a subject specified music therapy method.