Emotion recognition from thermal infrared images using deep Boltzmann machine

Facial expression and emotion recognition from thermal infrared images has attracted more and more attentions in recent years. However, the features adopted in current work are either temperature statistical parameters extracted from the facial regions of interest or several hand-crafted features that are commonly used in visible spectrum. Till now there are no image features specially designed for thermal infrared images. In this paper, we propose using the deep Boltzmann machine to learn thermal features for emotion recognition from thermal infrared facial images. First, the face is located and normalized from the thermal infrared im- ages. Then, a deep Boltzmann machine model composed of two layers is trained. The parameters of the deep Boltzmann machine model are further fine-tuned for emotion recognition after pre-tralning of feature learning. Comparative experimental results on the NVIE database demonstrate that our approach outperforms other approaches using temperature statistic features or hand-crafted features borrowed from visible domain. The learned features from the forehead, eye, and mouth are more effective for discriminating valence dimension of emotion than other facial areas. In addition, our study shows that adding unlabeled data from other database during training can also improve feature learning performance.     

基于复合优化的深度玻尔兹曼机的路牌文字图像识别算法

为提高自然场景中路牌文字图像的识别率,提出一种复合优化的深度玻尔兹曼机文字识别算法。算法以提高目标概率分布的逼近程度为目的,采用两种抽样初始化方法:灰度初始化抽样与二值初始化抽样,构造受限玻尔兹曼机,并由两种初始化方法的受限玻尔兹曼机交叠构成深度玻尔兹曼机。文中提出复合共轭梯度法改进深度玻尔兹曼机的微调算法。实验结果表明,使用文中获取的路牌文字数据,所提算法能够对路牌文字实现较理想的识别效果。与原深度玻尔兹曼机相比,识别率取得有效提高。     

基于深度卷积限制玻尔兹曼机的步态识别

传统的步态识别方法难以得到有效的步态特征,而深度学习方法可以通过学习自动获得特征,然而现有的深度学习模型用于步态识别时存在一些问题。深度卷积神经网络训练速度快,但训练精度较低;深度置信网络模型精度较高,但模型收敛速度较慢。针对这两种模型的特点,提出一种两者平衡的算法模型,即深度卷积限制玻尔兹曼机。将卷积神经网络中权值共享、提取图像局部特征等方面的优势融入深度玻尔兹曼机模型中,提高训练精度,减少参数数量。所提算法在CASIA步态数据库上的实验结果验证了该算法在步态识别问题上的有效性和可行性。     

Fusion of visible and thermal images for facial expression recognition

Most present research into facial expression recognition focuses on the visible spectrum, which is sensitive to illumination change. In this paper, we focus on integrating thermal infrared data with visible spectrum images for spontaneous facial expression recognition. First, the active appearance model AAM parameters and three defined head motion features are extracted from visible spectrum images, and several thermal statistical features are extracted from infrared (IR) images. Second, feature selection is performed using the F-test statistic. Third, Bayesian networks BNs and support vector machines SVMs are proposed for both decision-level and feature-level fusion. Experiments on the natural visible and infrared facial expression (NVIE) spontaneous database show the effectiveness of the proposed methods, and demonstrate thermal IR images’ supplementary role for visible facial expression recognition.     

基于深度玻尔兹曼机的乐器分类问题研究

应用传统浅层模型处理乐器分类任务存在非线性拟合能力较差的问题,使分类准确率得不到有效保证,有必要引入深度学习方法提升复杂任务的非线性建模能力。将深度玻尔兹曼机作为特征提取器提取表达能力更强的数据特征,分别以SVM与Softmax分类器作为深度神经网络的顶层设置形成DBM SVM组合模型与DBM Softmax组合模型,引入平均场理论和动量项因子优化网络训练过程。将上述两组模型及单一SVM分类器在5类乐器音频数据上进行对比实验,两种深度学习组合模型的分类准确率分别达到89.29%和87.5%,与传统浅层分类方法SVM的73.21%的准确率相比优势明显。实验结果表明深度玻尔兹曼机在乐器分类领域的应用颇具前景。     

采用瞳孔位置实现情感识别的方案

目前多种眼部信息被用于情感识别,针对一般眼部信息如注视时间、瞳孔直径、扫视时间等数据需要使用专业的眼动仪采集这一问题,提出了一种基于瞳孔位置的情感识别方案.该方案采用基于梯度的瞳孔定位算法,直接从面部视频中获取瞳孔位置坐标数据.对于预处理后的数据,又提出了综合波形复杂度作为特征值,通过对瞳孔位置坐标波形进行分段,求取所有分段之间的相关性系数绝对值之和,从而得到波形的综合复杂度.实验结果表明,瞳孔位置坐标可以用来进行情感识别,且对于瞳孔位置坐标波形,综合波形复杂度特征值情感识别的效果优于样本熵和Lemple-Ziv复杂度.今后可将瞳孔位置坐标与其他生理信号进行多模态融合,提高情感识别准确率.     

结合深度学习与支持向量机的金属零件识别

目的 在视觉引导的工业机器人自动拾取研究中，关键技术难点之一是机器人抓取目标区域的识别问题。特别是金属零件，其表面的反光、随意摆放时相互遮挡等非结构化因素都给抓取区域的识别带来巨大的挑战。因此，本文提出一种结合深度学习和支持向量机的抓取区域识别方法。方法 分别提取抓取区域的方向梯度直方图（HOG）和局部二进制模式（LBP）特征，利用主成分分析法（PCA）对融合后的特征进行降维，以此来训练支持向量机（SVM）分类器。通过训练Mask R-CNN（regions with convolutional neural network）神经网络完成抓取区域的初步分割。然后利用SVM对Mask R-CNN识别的抓取区域进行二次分类，完成对干扰区域的剔除。最后计算掩码完成实例分割，以此达到对抓取区域的精确识别。结果 对于随机摆放的铜质金属零件，本文算法与单一的Mask R-CNN及多特征融合的SVM算法就识别准确率、错检率、漏检率3个指标进行了比较，结果表明本文算法在识别准确率上较Mask R-CNN和SVM算法分别提高了7%和25%，同时有效降低了错检率与漏检率。结论 本文算法结合了Mask R-CNN与SVM两种方法，对于反光和遮挡情况具有一定的鲁棒性，同时有效地提升了目标识别的准确率。     

用遗传算法优化Boltzmann机

Boltzmann机是一种应用广泛的随机神经网络。它通过模拟退火算法进行网络学习,能取得一个全局或接近全局最优的最优值;通过期望网络模式和实际学习得到的网络模式比较来调节网络的权值,使网络能尽可能地达到或逼近期望的网络模式。将遗传算法运用到Boltzmann机的网络学习中,在对BM机编码后,通过选择、交叉和变异等遗传操作算子对网络进行训练,调整网络的权值,使适应度函数值大的网络保留下来,最终使网络达到期望的模式。通过实例验证,这是一种简单可行的调节网络权值的方法。     

级联型P-RBM神经网络的人脸检测

目的 针对非理想条件下快速准确的人脸检测问题,提出一种基于概率态多层受限玻尔兹曼机(RBM)级联神经网络的检测方法。方法 它采用RBM中神经元的概率态表征来模拟人脑神经元连续分布的激活状态,并且利用多层P-RBM(概率态RBM)级联来仿真人脑对视觉的层次学习模式,又以逐层递减隐藏层神经元数来控制网络规模,最后采用分层训练和整体优化的机制来缓解鲁棒性和准确性的矛盾。结果 在LFW、FERET、PKU-SVD-B以及CAS-PEAL数据集上的测试都实现了优于现有典型算法的检测性能。对于单人脸检测,相比于Adaboost算法,将漏检率降低了2.92%;对于多人脸检测,相比于结合肤色的Adaboost算法,将误检率降低了14.9%,同时漏检率降低了5.0%,检测时间降低了50%。结论 无论是静态单张人脸,还是复杂条件下视频多人脸检测,该方法不仅在误检率和漏检率上表现更好,而且具有较快的检测速度,同时对于旋转人脸检测具有较强的鲁棒性。针对基于肤色的多人脸检测研究,该方法能显著降低误检率。     

Shape-based object extraction in high-resolution remote-sensing images using deep Boltzmann machine

In this article, we proposed a novel method based on deep learning shape priors for object extraction in high-resolution (HR) remote-sensing images. Specifically, the deep Boltzmann machines (DBMs) are applied to model the shape priors via the unsupervised training process, which qualify for the advantages of deep learning method, especially the powerful feature learning and modelling ability. The deep shape model is integrated into a new energy function to eliminate the influence of disturbing background. The energy function combines image appearance information and region information. A new region term in the function is proposed to eliminate the influence of object shadow. The process of object extraction is achieved by minimizing the energy function with an iterative optimization algorithm and the Split Bregman method is applied to derive a global solution during the minimization process. Quantitative and qualitative experiments are conducted on the aircraft data set acquired by QuickBird with 60 cm resolution and the results demonstrate the effectiveness of the proposed method.     

基于ToF红外图像的手部轻量化检测算法设计与优化

嵌入式设备上实现快速精准的手部检测主要面临两个挑战：一是复杂的深度学习网络很难实现实时的手部检测;二是场景复杂性导致基于RGB彩色图像的手部检测算法准确率下降。与主流基于RGB图像的检测技术不同,基于ToF红外图像的轻量化手部检测算法实现了红外图像中手部的精准快速检测。首先,通过自主研发设备采集了22 419张静态红外图片,构建了用于手部检测的红外数据集;其次,通过对通用目标检测算法进行轻量化改进,设计了RetinaHand轻量化手部检测网络,其中采用了MobileNetV1和ShuffleNetV2两种不同的轻量化网络作为模型骨干网络,并提出了一种融合注意力机制的特征金字塔结构Attention-FPN;最后,在红外数据集上与常规方法进行了对比实验,验证了该方法的有效性。     

Lattice Boltzmann equation for axisymmetric thermal flows

A simple lattice Boltzmann equation (LBE) model for axisymmetric thermal flow is proposed in this paper. The flow field is solved by a quasi-two-dimensional nine-speed (D2Q9) LBE, while the temperature field is solved by another four-speed (D2Q4) LBE. The model is validated by a thermal flow in a pipe and some nontrivial thermal buoyancy-driven flows in vertical cylinders, including Rayleigh-Bénard convection, natural convection, and heat transfer of swirling flows. It is found that the numerical results agree excellently with analytical solution or other numerical results.     

A crosswalk pedestrian recognition system by using deep learning and zebra-crossing recognition techniques

Pedestrian detection is essential for improving pedestrian safety in an intelligent traffic system. The efficiency of the system is affected by real-time processing and the error rate of detection. These concerns have not been completely addressed in previous studies. Therefore, this study proposes a real-time pedestrian recognition system that ensures high accuracy by using a deep learning classifier and zebra-crossing recognition techniques. The proposed system was designed to improve pedestrian safety and reduce accidents at intersections. Environmental feature vectors were first used to detect zebra crossings and to determine crossing areas. An adaptive mapping technique was then used to map the pedestrian waiting area based on the crossing area. A dual camera mechanism was used to maintain detection accuracy and improve system fault tolerance. Finally, the you-only-look-once model was used to recognize pedestrians at intersections. A system prototype was implemented to verify the feasibility of the proposed system. The results revealed that the proposed scheme outperforms the conventional histogram of oriented gradients and Haarcascade schemes.     

Gender recognition from visible and thermal infrared facial images

Most present research of gender recognition focuses on visible facial images, which are sensitive to illumination changes. In this paper, we proposed hybrid methods for gender recognition by fusing visible and thermal infrared images. First, the active appearance model is used to extract features from visible images, as well as local binary pattern features and several statistical temperature features are extracted from thermal infrared images. Then, feature selection is performed by using the F-test statistic. Third, we propose using Bayesian Networks to perform explicit and implicit fusion of visible and thermal infrared image features. For explicit fusion, we propose two Bayesian Networks to perform decision-level and feature-level fusion. For implicit fusion, we propose using features from one modality as privileged information to improve gender recognition by another modality. Finally, we evaluate the proposed methods on the Natural Visible and Infrared facial Expression spontaneous database and the Equinox face database. Experimental results show that both feature-level and decision-level fusion improve the gender recognition performance, compared to that achieved from one modality. The proposed implicit fusion methods successfully capture the role of privileged information of one modality, thus enhance the gender recognition from another modality.     

模糊支持向量机情感状态识别的研究*

针对已有的情感生理参数样本类内聚合度低、不同状态较难区分的特点,提出了一种改进的模糊支持向量机识别方法.模糊隶属度函数采用高斯分布形式,高斯分布的参数分别由同类样本数据形成的最小超球体半径和样本之间的紧密程度决定.该方法计算样本模糊隶属度时,不仅考虑样本与类中心的距离关系,还要考虑样本与样本之间的关系.实验显示改进的模...     

遮挡人脸表情识别深度学习方法研究进展

在真实环境下遮挡是准确分析识别人脸表情的主要障碍之一。近年来研究者采用深度学习技术解决遮挡条件下表情误识别率高的问题。针对遮挡表情识别的深度学习算法和遮挡相关的问题进行归纳总结。首先,概括局部遮挡条件下表情识别的发展现状、表情的表示方式以及研究遮挡表情用到的数据集;其次,回顾遮挡表情识别深度学习方法的最新进展和分析遮挡对表情的影响;最后,总结主要技术挑战,研究难点及其可能的应对策略。目的是为将来的遮挡表情识别研究提供更有益的参考依据和基准。     

文本情感原因自动提取综述

情感原因提取是情感计算领域研究的一个新方向,是一种细粒度的情感分析,其目的是要找出给定文档中触发情感的那部分文本,是对情感的一种追根溯源.情感原因提取涉及到语言学、心理学等相关的领域知识,具有较高的学术研究价值和广泛的应用场景.尽管情感计算的相关研究大多集中在情感识别、情感预测、情感信息抽取等方面,但近些年不少学者已开始深入到情感背后的原因分析与提取上,并产生了较为丰富的成果.从问题定义、任务类别、研究方法、主流数据集、评测指标等多个角度对基于文本的情感原因自动提取的研究成果进行全面回顾和分析,重点对情感原因提取的方法特别是基于深度学习的方法进行了梳理,最后总结了现有情感原因提取工作的不足及其未来所面临的挑战.     

基于深度自编码器子域自适应的跨库语音情感识别

针对不同语料库之间数据分布差异问题,提出一种基于深度自编码器子域自适应的跨库语音情感识别算法.首先,该算法采用两个深度自编码器分别获取源域和目标域表征性强的低维情感特征;然后,利用基于LMMD(local maximum mean discrepancy)的子域自适应模块,实现源域和目标域在不同低维情感类别空间中的特征分布对齐;最后,使用带标签的源域数据进行有监督地训练该模型.在eNTERFACE库为源域、Berlin库为目标域的跨库识别方案中,所提算法的跨库识别准确率相比于其他算法提升了5.26％～19.73％;在Berlin库为源域、eNTERFACE库为目标域的跨库识别方案中,所提算法的跨库识别准确率相比于其他算法提升了7.34％～8.18％.因此,所提方法可以有效地提取不同语料库的共有情感特征并提升了跨库语音情感识别的性能.     

基于对比散度受限玻尔兹曼机深度学习的产品评论情感分析

针对目前大部分情感分析技术需要人工标注建立情感词典提取情感特征的问题,提出一种基于对比散度-受限玻尔兹曼机(CD-RBM)深度学习的产品评论情感分析方法。该方法在对产品评论时进行数据预处理并利用词袋模型产生产品评论的向量表示,然后通过CD-RBM提取产品评论的情感特征,最后结合支持向量机(SVM)将提取出来的情感特征进行文本情感分类。CD-RBM无需人工标注情感词典,即可获得情感特征,且可以提高特征的情感语义关联性;同时,SVM可以保证产品评论情感分类的准确度。通过实验确定了RBM最优训练周期为10,在此训练周期下对RBM、SVM、PCA+SVM,以及RBM+SVM方法进行了比较。实验结果表明,RBM特征提取和SVM分类结合方法能够获得最好的准确率和F值,并获得较好的召回率。     

深度学习支持下的情感识别探讨

随着人工神经网络技术的发展,深度学习逐渐成为人工智能领域的研究重点。在情感识别领域,深度学习也有着极为强大的理论与技术优势,其能够显著提升情感识别的准确性。同时,深度学习下的情感识别对于智慧化学习环境的营造有着积极意义,所以将之应用到教育教学工作中是切实可行的。本文对深度学习进行了简单介绍,并分析了当前情感识别的研究进展,进而对深度学习支持下的情感识别构建和应用展开探讨。