基于3D卷积神经网络的活体人脸检测

非法入侵者通过伪装人脸骗取系统认证,给人脸认证系统带来了严重的威胁。因此,活体人脸检测成了人脸认证系统走向实用必须解决的一个重要课题。现有活体人脸检测方法多为基于照片的人脸攻击方面的研究成果,对于基于视频的人脸攻击,效果并不理想。3D卷积神经网络（Convolutional Neural Network,CNN）具有深度学习的特点,能自动学到图像的分布式特征表示;与2D卷积相比,它能学到连续视频帧的动作信息。本文结合3D卷积神经网络的特性,提出利用3D卷积实现视频人脸伪装检测。通过提取3D卷积神经网络最后全连接层学到的时间空间特征,训练SVM（Support Vector Machine）分类器,实现真实人脸和伪装人脸的分类。实验采用两个人脸伪装公开数据库ReplayAttack和CASIA,实现多尺度内部数据库测试和交叉数据库测试。实验结果相对于纹理特征及2D卷积方法有较大提高,可应用于视频人脸攻击的活体人脸检测。      

Integration of image quality and motion cues for face anti-spoofing: A neural network approach

Many trait-specific countermeasures to face spoofing attacks have been developed for security of face authentication. However, there is no superior face anti-spoofing technique to deal with every kind of spoofing attack in varying scenarios. In order to improve the generalization ability of face anti-spoofing approaches, an extendable multi-cues integration framework for face anti-spoofing using a hierarchical neural network is proposed, which can fuse image quality cues and motion cues for liveness detection. Shearlet is utilized to develop an image quality-based liveness feature. Dense optical flow is utilized to extract motion-based liveness features. A bottleneck feature fusion strategy can integrate different liveness features effectively. The proposed approach was evaluated on three public face anti-spoofing databases. A half total error rate (HTER) of 0% and an equal error rate (EER) of 0% were achieved on both REPLAY-ATTACK database and 3D-MAD database. An EER of 5.83% was achieved on CASIA-FASD database.     

基于特征金字塔融合表征网络的跨模态哈希方法

随着多模态数据的爆发式增长,跨模态检索作为一种搜索多模态数据的最常用方法,受到越来越多的关注。然而,目前存在的大多数深度学习的方法仅仅采用模型后端最后一个全连接层输出作为模态独有的高层语义表征,忽视了多个层次上不同尺度特征之间的语义相关性,具有一定的局限性。为此,本文提出一种基于特征金字塔融合表征网络的跨模态哈希检索方法。该方法设计了一种特征金字塔融合表征网络,通过在多个层次和不同尺度上进行特征提取并融合,挖掘多个层次上不同尺度下模态特征的语义相关性,充分利用模态特有的特征,使网络输出的语义表征更具有代表性。最后设计了三重损失函数:模态间损失,模态内损失和汉明空间损失对模型进行训练学习。实验结果表明,本文所提方法在MIRFLICKR-25K和NUS-WIDE数据集上均获得了良好的跨模态检索效果。      

Gesture image recognition method based on DC-Res2Net and a feature fusion attention module

To extract decisive features from gesture images and solve the problem of information redundancy in the existing gesture recognition methods, we propose a new multi-scale feature extraction module named densely connected Res2Net (DC-Res2Net) and design a feature fusion attention module (FFA). Firstly, based on the new dimension residual network (Res2Net), the DC-Res2Net uses channel grouping to extract fine-grained multi-scale features, and dense connection has been adopted to extract stronger features of different scales. Then, we apply a selective kernel network (SK-Net) to enhance the representation of effective features. Afterwards, the FFA has been designed to remove redundant information in features by fusing low-level location features with high-level semantic features. Finally, experiments have been conducted to validate our method on the OUHANDS, ASL, and NUS-II datasets. The results demonstrate the superiority of DC-Res2Net and FFA, which can extract more decisive features and remove redundant information while ensuring high recognition accuracy and low computational complexity.     

基于多尺度特征结合细节恢复的单幅图像去雾方法

为提高单幅图像去雾方法的准确性及其去雾结果的细节可见性,该文提出一种基于多尺度特征结合细节恢复的单幅图像去雾方法。首先,根据雾在图像中的分布特性及成像原理,设计多尺度特征提取模块及多尺度特征融合模块,从而有效提取有雾图像中与雾相关的多尺度特征并进行非线性加权融合。其次,构造基于所设计多尺度特征提取模块和多尺度特征融合模块的端到端去雾网络,并利用该网络获得初步去雾结果。再次,构造基于图像分块的细节恢复网络以提取细节信息。最后,将细节恢复网络提取出的细节信息与去雾网络得到的初步去雾结果融合得到最终清晰的去雾图像,实现对去雾后图像视觉效果的增强。实验结果表明,与已有代表性的图像去雾方法相比,所提方法能够对合成图像及真实图像中的雾进行有效去除,且去雾结果细节信息保留完整。     

Robust face anti-spoofing with depth information

With the prevalence of face authentication applications, the prevention of malicious attack from fake faces such as photos or videos, i.e., face anti-spoofing, has attracted much attention recently. However, while an increasing number of works on the face anti-spoofing have been reported based on 2D RGB cameras, most of them cannot handle various attacking methods. In this paper we propose a robust representation jointly modeling 2D textual information and depth information for face anti-spoofing. The textual feature is learned from 2D facial image regions using a convolutional neural network (CNN), and the depth representation is extracted from images captured by a Kinect. A face in front of the camera is classified as live if it is categorized as live using both cues. We collected a face anti-spoofing experimental dataset with depth information, and reported extensive experimental results to validate the robustness of the proposed method.     

基于新型多尺度注意力机制的密集人群计数算法

密集人群计数是计算机视觉领域的一个经典问题,仍然受制于尺度不均匀、噪声和遮挡等因素的影响.该文提出一种基于新型多尺度注意力机制的密集人群计数方法.深度网络包括主干网络、特征提取网络和特征融合网络.其中,特征提取网络包括特征支路和注意力支路,采用由并行卷积核函数组成的新型多尺度模块,能够更好地获取不同尺度下的人群特征,以...     

Multi-level fusion with deep neural networks for multimodal sentiment classification

The task of multimodal sentiment classification aims to associate multimodal information, such as images and texts with appropriate sentiment polarities. There are various levels that can affect human sentiment in visual and textual modalities. However, most existing methods treat various levels of features independently without having effective method for feature fusion. In this paper, we propose a multi-level fusion classification (MFC) model to predict the sentiment polarity based on the fusing features from different levels by exploiting the dependency among them. The proposed architecture leverages convolutional neural networks ( CNNs) with multiple layers to extract levels of features in image and text modalities. Considering the dependencies within the low-level and high-level features, a bi-directional (Bi) recurrent neural network (RNN) is adopted to integrate the learned features from different layers in CNNs. In addition, a conflict detection module is incorporated to address the conflict between modalities. Experiments on the Flickr dataset demonstrate that the MFC method achieves comparable performance compared with strong baseline methods.     

基于多尺度局部二元模式共生直方图的红外人脸识别

不同尺度的局部二元模式(LBP)提取了红外人脸图中不同的微结构局部特征。为了挖掘不同尺度中局部特征的相关性,提出了一种基于多尺度LBP 共生直方图的红外人脸识别方法。传统的多尺度LBP 特征提取方法,丢失了对多尺度特征间相关性信息的提取。为了充分考虑微结构间的相关统计信息,提出了多尺度LBP 共生直方图表示方法,以提取包含在红外人脸图像中的有用鉴别特征。多尺度LBP 共生直方图特征表示方法不仅可以消除环境温度对红外人脸图像特征提取的影响,而且还可以增强对局部特征表示的鉴别性。实验结果表明:多尺度局部二元模式共生矩阵可以增强对红外人脸鉴别特征提取的有效性,提出的红外人脸方法的性能优于基于传统多尺度LBP 和单尺度LBP方法,在相同环境情况下和在环境温度变化情况下可以达到99.2%和91.2%的识别率。     

Multi-scale aware dual path network for face detection in resource-constrained edge computing environment

Aiming at the problem that face detectors with complex deep neural structures are difficult to deploy in the resource-constrained edge computing environment,to reduce the resource consumption while maintain the accuracy in complex scenes such as multi-scale face changes,occlusion,blur,and illumination,SDPN(multi-scale aware dual path network) for face detection was proposed.The Face-ResNet (face residual neural network) was improved,and a dual path shallow feature extractor was used to understand the multi-scale information of the image through parallel branches.Then the deep and shallow feature fusion module,a combination of the underlying image information and the high-level semantic feature,was used in conjunction with the multi-scale awareness training strategy to supervise the multi-branch learning discriminating features.The experimental results show that SDPN can extract more diversified features,which effectively improve the accuracy and robustness of face detection while maintaining the efficiency of the model and low inference delay.     

红外与可见光图像多特征自适应融合方法

由于成像机理不同,红外图像以像素分布表征典型目标,而可见光图像以边缘和梯度描述纹理细节,现有的融合方法不能依据源图像特征自适应变化,造成融合结果不能同时保留红外目标特征与可见光纹理细节。为此,本文提出红外与可见光图像多特征自适应融合方法。首先,构建了多尺度密集连接网络,可以有效聚合所有不同尺度不同层级的中间特征,利于增强特征提取和特征重构能力。其次,设计了多特征自适应损失函数,采用VGG-16网络提取源图像的多尺度特征,以像素强度和梯度为测量准则,以特征保留度计算特征权重系数。多特征自适应损失函数监督网络训练,可以均衡提取源图像各自的特征信息,从而获得更优的融合效果。公开数据集的实验结果表明,该方法在主、客观评价方面均优于其他典型方法。     

基于LSTM网络和特征融合的通信干扰识别

针对现有通信干扰信号识别方法识别效果不佳的问题,提出了一种基于长短时记忆网络(Long Short-Term Memory,LSTM)和特征融合的通信干扰识别方法.该方法利用LSTM网络提取干扰信号的特征,通过LSTM强大的序列特征提取能力提升干扰信号特征提取的性能;通过提取信号的时域和频域特征后进行特征融合,使用全连...     

结合多尺度特征融合和多输入多输出编-解码器的去模糊算法

针对相机抖动、拍摄物体快速运动以及低快门速度等因素造成的图像非均匀模糊,提出一种结合多尺度特征融合和多输入多输出编-解码器的去模糊算法。首先使用多尺度特征提取模块来提取较小尺度模糊图像的初始特征,该模块使用扩张卷积来以较少的参数量获得更大的感受野。其次,通过特征注意力模块来自适应地学习不同尺度特征中的有效信息,该模块利用小尺度图像的特征来生成注意图,能够有效地减少冗余特征。最后,使用多尺度特征渐进融合模块逐步融合不同尺度的特征,使得不同尺度特征信息能够进行互补。相比以往的使用多个子网堆叠的多尺度方法,文中使用单个网络就能提取多尺度特征,从而降低了训练难度。为了评估网络的去模糊效果和泛化性能,提出的算法在基准数据集GoPro、HIDE和真实数据集RealBlur上均进行了测试。在GoPro和HIDE数据集上的峰值信噪比值分别为31.73 dB和29.39 dB,结构相似度值分别为0.951和0.923,其结果均高于目前先进的去模糊算法,并且在真实数据集RealBlur上也取得了最佳效果。实验结果表明,提出的去模糊算法相比现有算法去模糊更为彻底,能有效地复原图像的边缘轮廓和纹理细节信息,并且能够提升后续高级计算机视觉任务的鲁棒性。     

基于多尺度卷积特征融合的癫痫脑电信号识别

脑电信号(electroencephalography,EEG)已成为医生诊断神经系统疾病最 广泛使用的工具,实现癫痫EEG的自动识别对 于癫痫患者的临床诊断和治疗具有重要意义。为了提高癫痫EEG的识别精度,提出了一 种基于多尺 度卷积特征融合的癫痫EEG自动识别模型。首先采用多尺度卷积特征融合方法提取多粒 度数据特征, 实现卷积神经网络(convolutional neural network,CNN)中不同层次的信息互补;然后经过长短期记忆网络(long short-term memory network,LSTM)提取时间 特征,利用 softmax分类器给出最终的识别结果。为了评估提出方法的识别性能,在波恩大学癫痫病研 究中心数据集 中进行实验,并与CNN-LSTM模型、单一的LSTM等模型的识别性能进行了比较,实验结果表 明,提出 方法的识别精度明显高于其余方法, 平均可达到99.19%。该模型能够 有效识别癫痫EEG类别,具有较高的识别性能和临床应用潜力。     

基于解耦空间异常检测的人脸活体检测算法

现有基于异常检测的方法大多仅利用活体样本进行单类建模,这样的特征用于活体检测的泛化能力强但准确率不高。而且,活体人脸特征单类建模并没有考虑活体人脸样本的多样性。活体人脸样本的不同身份、环境、采集设备等因素都会导致活体人脸的特征表达不紧凑,这样使得假体样本特征容易混入其中。为了解决以上两个问题,本文提出了一种基于解耦空间异常检测的人脸活体检测算法。本文设计了单中心对比损失,使得活体人脸特征在不限制假体人脸特征分布的情况下表达地更加紧凑。本文还对活体人脸进行了特征解耦,将其特征分为两个子空间：活体检测特征空间、活体无关特征空间。活体检测特征空间不受其他无关因素的影响,结合单中心对比损失来提高模型的泛化能力。库内实验和跨库实验共在5个数据集上与最新的方法进行了比较,在OULU-NPU数据集中,协议1相比于性能第2的模型错误率下降超过一半,最具挑战的协议4取得了仅3.3%的错误率;在SiW数据集的三个协议中也取得更低的错误检测率;在跨库实验中本文算法也表现出不错的泛化能力,尤其是在从重放攻击和打印攻击跨到3D面具攻击的跨攻击类型的测试中相比于性能第2的模型错误率下降5.41%。本文提出的人脸活...     

Single image deraining using multi-scales context information and attention network

The existing deraining methods based on convolutional neural networks (CNNs) have made great success, but some remaining rain streaks can degrade images drastically. In this work, we proposed an end-to-end multi-scale context information and attention network, called MSCIANet. The proposed network consists of multi-scale feature extraction (MSFE) and multi-receptive fields feature extraction (MRFFE). Firstly, the MSFE can pick up features of rain streaks in different scales and propagate deep features of the two layers across stages by skip connections. Secondly, the MRFFE can refine details of the background by attention mechanism and the depthwise separable convolution of different receptive fields with different scales. Finally, the fusion of these outputs of two subnetworks can reconstruct the clean background image. Extensive experimental results have shown that the proposed network achieves a good effect on the deraining task on synthetic and real-world datasets. The demo can be available at https://github.com/CoderLi365/MSCIANet.     

Smoky vehicle detection based on multi-scale block Tamura features

Smoky vehicle, emitting visible black exhaust emissions from vehicle exhaust pipe, is representative heavy pollution vehicle. This paper presents an intelligent smoky vehicle detection method based on multi-scale block Tamura features. In this method, the Vibe background subtraction algorithm is adopted to detect vehicle objects. We propose the multi-scale block Tamura features and use this features to distinguish smoky vehicle images and non-smoke vehicle images. More specifically, the region at the back of the vehicle is divided into 1\(\times \)2 blocks. For each block, the multi-scale strategy based on Gaussian kernel with different standard deviations is proposed to extract features and utilize different scales information. Finally, the back-propagation neural network classifier is trained and used for classification. Our method can automatically detect smoky vehicle through analyzing road surveillance videos. The experimental results show that the proposed algorithm framework performs better than common smoke and fire detection method, and the proposed multi-scale block Tamura features can obtain higher detection accuracy than common Tamura features.     

卷积神经网络结合NSST的红外与可见光图像融合

传统的多尺度红外与可见光图像融合方法,所提取的图像特征固定,并不能很好的应用于各类复杂的图像环境,而深度学习可以自主选择合适图像特征,改良特征提取单一性问题,因此提出一种基于卷积神经网络与非下采样剪切波变换（NSST）相结合的红外与可见光图像融合方法。首先,用卷积神经网络提取红外目标与背景的二分类图,利用调频（FT）显著性检测算法对分类图进行精准分割,同时,利用NSST将源图像多尺度、多方向进行分解;其次,利用目标显著性结合自适应模糊逻辑算法进行低频子带融合,利用高频系数局部方差对比度方法进行高频子带融合;最后,通过NSST逆变换得到融合后图像。实验结果表明:相比于传统图像融合算法,该方法在信息熵、平均梯度、空间频率、互信息和交叉熵等多个客观评价指标上至少分别提高了0.01%、0.30%、1.43%、2.32%、1.14%。一定程度提高了融合图像对比度,丰富了背景细节信息,更有利于人眼识别,可以广泛的应用于光电侦察、光电告警、多传感器信息融合等光电信息领域。     

Presentation attack detection based on two-stream vision transformers with self-attention fusion

Aiming at the performance degradation of the existing presentation attack detection methods due to the illumination variation, a two-stream vision transformers framework (TSViT) based on transfer learning in two complementary spaces is proposed in this paper. The face images of RGB color space and multi-scale retinex with color restoration (MSRCR) space are fed to TSViT to learn the distinguishing features of presentation attack detection. To effectively fuse features from two sources (RGB color space images and MSRCR images), a feature fusion method based on self-attention is built, which can effectively capture the complementarity of two features. Experiments and analysis on Oulu-NPU, CASIA-MFSD, and Replay-Attack databases show that it outperforms most existing methods in intra-database testing and achieves good generalization performance in cross-database testing.     

复杂环境下多尺度行人实时检测方法

作为计算机视觉和图像处理研究领域中的经典课题,行人检测技术在智能驾驶、视频监控等领域中具有广泛的应用空间。然而,面对一些复杂的环境和情况,如阴雨、雾霾、被遮挡、照明度变化、目标尺度差异大等,常见的基于可见光或红外图像的行人检测方法的效果尚不尽如人意,无论是在检测准确率还是检测速度上。该文分析并抓住可见光和红外检测系统中行人特征差异较大,但在不同环境中又各有优势的特点,并结合多尺度特征提取方法,提出一种适用于多样复杂环境下多尺度行人实时检测的方法——融合行人检测网络(FPDNet)。该网络主要由特征提取骨干网络、多尺度检测和信息决策融合3个部分构成,可自适应提取可见光或红外背景下的多尺度行人。实验结果证明,该检测网络在多种复杂视觉环境下都具有较好的适应能力,在检测准确性和检测速度上均能满足实际应用的需求。