多模态情感分析算法综述

多模态情感分析作为近年来的研究热点,比单模态情感分析具有更高的稳定性和准确率.介绍了多模态情感分析研究背景,分别对基于文本的、基于音频的和基于视频的单模态情感算法分析进行了阐述,又讲解了多模态的3种融合方法:特征级融合、决策级融合和混合融合以及相关算法,并分析了多模态情感分析存在的问题.     

基于多模态生物识别的安全身份认证技术

本文分析了传统身份认证方法在电子取证中存在的问题,突出了生物特征识别技术应用于电子取证中的优点。针对单模态生物特征技术存在识别精度不够等问题,提出了将多模态生物特征技术的安全身份认证的体系结构,并探讨了多种特征信息融合的方法。[编者按]     

多模态信息处理前沿综述:应用、融合和预训练

随着视觉、听觉、语言等单模态人工智能技术的突破,让计算机拥有更接近人类理解多模态信息的能力受到研究者们的广泛关注。另一方面,随着图文社交、短视频、视频会议、直播和虚拟数字人等应用的涌现,对多模态信息处理技术提出了更高要求,同时也给多模态研究提供了海量的数据和丰富的应用场景。该文首先介绍了近期自然语言处理领域关注度较高的多模态应用,并从单模态的特征表示、多模态的特征融合阶段、融合模型的网络结构、未对齐模态和模态缺失下的多模态融合等角度综述了主流的多模态融合方法,同时也综合分析了视觉-语言跨模态预训练模型的最新进展。     

复合层次融合的多模态情感分析

针对传统情感分析方法无法解决短视频情感表达问题以及现有多模态情感分析方法准确率不高、不同模态信息之间交互性差等问题,通过对多模态情感分析方法进行研究,结合时域卷积网络（TCN）和软注意力机制建立了复合层次融合的多模态情感分析模型。该模型首先将视频中提取到的文本特征、视频面部特征和音频特征进行维度均衡,然后将得到的不同模态的信息特征进行复合式融合,即先将单模态信息进行融合得到双模态特征信息,再将得到的三个双模态信息进行融合,得到最终的三模态信息,最后将得到的三模态信息和每个单模态信息进行融合得到最终的多模态情感特征信息。每次融合的特征信息都经过TCN网络层进行序列特征的提取,将最终得到的多模态特征信息通过注意力机制进行筛选过滤后用于情感分类,从而得到预测结果。在数据集CMU-MOSI和CMU-MOSEI上的实验表明,该模型能够充分利用不同模态间的交互信息,有效提升多模态情感分析的准确率。     

基于三角范数的多模态手指生物特征的分数层融合方法

多模态的生物特征融合已成为目前生物识别研究领域的主要发展趋势,从防伪性角度出发,满足普适性和易采集、易接受性的应用需求,提出了基于手指静脉,并结合指纹、指关节纹和指形的分数层融合来实现多模态生物特征的身份认证方案。实验结果表明,本文提出基于Sugeno-Weber三角范数的分数层融合方法,能够增大类内与类间匹配分数分布间的距离,提高了多模态生物特征的身份认证性能。     

基于深度学习的多模态生物特征融合模型

面对公安实战中获取的低质量生物特征数据,单模态生物特征识别技术的精度并不理想,现有的多模态融合算法存在融合层次单一、泛化性不强等问题,深度神经网络的发展为其提供了有效的解决途径。构建基于深度神经网络的多模态生物特征融合模型,将像素层、特征层、分数层等不同层次的融合方法统一到融合模型中,在像素层采用空间、通道和强度融合三种策略;在特征层通过反向传播整体优化模态专用分支与联合表示层,构建模态之间一阶依赖关系;在分数层使用基于Rank1评价和基于模态评价两种方法完成匹配分数融合。模拟实战数据构建虚拟同源多模态数据集进行模型验证。实验结果表明,多模态像素层融合方法提升效果有限,难以增强数据的区分度;多模态特征层融合方法相比单模态算法提升2.2个百分点;分数层融合方法相比单模态算法提升3.5个百分点,最佳检索精度可达99.6%。基于深度学习方法提出的多模态生物特征融合模型极大地提高了模型的泛化性和检索精度。     

融合动作特征的多模态情绪识别

近年来,利用计算机技术实现基于多模态数据的情绪识别成为自然人机交互和人工智能领域重要 的研究方向之一。利用视觉模态信息的情绪识别工作通常都将重点放在脸部特征上,很少考虑动作特征以及融合 动作特征的多模态特征。虽然动作与情绪之间有着紧密的联系,但是从视觉模态中提取有效的动作信息用于情绪 识别的难度较大。以动作与情绪的关系作为出发点,在经典的 MELD 多模态情绪识别数据集中引入视觉模态的 动作数据,采用 ST-GCN 网络模型提取肢体动作特征,并利用该特征实现基于 LSTM 网络模型的单模态情绪识别。 进一步在 MELD 数据集文本特征和音频特征的基础上引入肢体动作特征,提升了基于 LSTM 网络融合模型的多 模态情绪识别准确率,并且结合文本特征和肢体动作特征提升了上下文记忆模型的文本单模态情绪识别准确率, 实验显示虽然肢体动作特征用于单模态情绪识别的准确度无法超越传统的文本特征和音频特征,但是该特征对于 多模态情绪识别具有重要作用。基于单模态和多模态特征的情绪识别实验验证了人体动作中含有情绪信息,利用 肢体动作特征实现多模态情绪识别具有重要的发展潜力。     

多模态生物特征提取及相关性评价综述

生物特征识别是身份认证的重要手段,特征提取技术在其中扮演了关键角色,直接影响识别的结果。随着特征提取技术日趋成熟,学者们逐渐将目光投向了生物特征间的相关性问题。本文以单模态和多模态生物识别中的特征提取方法为研究对象,回顾了人脸与指纹的特征提取方法,分析了基于经验知识的特征分类提取方法以及基于深度学习的计算机逻辑采样提取方法,并从图像处理的角度对单模态与多模态方法进行对比。以当前多模态生物特征提取方法和DNA表达过程为引,提出了不同模态的生物特征之间存在相关性的猜想,以及对这一猜想进行建模的思路。在多模态生物特征提取的基础上,对今后可能有进展的各生物特征之间的相关性建模进行了展望。     

金融交易密码多模态化方法

针对金融交易系统目前单一密码键盘模态客户身份认证存在的安全性、可靠性问题,提出了密码多模态化方法。多模态感知器获取密码码字信息,格式归一化传送所获原始模态信息,按属性前置处理、分类当前模态码字信息,公共单元融合多模态密码。对于确认的M位密码,每位分别采用N种模态输入,密码窃取率为1/(10MCN×MM)。以密码键盘与黑箱子指书为缺省模态的多模态密码输入系统的实用效果表明,所提方法实现了多模态密码码字无序交融输入,M=6时系统的密码窃取率为1/(106C2×66),而密码破解技术门类数及难度则随着模态数量增加而增加。密码输入的安全性及可靠性等多方面性能较现有单模态身份认证工作状态得到大幅提升。     

基于注意力机制的人脸虹膜双特征融合识别

针对单一生物识别方法存在的固有局限性，利用人脸和虹膜双生物模态信息，提出一种基于注意力机制和低秩多模态融合的身份识别模型(attention mechanism and low-rank multimodal fusion, ALMF)。在模型的人脸和虹膜特征提取网络中均嵌入改进的混合注意力机制(I＿CBAM),增强有用特征的提取。利用模态特定低秩因子完成低秩多模态特征级融合(low-rank multimodal fusion, LMF),解决传统特征拼接方式无法充分实现各模态特征的互补、容易造成冗余信息和维度灾难等问题。使用简单高效的余弦距离完成特征模板的比对实现身份识别。实验结果表明，ALMF模型相比单一生物特征识别和传统融合识别算法具有更强的鲁棒性和准确率。     

Secure biometric template generation for multi-factor authentication

In the light of recent security incidents, leading to compromise of services using single factor authentication mechanisms, industry and academia researchers are actively investigating novel multi-factor authentication schemes. Moreover, exposure of unprotected authentication data is a high risk threat for organizations with online presence. The challenge is how to ensure security of multi-factor authentication data without deteriorating the performance of an identity verification system? To solve this problem, we present a novel framework that applies random projections to biometric data (inherence factor), using secure keys derived from passwords (knowledge factor), to generate inherently secure, efficient and revocable/renewable biometric templates for users? verification. We evaluate the security strength of the framework against possible attacks by adversaries. We also undertake a case study of deploying the proposed framework in a two-factor authentication setup that uses users? passwords and dynamic handwritten signatures. Our system preserves the important biometric information even when the user specific password is compromised – a highly desirable feature but not existent in the state-of-the-art transformation techniques. We have evaluated the performance of the framework on three publicly available signature datasets. The results prove that the proposed framework does not undermine the discriminating features of genuine and forged signatures and the verification performance is comparable to that of the state-of-the-art benchmark results.     

An improved timestamp-based remote user authentication scheme

To protect the remote server from various malicious attacks, many authentication schemes have been proposed. Some schemes have to maintain a password verification table in the remote server for checking the legitimacy of the login users. To overcome potential risks of verification tables, researchers proposed remote user authentication schemes using smartcard, in which the remote server only keeps a secret key for computing the user’s passwords and does not need any verification table for verifying legal user. In 2003 Shen, Lin, and Hwang proposed a timestamp-based password authentication scheme using smartcards in which the remote server does not need to store the passwords or verification table for user authentication. Unfortunately, this scheme is vulnerable to some deadly attacks. In this paper, we analyze few attacks and finally propose an improved timestamp-based remote user authentication scheme. The modified scheme is more efficient and secure than original scheme.     

Keystroke dynamics‐based user authentication service for cloud computing

User authentication is a crucial requirement for cloud service providers to prove that the outsourced data and services are safe from imposters. Keystroke dynamics is a promising behavioral biometrics for strengthening user authentication, however, current keystroke based solutions designed for certain datasets, for example, a fixed length text typed on a traditional personal computer keyboard and their authentication performances were not acceptable for other input devices nor free length text. Moreover, they suffer from a high dimensional feature space that degrades the authentication accuracy and performance. In this paper, a keystroke dynamics based authentication system is proposed for cloud environments that is applicable to fixed and free text typed on traditional and touch screen keyboards. The proposed system utilizes different feature extraction methods, as a preprocessing step, to minimize the feature space dimensionality. Moreover, different fusion rules are evaluated to combine the different feature extraction methods so that a set of the most relevant features is chosen. Because of the huge number of users' samples, a clustering method is applied to the users' profile templates to reduce the verification time. The proposed system is applied to three different benchmark datasets using three different classifiers. Experimental results demonstrate the effectiveness and efficiency of the proposed system. Copyright © 2015 John Wiley & Sons, Ltd.     

基于身份的远程用户认证方案

研究近期提出的2个远程用户认证方案,对其进行伪造攻击。利用基于身份的签名思想提出一个基于身份的远程用户认证方案,在实现动态认证的同时无须用户与远程服务器端交互,通信量小,远端服务器无须保存或维护任何口令或验证表,存储代价低,可以避免口令攻击、重放攻击、伪造攻击、中间人攻击等,安全性高。     

基于SVM算法的用户行为认证方法

为提高手机安全性,提出一种基于SVM的用户操作行为认证方法.通过监听手机触摸屏设备,持续获取用户操作时的滑动轨迹、接触面积等原始数据.设计用户行为特征提取算法以建立用户特征样本,经SVM算法加以训练形成用户行为特征模型;综合用户访问目标及历史认证结果采用不同认证策略,达到重点保护敏感数据,方便用户访问非敏感数据的效果.在Android系统环境下的实验验证表明,该方法具有良好的认证效果.     

A graphical-based password keystroke dynamic authentication system for touch screen handheld mobile devices

Since touch screen handheld mobile devices have become widely used, people are able to access various data and information anywhere and anytime. Most user authentication methods for these mobile devices use PIN-based (Personal Identification Number) authentication, since they do not employ a standard QWERTY keyboard for conveniently entering text-based passwords. However, PINs provide a small password space size, which is vulnerable to attacks. Many studies have employed the KDA (Keystroke Dynamic-based Authentication) system, which is based on keystroke time features to enhance the security of PIN-based authentication. Unfortunately, unlike the text-based password KDA systems in QWERTY keyboards, different keypad sizes or layouts of mobile devices affect the PIN-based KDA system utility. This paper proposes a new graphical-based password KDA system for touch screen handheld mobile devices. The graphical password enlarges the password space size and promotes the KDA utility in touch screen handheld mobile devices. In addition, this paper explores a pressure feature, which is easy to use in touch screen handheld mobile devices, and applies it in the proposed system. The experiment results show: (1) EER is 12.2% in the graphical-based password KDA proposed system. Compared with related schemes in mobile devices, this effectively promotes KDA system utility; (2) EER is reduced to 6.9% when the pressure feature is used in the proposed system. The accuracy of authenticating keystroke time and pressure features is not affected by inconsistent keypads since the graphical passwords are entered via an identical size (50 mm × 60 mm) human–computer interface for satisfying the lowest touch screen size and a GUI of this size is displayed on all mobile devices.     

Continuous Authentications Using Frequent English Terms

Most of the current computer systems authenticate a user’s identity only at the point of entry to the system (i.e., login). However, an effective authentication system includes continuous or frequent monitoring of the identity of a user already logged into a system to ensure the validity of the identity of the user throughout a session. Such a system is called a “continuous or active authentication system.” An authentication system equipped with such a security mechanism protects the system against certain attacks including session hijacking that can be performed later by a malicious user. The aim of this research is to advance the state-of-the-art of the user-active authentication research using keystroke dynamics. Through this research, we assess the performance and influence of various keystroke features on keystroke dynamics authentication systems. In particular, we investigate the performance of keystroke features on a subset of most frequently used English words. The performance of four features including key duration, flight time latency, diagraph time latency, and word total time duration are analyzed. A series of experiments is performed to measure the performance of each feature individually as well as the results from the combinations of these features. More specifically, four machine learning techniques are adapted for the purpose of assessing keystroke authentication schemes. The selected classification methods are Support Vector Machine (SVM), Linear Discriminate Classifier (LDC), K-Nearest Neighbors (K-NN), and Naive Bayesian (NB). Moreover, this research proposes a novel approach based on sequential change-point methods for early detection of an imposter in computer authentication without the needs for any modeling of users in advance, that is, no need for a-priori information regarding changes. The proposed approach based on sequential change-point methods provides the ability to detect the impostor in early stages of attacks. The study is performed and evaluated based on data collected for 28 users. The experimental results indicate that the word total time feature offers the best performance result among all four keystroke features, followed by diagraph time latency. Furthermore, the results of the experiments also show that the combination of features enhances the performance accuracy. In addition, the nearest neighbor method performs the best among the four machine learning techniques.     

5G异构网络中基于群组的切换认证方案

随着5G网络的发展,各类网络服务质量极大提升的同时网络环境也愈加复杂,从而带来了一系列安全挑战。切换认证可以解决用户在不同类型网络间的接入认证问题,但现存方案仍存在一些不足,还需要解决如全局切换认证、密钥协商、隐私保护、抵抗伪装攻击、抵抗中间人攻击、抵抗重放攻击以及群组用户切换效率等问题。针对这些问题,提出了一个5G异构网络中基于群组的切换认证方案。在所提出的方案中,注册域服务器在区块链上为每个用户存入一个通行证,任何实体都可以利用该通行证对用户进行认证,从而实现全局切换认证。对于群组用户,各用户分别设置可聚合的认证参数,验证者通过验证聚合签名实现对群组用户的批量验证。新方案不仅提升了群组用户切换时的效率,同时还满足上述安全性要求。基于形式化分析软件AVISPA的分析结果表明,所提出的方案是安全的。性能分析表明,所提出的方案执行批量验证时的效率比现存方案至少提升了89.8%。     

Cryptanalysis and improvement of ‘a secure authentication scheme for telecare medical information system’ with nonce verification

In 2009, Xu et al. presented an improved smartcard based authentication scheme while using a security model previously applied by Bellare et al. to prove the security of their authentication methods. Later on, in 2012, Wu et al. pointed out number of authentication attacks in Xu et al. scheme. To address these issues, Wu et al. presented a Smartcard based Two-Factor Authentication (2FA) scheme for Telecare Medical Information System (TMIS) facility. In this study, we prove that authentication scheme of Wu et al. is still vulnerable to impersonation attack, offline password guessing attack, forgery attack and many other attacks. Moreover, number of performance and verification issues are also outlined in the authentication scheme of Wu et al. To overcome these issues, an improved and enhanced 3FA Smartphone based authentication method is proposed on a Cloud Computing environment. The proposed scheme is further corroborated using Burrows-Abadi-Needham logic (BAN logic) nonce verification. The detailed BAN logic verification and further security analysis shows that the proposed authentication protocol is highly reliable and secure in terms of message verifications, message freshness and trustworthiness of its origin. Moreover, the comparative security, performance and feature analysis shows that the proposed work yields an even more improved and enhanced authentication framework as compared to Wu et al. authentication scheme.     

基于手机触摸屏传感器多点触摸身份认证算法

针对智能手机所面临的信息安全威胁问题,提出一种基于手机触摸屏传感器的多点触摸身份认证方法。首先由触摸屏传感器采集手指滑动原始数据序列,通过平滑去噪、位置及长度归一化预处理;然后提取手势运动一阶、二阶归一化导数序列及运动方向为身份验证特征序列;最后采用模板匹配方法,使用动态时间规整算法匹配比较注册模板特征序列与测试特征序列,判断用户身份真实性。仿真结果表明,所提算法对不同用户身份认证的平均错误拒绝率和错误接受率分别为3.83%和2.07%,与使用径向基函数为核函数的支持向量分布估计(SVDE)算法相比,平均错误拒绝率和错误接受率分别降低1.81%和2.35%。经性能分析,所提算法能明显提高身份认证的准确性。