基于深度学习的文本分类技术研究进展

随着深度学习技术的快速发展,许多研究者尝试利用深度学习来解决文本分类问题,特别是在卷积神经网络和循环神经网络方面,出现了许多新颖且有效的分类方法。对基于深度神经网络的文本分类问题进行分析,介绍卷积神经网络、循环神经网络、注意力机制等方法在文本分类中的应用和发展,分析多种典型分类方法的特点和性能,从准确率和运行时间方面对基础网络结构进行比较,表明深度神经网络较传统机器学习方法在用于文本分类时更具优势,其中卷积神经网络具有优秀的分类性能和泛化能力。在此基础上,指出当前深度文本分类模型存在的不足,并对未来的研究方向进行展望。     

基于相空间重构理论与递归神经网络相结合的股票短期预测方法*

根据股票指数时间序列复杂的非线性特性,提出以相空间重构理论与递归神经网络相结合的股票短期预测新方法.以相空间重构理论确定最佳延迟时间和最小嵌入维数,以最佳延迟时间为间隔的最小嵌入维数作为递归神经网络的输入维数,并按预测相点步进递归的生成训练数据进行短期预测,提高了预测精度和稳定性.该方法应用于沪市股票综合指数预测,其结果与传统的单纯用BP网络模型预测的结果相比较,精度大大提高,证明了该预测模型和方法在实际时间序列预测领域的有效性和实用性.     

基于EMD和RBFNN的冷负荷组合预测模型

针对冷负荷预测问题,提出了一种基于相空间重构（PSR）、经验模态分解(EMD) 和径向基神经网络(RBFNN) 的 冷负荷组合预测模型。该模型首先利用经验模态分解方法,把冷负荷序列分解为少数模态分量,然后利用分组分量法将其分 为多个高频子分量、总低频分量和残余量,最后以PSR为基础对各分量利用RBFNN方法建模并将预测结果重构。该方法应 用于实际冷站负荷预测后,与单一RBFNN、SVM、LSSVM及基于EMD的SVM、基于EMD的RBFNN5类方法进行比较,结果 表明该方法对冷负荷预测精度有明显提高。     

基于DNN处理的鲁棒性I-Vector说话人识别算法

提出了一种将基于深度神经网络（Deep Neural Network,DNN）特征映射的回归分析模型应用到身份认证矢量（identity vector,i-vector）/概率线性判别分析（Probabilistic Linear Discriminant Analysis,PLDA）说话人系统模型中的方法。DNN通过拟合含噪语音和纯净语音i-vector之间的非线性函数关系,得到纯净语音i-vector的近似表征,达到降低噪声对系统性能影响的目的。在TIMIT数据集上的实验验证了该方法的可行性和有效性。     

推荐系统综述

随着网络应用的不断发展,网络资源呈指数型增长,信息过载现象日益严重,如何高效获取符合需求的资源成为困扰人们的问题之一。推荐系统能对海量信息进行有效过滤,为用户推荐符合其需求的资源。对推荐系统的研究现状进行详细介绍,包括基于内容的推荐、协同过滤推荐和混合推荐这三种传统推荐方式,并重点分析了基于卷积神经网络（CNN）、深度神经网络（DNN）、循环神经网络（RNN）和图神经网络（GNN）这四种常见的深度学习推荐模型的研究进展;归纳整理了推荐领域常用的数据集,同时分析对比了传统推荐算法和基于深度学习的推荐算法的差异。最后,总结了实际应用中具有代表性的推荐模型,讨论了推荐系统面临的挑战和未来的研究方向。     

径向函数网络对国民经济生产总值预测研究

为了建立国民经济生产总值（GDP）神经网络预测模型,构造了双层网络结构的基于径向基网络,通过学习训练,确定径向基数神经网络参数和结构。仿真结果表明,生成的径向基函数模型应用于国民经济预测比BP神经网络模型具有更高的预测精度和良好的泛化能力。     

Applying Wide & Deep Learning Model for Android Malware Classification

Android malware has exploded in popularity in recent years, due to the platform’s dominance of the mobile market. With the advancement of deep learning technology, numerous deep learning-based works have been proposed for the classification of Android malware. Deep learning technology is designed to handle a large amount of raw and continuous data, such as image content data. However, it is incompatible with discrete features, i.e., features gathered from multiple sources. Furthermore, if the feature set is already well-extracted and sparsely distributed, this technology is less effective than traditional machine learning. On the other hand, a wide learning model can expand the feature set to enhance the classification accuracy. To maximize the benefits of both methods, this study proposes combining the components of deep learning based on multi-branch CNNs (Convolutional Network Neural) with wide learning method. The feature set is evaluated and dynamically partitioned according to its meaning and generalizability to subsets when used as input to the model’s wide or deep component. The proposed model, partition, and feature set quality are all evaluated using the K-fold cross validation method on a composite dataset with three types of features: API, permission, and raw image. The accuracy with Wide and Deep CNN (WDCNN) model is 98.64%, improved by 1.38% compared to RNN (Recurrent Neural Network) model.     

Taylor-AMS features and deep convolutional neural network for converting nonaudible murmur to normal speech

Communication becomes effective when the speech signal arrives with the profound characteristics. This insisted the researchers to develop an automatic system of recognizing the speech signals from the murmurs. Some of the traditional automatic recognition systems are unfit for the silent environments imposing a need for an effective recognition system. Also, the traditional automatic recognition methods, like Neural Networks, render poor performance in the presence of the murmurs. Thus, this article proposes a method for automatic whisper recognition using the Deep Convolutional Neural Network (DCNN). The training of the DCNN is performed using the proposed Stochastic-Whale Optimization Algorithm (Stochastic-WOA), which is designed by the integration of Stochastic Gradient Descent algorithm with WOA. The input to the classifier is the features that include pitch chroma, spectral centroid, spectral skewness, and Taylor-Amplitude Modulation Spectrogram (Taylor-AMS), which is obtained by combining Taylor series and Amplitude Modulation Spectrogram (AMS) features, of the preprocessed input speech signal. The experimentation of the method is performed using the real database and the analysis proves that the proposed method acquired a maximal accuracy of 0.9723, minimal False Positive Rate of 0.0257, and maximal True Positive Rate of 0.9981, respectively.     

基于SGMM和DNN结合提高音素识别率的研究

为降低声学特征在语音识别系统中的音素识别错误率,提高系统性能,提出一种子空间高斯混合模型和深度神经网络结合提取特征的方法,分析了子空间高斯混合模型的参数规模并在减少计算复杂度后将其与深度神经网络串联进一步提高音素识别率。把经过非线性特征变换的语音数据输入模型,找到深度神经网络结构的最佳配置,建立学习与训练更可靠的网络模型进行特征提取,通过比较音素识别错误率来判断系统性能。实验仿真结果证明,基于该系统提取的特征明显优于传统声学模型。     

基于SMOTE和RNN的肾移植排斥反应预测

肾移植手术在当今的应用越来越广泛,对于排斥反应的预测变得更加重要。针对排斥反应数据特点中存在的数据的维度高、数据时序性、样本不均衡等问题,将循环神经网络应用于肾移植排斥反应的预测,本文提出一种结合SMOTE(Synthetic Minority Over-sampling Technique)以及RNN(Recurrent Neural Network)的算法。该方法先处理数据,降低正负样本的不平衡度,且解决样本量不足的问题,再根据RNN的学习过程进行关键参数调整、优化。经过实验发现,该方法可以有效提升正负分类的准确率,与传统的马尔可夫时间序列预测算法相比,准确率提高了16.7%,传统RNN训练经过优化后,相对错误率下降了5.03%,可以使用该方法进行肾移植排斥反应的有效预测。     

基于DNN与基音周期的说话人识别

传统说话人识别框架大多建立在高斯混合模型（GMM）上的,然而这种浅层学习模型不能有效地表征数据特征之间的高阶相关性,识别效果较差。本文提出一种基于深度神经网络（Deep Neural Network, DNN）与基音周期（Pitch Period, PP）相结合的说话人识别方法,模型主线识别以对数梅尔滤波器组特征参数作为DNN的输入,通过训练DNN模型提取说话人的声纹特征;针对DNN模型阈值设定人的主观性影响,利用动态时间规整技术匹配说话人基音周期进行辅助识别。实验结果表明,这种双重识别方法等错误率可以达到1.6%,较DNN系统与EM-GMM系统等错误率分别降低了1.2%和2.4%,并且在噪声环境中仍具有较好的鲁棒性。     

多噪声环境下的层级语音识别模型

针对多噪声环境下的语音识别问题,提出了将环境噪声作为语音识别上下文考虑的层级语音识别模型。该模型由含噪语音分类模型和特定噪声环境下的声学模型两层组成,通过含噪语音分类模型降低训练数据与测试数据的差异,消除了特征空间研究对噪声稳定性的限制,并且克服了传统多类型训练在某些噪声环境下识别准确率低的弊端,又通过深度神经网络（DNN）进行声学模型建模,进一步增强声学模型分辨噪声的能力,从而提高模型空间语音识别的噪声鲁棒性。实验中将所提模型与多类型训练得到的基准模型进行对比,结果显示所提层级语音识别模型较该基准模型的词错率（WER）相对降低了20.3%,表明该层级语音识别模型有利于增强语音识别的噪声鲁棒性。     

基于Fisher-PCA和深度学习的入侵检测方法研究

为了在攻击形式多样化、入侵数据海量及多维化的环境中快速、准确地识别网络攻击,提出了一种融合Fisher-PCA特征提取与深度学习的入侵检测算法。通过Fisher特征选择算法选出重要的特征组成特征子集,然后基于主成分分析法（Principal Component Analysis,PCA）将特征子集进行降维,提取出了分类能力强的特征集。构建了一种新的DNN（Deep Neural Networks）深度神经网络模型对网络攻击数据和正常数据进行识别与分类。在KDD99数据集上进行试验,结果表明这种入侵检测算法与传统的ANN、SVM算法相比,在准确率上分别提高了12.63%、6.77%,在误报率上由原来的2.31%、1.96%降为0.28%,与DBN4 、PCA-CNN算法相比,在准确率和检测率保持基本相同的同时有着更低的误报率。     

基于多尺度跳跃深度长短期记忆网络的短期多变量负荷预测

近年来,以循环神经网络（RNN）为主体构建的预测模型在短期电力负荷预测中取得了优越的性能。然而,由于RNN不能有效捕捉存在于短期电力负荷数据的多尺度时序特征,因而难以进一步提升负荷预测精度。为了捕获短期电力负荷数据中的多尺度时序特征,提出了一种基于多尺度跳跃深度长短期记忆（MSD-LSTM）网络的短期电力负荷预测模型。具体来说,以长短期记忆（LSTM）网络为主体构建预测模型能够较好地捕获长短期时序依赖,从而缓解时序过长时重要信息容易丢失的问题。进一步地,采用多层LSTM架构并且对各层设置不同的跳跃连接数,使得MSD-LSTM的每一层能够捕获不同时间尺度的特征。最后,引入全连接层把各层提取到的多尺度时序特征进行融合,再利用该融合特征进行短期电力负荷预测。实验结果表明,与单层LSTM和多层LSTM相比,MSD-LSTM的均方误差总体下降了10%。可见MSD-LSTM能够更好地提取短期负荷数据中的多尺度时序特征,从而提高短期电力负荷预测的精度。     

Generative adversarial network (GAN) and enhanced root mean square error (ERMSE): deep learning for stock price movement prediction

The prediction of stock price movement direction is significant in financial circles and academic. Stock price contains complex, incomplete, and fuzzy information which makes it an extremely difficult task to predict its development trend. Predicting and analysing financial data is a nonlinear, time-dependent problem. With rapid development in machine learning and deep learning, this task can be performed more effectively by a purposely designed network. This paper aims to improve prediction accuracy and minimizing forecasting error loss through deep learning architecture by using Generative Adversarial Networks. It was proposed a generic model consisting of Phase-space Reconstruction (PSR) method for reconstructing price series and Generative Adversarial Network (GAN) which is a combination of two neural networks which are Long Short-Term Memory (LSTM) as Generative model and Convolutional Neural Network (CNN) as Discriminative model for adversarial training to forecast the stock market. LSTM will generate new instances based on historical basic indicators information and then CNN will estimate whether the data is predicted by LSTM or is real. It was found that the Generative Adversarial Network (GAN) has performed well on the enhanced root mean square error to LSTM, as it was 4.35% more accurate in predicting the direction and reduced processing time and RMSE by 78 s and 0.029, respectively. This study provides a better result in the accuracy of the stock index. It seems that the proposed system concentrates on minimizing the root mean square error and processing time and improving the direction prediction accuracy, and provides a better result in the accuracy of the stock index.

     

语音对抗攻击与防御方法综述

人工智能的不断发展,使得人与机器的交互变得至关重要.语音是人与智能通讯设备之间通信的重要手段,在近几年飞速发展,说话人识别、情感识别、语音识别得到广泛地普及与应用.特别的,随着深度学习的兴起,基于深度学习的语音技术使机器理解语音内容、识别说话人方面达到近似人的水平,无论是效率还是准确度都得到了前所未有的提升.例如手机语...     

基于深度学习的监督型典型相关分析

典型相关分析（CCA）是利用综合变量对之间的相关关系反映两组指标之间整体相关性的多元统计方法。传统的CCA方法无法有效利用样本的标签信息,导致准确率降低。将类信息融入到深度学习与CCA相结合的深度典型相关分析中,提出一种监督型降维方法DL-SCCA,用于处理带标签的非线性可分数据。在2个独立的深度神经网络（DNN）结构上,增加1个公共的输出维数与数据集类别数相同的全连接层,并且以softmax函数作为该层的激活函数,输出带有概率意义的编码向量。在此基础上,利用全连接输出与样本标签信息之间的交叉熵对DNN进行训练,获得分类性能较优的低维特征。实验结果表明,该方法采用最近邻分类器和网络本身结构得到的分类准确率分别为98.00%和97.82%,相比CCA、DisCCA、DCCA等方法,能够有效利用样本的标签信息,并且具有较优的分类性能。     

基于长短时记忆网络的人体姿态检测方法

针对在循环神经网络（RNN）网络结构下较为遥远的历史信号无法传递至当前时刻的问题,长短时记忆（LSTM）网络作为RNN的一种变体被提出,在继承RNN对时间序列优秀的记忆能力的前提下,LSTM克服了这种时间序列的长期依赖问题,并在自然语言处理与语音识别领域有较好的表现。对于人体行为动作中也存在作为时间序列的长期依赖问题与使用传统滑窗算法采集数据时造成的无法实时检测的问题,将LSTM扩展应用到人体姿态检测,提出了基于LSTM的人体姿态检测方法。通过目前智能手机中一般都带有的加速度传感器、陀螺仪、气压计和方向传感器实时采集的时序数据,制作了包含3336条带有人工标注数据的人体姿态数据集,对行走、奔跑、上楼梯、下楼梯和平静五种日常持续性行为姿态与跌倒、起立、坐下和跳跃这四个突发行为姿态进行预测分类。对比LSTM网络与该研究领域内常用的浅层学习算法、深度学习全连接神经网络与卷积神经网络,实验结果表明,所提方法使用端对端的深度学习的方法相比基于所制作数据集的人体姿态检测算法模型的正确率提高了4.49个百分点,验证了该网络结构的泛化能力且更适合姿态检测。     

现代医院门诊量的灰色RBF神经网络预测

门诊量预测是现代医院电梯交通以及医疗资源优化配置的重要前提。为了有效地预测医院的门诊量,提出一种将灰色预测方法与RBF神经网络有机结合的灰色神经网络组合预测方法。该方法利用灰色预测中的累加生成运算（AGO）对原始观测数据进行变换,得到规律性较强的累加数据,作为神经网络的建模和训练样本。所提出的方法既避免了灰色预测方法存在的理论误差,又提高了神经网络的训练速度和预测精度,对短期的医院门诊量预测具有较强的实用价值。结果表明：所提出的方法具有良好的预测精度。     

Deep Learning Based Process Analytics Model for Predicting Type 2 Diabetes Mellitus

Process analytics is one of the popular research domains that advanced in the recent years. Process analytics encompasses identification, monitoring, and improvement of the processes through knowledge extraction from historical data. The evolution of Artificial Intelligence (AI)-enabled Electronic Health Records (EHRs) revolutionized the medical practice. Type 2 Diabetes Mellitus (T2DM) is a syndrome characterized by the lack of insulin secretion. If not diagnosed and managed at early stages, it may produce severe outcomes and at times, death too. Chronic Kidney Disease (CKD) and Coronary Heart Disease (CHD) are the most common, long-term and life-threatening diseases caused by T2DM. Therefore, it becomes inevitable to predict the risks of CKD and CHD in T2DM patients. The current research article presents automated Deep Learning (DL)-based Deep Neural Network (DNN) with Adagrad Optimization Algorithm i.e., DNN-AGOA model to predict CKD and CHD risks in T2DM patients. The paper proposes a risk prediction model for T2DM patients who may develop CKD or CHD. This model helps in alarming both T2DM patients and clinicians in advance. At first, the proposed DNN-AGOA model performs data preprocessing to improve the quality of data and make it compatible for further processing. Besides, a Deep Neural Network (DNN) is employed for feature extraction, after which sigmoid function is used for classification. Further, Adagrad optimizer is applied to improve the performance of DNN model. For experimental validation, benchmark medical datasets were used and the results were validated under several dimensions. The proposed model achieved a maximum precision of 93.99%, recall of 94.63%, specificity of 73.34%, accuracy of 92.58%, and F-score of 94.22%. The results attained through experimentation established that the proposed DNN-AGOA model has good prediction capability over other methods.