基于DNN的低资源语音识别特征提取技术

针对低资源训练数据条件下深层神经网络（Deep neural network,DNN）特征声学建模性能急剧下降的问题,提出两种适合于低资源语音识别的深层神经网络特征提取方法.首先基于隐含层共享训练的网络结构,借助资源较为丰富的语料实现对深层瓶颈神经网络的辅助训练,针对BN层位于共享层的特点,引入Dropout,Maxout,Rectified linear units等技术改善多流训练样本分布不规律导致的过拟合问题,同时缩小网络参数规模、降低训练耗时;其次为了改善深层神经网络特征提取方法,提出一种基于凸非负矩阵分解（Convex-non-negative matrix factorization,CNMF）算法的低维高层特征提取技术,通过对网络的权值矩阵分解得到基矩阵作为特征层的权值矩阵,然后从该层提取一种新的低维特征.基于Vystadial 2013的1小时低资源捷克语训练语料的实验表明,在26.7小时的英语语料辅助训练下,当使用Dropout和Rectified linear units时,识别率相对基线系统提升7.0%;当使用Dropout和Maxout时,识别率相对基线系统提升了12.6%,且网络参数数量相对其他系统降低了62.7%,训练时间降低了25%.而基于矩阵分解的低维特征在单语言训练和辅助训练的两种情况下都取得了优于瓶颈特征（Bottleneck features,BNF）的识别率,且在辅助训练的情况下优于深层神经网络隐马尔科夫识别系统,提升幅度从0.8%~3.4%不等.     

Validation of artificial neural network techniques in the estimation of nitrogen concentration in rape using canopy hyperspectral reflectance data

A systematic comparison of two types of method for estimating the nitrogen concentration of rape is presented: the traditional statistical method based on linear regression and the emerging computationally powerful technique based on artificial neural networks (ANN). Five optimum bands were selected using stepwise regression. Comparison between the two methods was based primarily on analysis of the statistic parameters. The rms. error for the back-propagation network (BPN) was significantly lower than that for the stepwise regression method, and the T-value was higher for BPN. In particular, for the first-difference of inverse-log spectra (log 1/R)′, T-values performed with a 127.71% success rate using BPN. The results show that the neural network is more robust to training and estimating rape nitrogen concentrations using canopy hyperspectral reflectance data.     

Dealing with missing values in neural network-based diagnostic systems

Backpropagation neural networks have been applied to prediction and classification problems in many real world situations. However, a drawback of this type of neural network is that it requires a full set of input data, and real world data is seldom complete. We have investigated two ways of dealing with incomplete data — network reduction using multiple neural network classifiers, and value substitution using estimated values from predictor networks — and compared their performance with an induction method. On a thyroid disease database collected in a clinical situation, we found that the network reduction method was superior. We conclude that network reduction can be a useful method for dealing with missing values in diagnostic systems based on backpropagation neural networks.     

Counter-Propagation Neural Networks for Molecular Sequence Classification: Supervised LVQ and Dynamic Node Allocation

A modified counter-propagation (CP) algorithm with supervised learning vector quantizer (LVQ) and dynamic node allocation has been developed for rapid classification of molecular sequences. The molecular sequences were encoded into neural input vectors using an n–gram hashing method for word extraction and a singular value decomposition (SVD) method for vector compression. The neural networks used were three-layered, forward-only CP networks that performed nearest neighbor classification. Several factors affecting the CP performance were evaluated, including weight initialization, Kohonen layer dimensioning, winner selection and weight update mechanisms. The performance of the modified CP network was compared with the back-propagation (BP) neural network and the k–nearest neighbor method. The major advantages of the CP network are its training and classification speed and its capability to extract statistical properties of the input data. The combined BP and CP networks can classify nucleic acid or protein sequences with a close to 100% accuracy at a rate of about one order of magnitude faster than other currently available methods.     

Application of InP Neural Network to ECG BeatClassification

This paper presents an application of a hybrid neural network structure to the classification of the electrocardiogram (ECG) beats. Three different feature extraction methods are comparatively examined: discrete cosine transform, wavelet transform and a direct method. Classification performances, training times and the numbers of nodes of Kohonen network, Restricted Coulomb Energy (RCE) network and the hybrid neural network are presented. To increase the classification performance and to decrease the number of nodes, the hybrid neural network is trained by Genetic Algorithms (GAs). Ten types of ECG beats obtained from the MIT-BIH database and from a real-time ECG measurement system are classified with a success of 98% by using the hybrid neural network structure and discrete cosine transform together.     

基于半监督动态深度融合神经网络的软测量

半监督深度神经网络建模方法已被广泛应用于软测量,但基于分层训练的网络在特征提取过程局限于挖掘每层输入的有效信息,忽略了原始输入有效信息的丢失,逐层累积,从而导致原始输入的特征表示准确率低下;另外,缺乏挖掘过程时空相关性,也会导致模型性能退化。提出一种半监督动态深度融合神经网络（semisupervised dynamics deep fusion neural network,SS-DDFNN）方法。该方法在特征提取网络的每层都重构原始输入数据并预测质量变量,通过在预训练损失中使用重构原始输入误差,减小原始输入有效信息的丢失;同时融入注意力机制和t分布随机邻域嵌入提取时空相关信息,应用提取的特征建立门控神经网络质量预测模型。实验结果显示,相较于SAE、GSTAE和SIAE模型,所提方法在脱丁烷塔案例中的预测精度分别提升了2.8%、1.1%和0.9%;在工业聚乙烯生产案例中,分别提升了2.7%、1.0%和0.7%。实验结果验证了所提方法的有效性。     

Design of a feature-tuned ANN model based on bulk rock-derived mineral spectra for endmember classification of a hyperspectral image from an iron ore deposit

In this article, a new feature-tuned artificial neural network (ANN) model has been developed for endmember classification of a hyperspectral image. This model is developed on the basis of using only the essential absorption bands of mineral spectra as opposed to using all the spectral bands of the hyperspectral image. This approach has the added advantages of reducing the dimensionality of input features to the ANN as well as inhibiting the influences of noisy bands for classification of endmembers. The proposed ANN model is trained using input features extracted from laboratory spectra of in situ bulk ore materials collected from an existing iron ore deposit. The input features are basically the constituent absorption bands of mineral spectra where each absorption band is mathematically characterized by the centre, width, and strength parameters of a Gaussian curve. For extracting absorption bands from a mineral spectrum, a modified Gaussian model has been used. The application of this model also necessitates the design of a special template for the input layer ANN model. After training the model, its generalization property is assessed through a testing data set. The model has achieved nearly 97% of classification accuracy in a training set, and 71% of accuracy in a testing set. The trained model is then applied on Hyperion imagery collected over an iron ore deposit. All the endmember spectra of this deposit are classified into either vegetation or any of the ores or rock present in the deposit. None of the endmembers is classified into non-iron ore minerals.     

A neural network-based approach for dynamic quality prediction in a plastic injection molding process

This paper presents an innovative neural network-based quality prediction system for a plastic injection molding process. A self-organizing map plus a back-propagation neural network (SOM-BPNN) model is proposed for creating a dynamic quality predictor. Three SOM-based dynamic extraction parameters with six manufacturing process parameters and one level of product quality were dedicated to training and testing the proposed system. In addition, Taguchi’s parameter design method was also applied to enhance the neural network performance. For comparison, an additional back-propagation neural network (BPNN) model was constructed for which six process parameters were used for training and testing. The training and testing data for the two models respectively consisted of 120 and 40 samples. Experimental results showed that such a SOM-BPNN-based model can accurately predict the product quality (weight) and can likely be used for various practical applications.     

基于自适应模糊聚类的神经网络软测量建模方法

提出一种基于模糊聚类的神经网络软测量建模方法．该方法采用数据分组训练、自动确定模糊分类数、在线测量时分类中心自适应修正，降低了计算量，提高了建模精度．将该算法用于步进式加热炉钢坯温度预报的仿真结果表明，它能够解决钢坯温度难以在线测量的问题。     

基于PSO的神经网络在传感器 数据融合中的应用

针对压力传感器对温度存在交叉灵敏度这一具体问题,常采用BP神经网络对其进行数据融合.但BP神经网络方法训练收敛速度慢,易陷入局部最优.采用PSO全局优化算法训练多层前向神经网络权值,使网络训练误差比BP方法降低了两个数量级,并且收敛速度明显加快.融合结果表明基于PSO神经网络方法更有效地消除了温度对压力传感器的影响,显著提高了传感器的稳定性和准确度.     

基于时域建模的自动语音识别

端到端神经网络能够根据特定的任务自动学习从原始数据到特征的变换,解决人工设计的特征与任务不匹配的问题。以往语音识别的端到端网络采用一层时域卷积网络作为特征提取模型,递归神经网络和全连接前馈深度神经网络作为声学模型的方式,在效果和效率两个方面具有一定的局限性。从特征提取模块的效果以及声学模型的训练效率角度,提出多时间频率分辨率卷积网络与带记忆模块的前馈神经网络相结合的端到端语音识别模型。实验结果表明,所提方法语音识别在真实录制数据集上较传统方法字错误率下降10%,训练时间减少80%。     

A Compact Convolutional Neural Network Augmented with Multiscale Feature Extraction of Acquired Monitoring Data for Mechanical Intelligent Fault Diagnosis

Considering all the monitoring data of bearings until failure, very few data are acquired when the bearings are faulty. Such circumstance leads to small faulty sample problem when an intelligent fault diagnosis method is applied. A deep neural network trained with small samples cannot be trained completely, and tends to overfit, which results in poor performance in practical application. To solve this problem, a compact convolutional neural network augmented with multiscale feature extraction is proposed in this paper. Multiscale feature extraction unit is introduced to extract features at different time scales without adding convolution layers, which can reduce the depth of the network while ensuring classification ability and alleviating the overfitting problem caused by the network being too complicated. Besides, a specially designed compact convolutional neural network synthetically analyzes the multiscale features. By combing these two tricks, the proposed neural network can extract more sensitive features with a relatively shallow structure, which increases classification accuracy under small samples. Dropout technique is also used to prevent the network from overfitting. Effectiveness of the proposed method is verified by three bearing datasets. Experiments show that this network can achieve competitive results with limited training samples even with different load and mixed rotating speed.     

基于RBFNN的铂电阻温度传感器非线性补偿

针对铂电阻温度传感器应用中存在的非线性问题,提出了应用径向基函数神经网络(RBFNN)强非线性逼近能力进行铂电阻温度传感器非线性补偿的方法。介绍了非线性补偿的原理和网络训练方法。结果表明:这种非线性补偿模型具有误差小、精度高、可在线标定和鲁棒性强等优点,与基于BP神经网络的非线性补偿模型相比,大大缩短了网络训练时间,从而方便了铂电阻温度传感器在测控系统中的应用。     

A neural network-based approach for calculating dissolved oxygen profiles in reservoirs

A Neural Network (NN) modelling approach has been shown to be successful in calculating pseudo steady state time and space dependent Dissolved Oxygen (DO) concentrations in three separate reservoirs with different characteristics using limited number of input variables. The Levenberg–Marquardt algorithm was adopted during training. Pre-processing before training and post processing after simulation steps were the treatments applied to raw data and predictions respectively. Generalisation was improved and over-fitting problems were eliminated: Early stopping method was applied for improving generalisation. The correlation coefficients between neural network estimates and field measurements were as high as 0.98 for two of the reservoirs with experiments that involve double layer neural network structure with 30 neurons within each hidden layer. A simple one layer neural network structure with 11 neurons has yielded comparable and satisfactorily high correlation coefficients for complete data set, and training, validation and test sets of the third reservoir.     

基于神经网络的无线传感器网络数据预测应用研究

无线传感器网络是一种由数量庞大的网络节点形成的复杂无线网络,是无线传感器的典型应用,目前已经广泛应用在多个领域当中。将神经网络引入到无线传感器网络当中,通过神经元描述每一个无线传感器数据,构建神经网络元模型。对传统的神经网络模型进行改进,利用无线传感器的神经网络模型,实现无线传感器网络采集数据的融合与提取。通过各种应用类型的差异,选择影响数据输出结果的主要因素,建立一种能够进行预测的模型。以某个区域是否发生火灾为实验原型,对该区域的火灾发生概率进行预测,采用已有的火灾发生数据为训练样本,通过收敛的网络预测火灾发生的概率。实验结果表明,基于神经网络的无线传感器网络数据预测是一种可行、有效的方法。     

高光谱图像小样本分类的卷积神经网络方法

目的 与传统分类方法相比,基于深度学习的高光谱图像分类方法能够提取出高光谱图像更深层次的特征。针对现有深度学习的分类方法网络结构简单、特征提取不够充分的问题,提出一种堆叠像元空间变换信息的数据扩充方法,用于解决训练样本不足的问题,并提出一种基于不同尺度的双通道3维卷积神经网络的高光谱图像分类模型,来提取高光谱图像的本质空谱特征。方法 通过对高光谱图像的每一像元及其邻域像元进行旋转、行列变换等操作,丰富中心像元的潜在空间信息,达到数据集扩充的作用。将扩充之后的像素块输入到不同尺度的双通道3维卷积神经网络学习训练集的深层特征,实现更高精度的分类。结果 5次重复实验后取平均的结果表明,在随机选取了10%训练样本并通过8倍数据扩充的情况下,Indian Pines数据集实现了98.34%的总体分类精度,Pavia University数据集总体分类精度达到99.63%,同时对比了不同算法的运行时间,在保证分类精度的前提下,本文算法的运行时间短于对比算法,保证了分类模型的稳定性、高效性。结论 本文提出的基于双通道卷积神经网络的高光谱图像分类模型,既解决了训练样本不足的问题,又综合了高光谱图像的光谱特征和空间特征,提高了高光谱图像的分类精度。     

CDUL: Class directed unsupervised learning

A novel neural network called Class Directed Unsupervised Learning (CDUL) is introduced. The architecture, based on a Kohonen self-organising network, uses additional input nodes to feed class knowledge to the network during training, in order to optimise the final positioning of Kohonen nodes in feature space. The structure and training of CDUL networks is detailed, showing that (a) networks cannot suffer from the problem of single Kohonen nodes being trained by vectors of more than one class, (b) the number of Kohonen nodes necessary to represent the classes is found during training, and (c) the number of training set passes CDUL requires is low in comparison to similar networks. CDUL is subsequently applied to the classification of chemical excipients from Near Infrared (NIR) reflectance spectra, and its performance compared with three other unsupervised paradigms. The results thereby obtained demonstrate a superior performance which remains relatively constant through a wide range of network parameters.     

Relating product specifications and performance data with a neural network model for design improvement

This paper presents research resulting in a neural network model relating product design specifications and performance testing results using data from a sanitary ware manufacturer. The main constraint of the work was the limited availability of actual data for neural network training and testing, a situation often found in real situations where a priori product knowledge is limited during the product design phase. The authors used two training techniques, the standard hold-back and the leave-k-out, for the neural network model to leverage the sparseness of the data. Neural network results are compared and contrasted to statistical models relating product design and performance. This work is an exploration of the value of neural network models to assist with interactive product design.     

Using counterpropagation neural networks for partial discharge diagnosis

In high voltage engineering, various methods of non-destructive fault diagnosis are applied for investigating the quality of insulating materials and systems. The methods are aimed at classifying patterns derived from the measured characteristics of the electrical signals typically resulting from insulation defects. In this paper, variants of the counterpropagation neural network architecture are used to classify patterns representing various properties of partial discharges. It is shown that the classification quality can be improved considerably when an extended counterpropagation network with a dynamically changing network topology, and an additional vigilance unit for monitoring the behaviour of the network during the learning phase is applied. The extended network has significant advantages over the standard counterpropagation network in cases where outliers in the training data seriously degrade the approximation quality of the standard network. When using the proposed network in conjunction with physically motivated discharge data, input patterns from defect categories not considered during training can be rejected more reliably. This rejection problem is particularly important for practical applications where misclassifications cannot be tolerated.