Combining neural networks and semantic feature space for email classification

Email is one of the most ubiquitous and pervasive applications used on a daily basis by millions of people worldwide, individuals and organizations more and more rely on the emails to communicate and share information and knowledge. However, the increase in email users has resulted in a dramatic increase in spam emails during the past few years. It is becoming a big challenge to process and manage the emails efficiently for and individuals and organizations. This paper proposes new email classification models using a linear neural network trained by perceptron learning algorithm and a nonlinear neural network trained by back-propagation learning algorithm. An efficient semantic feature space (SFS) method is introduced in these classification models. The traditional back-propagation neural network (BPNN) has slow learning speed and is prone to trap into a local minimum, so the modified back-propagation neural network (MBPNN) is presented to overcome these limitations. The vector space model based email classification system suffers from a large number of features and ambiguity in the meaning of terms, which will lead to sparse and noisy feature space. So we use the SFS to convert the original sparse and noisy feature space to a semantically richer feature space, which will helps to accelerate the learning speed. The experiments are conducted based on different training set size and extracted feature size. Experimental results show that the models using MBPNN outperform the traditional BPNN, and the use of SFS can greatly reduce the feature dimensionality and improve email classification performance.     

An efficient document classification model using an improved back propagation neural network and singular value decomposition

This paper proposed a new improved method for back propagation neural network, and used an efficient method to reduce the dimension and improve the performance. The traditional back propagation neural network (BPNN) has the drawbacks of slow learning and is easy to trap into a local minimum, and it will lead to a poor performance and efficiency. In this paper, we propose the learning phase evaluation back propagation neural network (LPEBP) to improve the traditional BPNN. We adopt a singular value decomposition (SVD) technique to reduce the dimension and construct the latent semantics between terms. Experimental results show that the LPEBP is much faster than the traditional BPNN. It also enhances the performance of the traditional BPNN. The SVD technique cannot only greatly reduce the high dimensionality but also enhance the performance. So SVD is to further improve the document classification systems precisely and efficiently.     

Text categorization algorithms using semantic approaches, corpus-based thesaurus and WordNet

In this paper, a corpus-based thesaurus and WordNet were used to improve text categorization performance. We employed the k-NN algorithm and the back propagation neural network (BPNN) algorithms as the classifiers. The k-NN is a simple and famous approach for categorization, and the BPNNs has been widely used in the categorization and pattern recognition fields. However the standard BPNN has some generally acknowledged limitations, such as a slow training speed and can be easily trapped into a local minimum. To alleviate the problems of the standard BPNN, two modified versions, Morbidity neurons Rectified BPNN (MRBP) and Learning Phase Evaluation BPNN (LPEBP), were considered and applied to the text categorization. We conducted the experiments on both the standard reuter-21578 data set and the 20 Newsgroups data set. Experimental results showed that our proposed methods achieved high categorization effectiveness as measured by the precision, recall and F-measure protocols.     

Modelling of diesel engine performance using advanced machine learning methods under scarce and exponential data set

Traditional methods on creating diesel engine models include the analytical methods like multi-zone models and the intelligent based models like artificial neural network (ANN) based models. However, those analytical models require excessive assumptions while those ANN models have many drawbacks such as the tendency to overfitting and the difficulties to determine the optimal network structure. In this paper, several emerging advanced machine learning techniques, including least squares support vector machine (LS-SVM), relevance vector machine (RVM), basic extreme learning machine (ELM) and kernel based ELM, are newly applied to the modelling of diesel engine performance. Experiments were carried out to collect sample data for model training and verification. Limited by the experiment conditions, only 24 sample data sets were acquired, resulting in data scarcity. Six-fold cross-validation is therefore adopted to address this issue. Some of the sample data are also found to suffer from the problem of data exponentiality, where the engine performance output grows up exponentially along the engine speed and engine torque. This seriously deteriorates the prediction accuracy. Thus, logarithmic transformation of dependent variables is utilized to pre-process the data. Besides, a hybrid of leave-one-out cross-validation and Bayesian inference is, for the first time, proposed for the selection of hyperparameters of kernel based ELM. A comparison among the advanced machine learning techniques, along with two traditional types of ANN models, namely back propagation neural network (BPNN) and radial basis function neural network (RBFNN), is conducted. The model evaluation is made based on the time complexity, space complexity, and prediction accuracy. The evaluation results show that kernel based ELM with the logarithmic transformation and hybrid inference is far better than basic ELM, LS-SVM, RVM, BPNN and RBFNN, in terms of prediction accuracy and training time.     

Parallel Training of An Improved Neural Network for Text Categorization

This paper studies parallel training of an improved neural network for text categorization. With the explosive growth on the amount of digital information available on the Internet, text categorization problem has become more and more important, especially when millions of mobile devices are now connecting to the Internet. Improved back-propagation neural network (IBPNN) is an efficient approach for classification problems which overcomes the limitations of traditional BPNN. In this paper, we utilize parallel computing to speedup the neural network training process of IBPNN. The parallel IBNPP algorithm for text categorization is implemented on a Sun Cluster with 34 nodes (processors). The communication time and speedup for the parallel IBPNN versus various number of nodes are studied. Experiments are conducted on various data sets and the results show that the parallel IBPNN together with SVD technique achieves fast computational speed and high text categorization correctness.     

一种通过反馈提高神经网络学习性能的新算法

为了有效提高前向神经网络的学习性能，需要从一个新的角度考虑神经网络的学习训练．基于此，提出了一种基于结果反馈的新算法——FBBP算法．将神经网络输入调整与通常的权值调整的反向传播算法结合起来，通过调整权值和输入矢量值的双重作用来最小化神经网络的误差函数．并通过几个函数逼近和模式分类问题的实例仿真，将FBBP算法与加动量项BP算法、最新的一种加快收敛的权值更新的算法进行了比较，来验证所提出的算法的有效性．实验结果表明，所提出的算法具有训练速度快和泛化能力高的双重优点，是一种非常有效的学习方法．     

基于支持向量机的丙酮精制建模

采用支持向量机建立了丙酮精制过程的产品质量与生产工艺参数之间的预测模型,并将其与反向传播神经网络和径向基神经网络模型相比较。在实际工业数据上进行的实验结果表明,支持向量机模型对丙酮纯度具有良好的预测效果,性能优于反向传播神经网络和径向基网络模型。     

An automatically constructed thesaurus for neural network based document categorization

This paper presents a method for computing a thesaurus from a text corpus, and combined with a revised back-propagation neural network (BPNN) learning algorithm for document categorization. Automatically constructed thesaurus is a data structure that accomplished by extracting the relatedness between words. Neural network is one of the efficient approaches for document categorization. However the conventional BPNN has the problems of slow learning and easy to involve into the local minimum. We use a revised algorithm to improve the conventional BPNN that can overcome these problems. A well constructed thesaurus has been recognized as valuable tool in the effective operation of document categorization, it overcome some problem for the document categorization based on bag of words which ignored the relationship between words. To investigate the effectiveness of our method, we conducted the experiments on the standard Reuter-21578. The experimental results show that the proposed model was able to achieve higher categorization effectiveness as measured by the precision, recall and F-measure.     

文本分类的神经网络模型

给出一种与文档段落结构相关联的文本分类神经网络模型。描述神经网络的训练算法,包括正向传播算法和反向修正算法。对于算法的主要步骤,给出了更详细计算方法。最后给出了神经网络模型性能测试结果。     

Automatic thesaurus construction for spam filtering using revised back propagation neural network

Email has become one of the fastest and most economical forms of communication. Email is also one of the most ubiquitous and pervasive applications used on a daily basis by millions of people worldwide. However, the increase in email users has resulted in a dramatic increase in spam emails during the past few years. This paper proposes a new spam filtering system using revised back propagation (RBP) neural network and automatic thesaurus construction. The conventional back propagation (BP) neural network has slow learning speed and is prone to trap into a local minimum, so it will lead to poor performance and efficiency. The authors present in this paper the RBP neural network to overcome the limitations of the conventional BP neural network. A well constructed thesaurus has been recognized as a valuable tool in the effective operation of text classification, it can also overcome the problems in keyword-based spam filters which ignore the relationship between words. The authors conduct the experiments on Ling-Spam corpus. Experimental results show that the proposed spam filtering system is able to achieve higher performance, especially for the combination of RBP neural network and automatic thesaurus construction.     

基于免疫RBF神经网络的语音情感识别

本文针对语音情感识别中BP神经网络收敛速度慢和正确率低的问题,提出了一种训练径向基函数(RBF)网络的混合算法。在语音情感特征提取的基础上,采用免疫RBF神经网络进行情感识别,同时还训练了一个BP网络进行对比实验,得到了比较理想的识别结果。     

TCBPL:一种高效文本分类新方法

根据模式聚合理论（PA）和隐含语义分析理论（LSA）提出了一种文本分类新方法——TCBPL方法,该方法应用PA理论和LSA理论来构造向量空间模型,大大削减了特征向量的维数,同时增强了稀有词的作用,并在特征向量中引入了语义成分,从而提高了分类的速度和精度。     

距离函数分类法在垃圾邮件过滤中的应用

为了得到实用性强的垃圾邮件过滤方法,将距离函数分类法首次引入到垃圾邮件过滤中.在通用邮件语料库上进行测试,并与目前过滤性能较好的KNN算法进行比较,实验结果显示距离函数分类法中的类中心向量法不适合用于垃圾邮件的过滤,而类重心向量法在保持较高过滤性能的同时,具有训练和过滤速度快的优点,是一种理想实用的垃圾邮件过滤方法.     

一种基于LSA和Kohonen网络的文本分类新方法

文中根据隐含语义分析理论(LSA)和Kohonen网络理论提出一种文本分类新方法。应用Kohonen网络进行文本分类存在训练速度慢的缺点，因此在网络训练阶段引入了有监督机制，提高了网络的分类速度和精度；但是对于高维的文本特征向量来说，分类速度很低，甚至应用Kohonen网络进行分类，不能取得理想结果；新方法应用LSA理论来建立文本集的向量空间模型，在词条的权重中引入了语义关系，消减了原词条矩阵中包含的“噪声”因素，从而更加突出了词和文本之间的语义关系。通过奇异值分解(SVD)，有效地降低了向量空间的维数，从而大大提高了文本分类的精度和速度，同时根据因子分析理论给出了维数K的选取方法。     

超机动飞机的动态建模与控制律设计及仿真

建立了带推力矢量的超机动飞机非线性动态模型,重点分析了气动力、气动力矩以及发动机的建模过程.采用基于神经网络的自适应逆方法,设计了超机动飞机大迎角机动下的控制律.首先应用动态逆方法,分别设计了快慢回路的飞行控制律;然后利用BP神经网络,在线补偿飞机模型不确定性以及外界干扰.眼镜蛇机动的仿真结果表明,所设计的控制律在大迎角机动条件下具有良好的控制性能,能够保证闭环系统的稳定性.     

Artificial neural network based prediction of drill flank wear from motor current signals

In this work, a multilayer neural network with back propagation algorithm (BPNN) has been applied to predict the average flank wear of a high speed steel (HSS) drill bit for drilling on a mild steel work piece. Root mean square (RMS) value of the spindle motor current, drill diameter, spindle speed and feed-rate are inputs to the network, and drill wear is the output. Drilling experiments have been carried out over a wide range of cutting conditions and the effects of drill wear, cutting conditions (speed, drill diameter, feed-rate) on the spindle motor current have been investigated. The performance of the trained neural network has been tested for new cutting conditions, and found to be in very good agreement to the experimentally determined drill wear values. The accuracy of the prediction of drill wear using neural network is found to be better than that using regression model.     

潜在语义分析在连续语音识别中的应用

研究了潜在语义分析（LSA）理论及其在连续语音识别中应用的相关技术,在此基础上利用WSJ0文本语料库上构建LSA模型,并将其与3-gram模型进行插值组合,构建了包含语义信息的统计语言模型;同时为了进一步优化混合模型的性能,提出了基于密度函数初始化质心的k-means聚类算法对LSA模型的向量空间进行聚类。WSJ0语料库上的连续语音识别实验结果表明：LSA+3-gram混合模型能够使识别的词错误率相比较于标准的3-gram下降13.3%。     

Nondestructive measurement of internal quality attributes of apple fruit by using NIR spectroscopy

Multimedia Tools and Applications - In this paper, a hybrid approach, which combines back propagation neural network (BPNN), generalized regression neural network (GRNN) and particle swarm...     

The parameters effect on performance in ANN for hand gesture recognition system

The effectiveness of a neural network function depends on the network architecture and parameters. For discussing the relationship of parameters and performance, this study proposes a novel hand gesture recognition system (HGRS) combining the VICON and the back propagation neural network (BPNN). In this study, different numbers of hidden layer neurons and different numbers of layers were compared for effects on system performance. Too many or too few neurons reduced the recognition rate. Further, the hidden layer was needed for improving the system performance of the system. The training epoch size affects the general ability of the system. If the epoch size is too large, the system “over fit” the training set, and its general ability is impaired. However, an overly small epoch size would impair system recognition. The learning rate and system momentum affect the RMSE of the trained system. A higher learning rate and reduced momentum decrease RMSE.     

Effective recognition of control chart patterns in autocorrelated data using a support vector machine based approach

The effective recognition of unnatural control chart patterns (CCPs) is a critical issue in statistical process control, as unnatural CCPs can be associated with specific assignable causes adversely affecting the process. Machine learning techniques, such as artificial neural networks (ANNs), have been widely used in the research field of CCP recognition. However, ANN approaches can easily overfit the training data, producing models that can suffer from the difficulty of generalization. This causes a pattern misclassification problem when the training examples contain a high level of background noise (common cause variation). Support vector machines (SVMs) embody the structural risk minimization, which has been shown to be superior to the traditional empirical risk minimization principle employed by ANNs. This research presents a SVM-based CCP recognition model for the on-line real-time recognition of seven typical types of unnatural CCP, assuming that the process observations are AR(1) correlated over time. Empirical comparisons indicate that the proposed SVM-based model achieves better performance in both recognition accuracy and recognition speed than the model based on a learning vector quantization network. Furthermore, the proposed model is more robust toward background noise in the process data than the model based on a back propagation network. These results show the great potential of SVM methods for on-line CCP recognition.