Text categorization based on combination of modified back propagation neural network and latent semantic analysis

This paper proposed a new text categorization model based on the combination of modified back propagation neural network (MBPNN) and latent semantic analysis (LSA). The traditional back propagation neural network (BPNN) has slow training speed and is easy to trap into a local minimum, and it will lead to a poor performance and efficiency. In this paper, we propose the MBPNN to accelerate the training speed of BPNN and improve the categorization accuracy. LSA can overcome the problems caused by using statistically derived conceptual indices instead of individual words. It constructs a conceptual vector space in which each term or document is represented as a vector in the space. It not only greatly reduces the dimension but also discovers the important associative relationship between terms. We test our categorization model on 20-newsgroup corpus and reuter-21578 corpus, experimental results show that the MBPNN is much faster than the traditional BPNN. It also enhances the performance of the traditional BPNN. And the application of LSA for our system can lead to dramatic dimensionality reduction while achieving good classification results.     

Text categorization algorithms using semantic approaches, corpus-based thesaurus and WordNet

In this paper, a corpus-based thesaurus and WordNet were used to improve text categorization performance. We employed the k-NN algorithm and the back propagation neural network (BPNN) algorithms as the classifiers. The k-NN is a simple and famous approach for categorization, and the BPNNs has been widely used in the categorization and pattern recognition fields. However the standard BPNN has some generally acknowledged limitations, such as a slow training speed and can be easily trapped into a local minimum. To alleviate the problems of the standard BPNN, two modified versions, Morbidity neurons Rectified BPNN (MRBP) and Learning Phase Evaluation BPNN (LPEBP), were considered and applied to the text categorization. We conducted the experiments on both the standard reuter-21578 data set and the 20 Newsgroups data set. Experimental results showed that our proposed methods achieved high categorization effectiveness as measured by the precision, recall and F-measure protocols.     

基于BPNN端到端时延预测的多路传输调度

针对移动异构网络环境网络参数动态变化、多路传输过程中数据包乱序引发的吞吐量下降等问题,提出一种基于BP神经网络(back propagation neural network,BPNN)端到端时延预测的多路传输调度方法,通过BP神经网络的构建、训练和学习,实现对端到端传输时延的更准确预测,以此为基础,对子流拥塞状况及网...     

Combining neural networks and semantic feature space for email classification

Email is one of the most ubiquitous and pervasive applications used on a daily basis by millions of people worldwide, individuals and organizations more and more rely on the emails to communicate and share information and knowledge. However, the increase in email users has resulted in a dramatic increase in spam emails during the past few years. It is becoming a big challenge to process and manage the emails efficiently for and individuals and organizations. This paper proposes new email classification models using a linear neural network trained by perceptron learning algorithm and a nonlinear neural network trained by back-propagation learning algorithm. An efficient semantic feature space (SFS) method is introduced in these classification models. The traditional back-propagation neural network (BPNN) has slow learning speed and is prone to trap into a local minimum, so the modified back-propagation neural network (MBPNN) is presented to overcome these limitations. The vector space model based email classification system suffers from a large number of features and ambiguity in the meaning of terms, which will lead to sparse and noisy feature space. So we use the SFS to convert the original sparse and noisy feature space to a semantically richer feature space, which will helps to accelerate the learning speed. The experiments are conducted based on different training set size and extracted feature size. Experimental results show that the models using MBPNN outperform the traditional BPNN, and the use of SFS can greatly reduce the feature dimensionality and improve email classification performance.     

基于BP神经网络的人体行为识别

针对人体行为识别问题,提出一种基于径向基函数（BP）神经网络的人体行为分类算法。首先,利用奇异值分解（SVD）算法提取视频每一帧的奇异值,将每一帧的奇异值按照行拼接起来即为一个视频的样本,样本按照行排成样本矩阵;然后,利用主成分分析（PCA）对得到的矩阵进行去相关并且降低维数,降低维数的矩阵再进行线性鉴别分析（LDA）,使样本变得线性可分;最后,利用BP神经网络对样本进行分类;实验结果表明,与采用最近邻分类和K近邻分类（kNN）相比,所提算法具有更高的识别率。     

一种提高云计算环境下检测入侵者速度和正确率的新方法

云计算环境要求入侵检测系统(IDS)极其快速和准确,用于云计算的智能型IDS——反向传播神经元网络(BPNN)经常出现“泛化问题”,即BPNN无解或总误差函数不能收敛于全局最小值。泛化问题降低了BPNN的识别速度和正确率。为了解决该问题,提出两种解决方法。第一种是剔除相关性大的那些特征,保留相关性较小或互相独立的重要特征,以便减少特征数量。第二种是综合利用遗传算法(GA)和最速下降法(梯度法)的优点,降低BPNN的泛化性。用GA求出BPNN全局最优解的近似值,把该近似值作为最速下降法的初始值,进行迭代,最终求得BPNN全局最优解的精确值。仿真实验表明给出的方法是有效的且能够解决BPNN的“泛化问题”,同时能够提高BPNN识别入侵者的速度和正确率。     

Modelling of diesel engine performance using advanced machine learning methods under scarce and exponential data set

Traditional methods on creating diesel engine models include the analytical methods like multi-zone models and the intelligent based models like artificial neural network (ANN) based models. However, those analytical models require excessive assumptions while those ANN models have many drawbacks such as the tendency to overfitting and the difficulties to determine the optimal network structure. In this paper, several emerging advanced machine learning techniques, including least squares support vector machine (LS-SVM), relevance vector machine (RVM), basic extreme learning machine (ELM) and kernel based ELM, are newly applied to the modelling of diesel engine performance. Experiments were carried out to collect sample data for model training and verification. Limited by the experiment conditions, only 24 sample data sets were acquired, resulting in data scarcity. Six-fold cross-validation is therefore adopted to address this issue. Some of the sample data are also found to suffer from the problem of data exponentiality, where the engine performance output grows up exponentially along the engine speed and engine torque. This seriously deteriorates the prediction accuracy. Thus, logarithmic transformation of dependent variables is utilized to pre-process the data. Besides, a hybrid of leave-one-out cross-validation and Bayesian inference is, for the first time, proposed for the selection of hyperparameters of kernel based ELM. A comparison among the advanced machine learning techniques, along with two traditional types of ANN models, namely back propagation neural network (BPNN) and radial basis function neural network (RBFNN), is conducted. The model evaluation is made based on the time complexity, space complexity, and prediction accuracy. The evaluation results show that kernel based ELM with the logarithmic transformation and hybrid inference is far better than basic ELM, LS-SVM, RVM, BPNN and RBFNN, in terms of prediction accuracy and training time.     

On-line tuning PID parameters in an idling engine based on a modified BP neural network by particle swarm optimization

PID control systems are widely used in many fields, and many methods to tune the parameters of PID controllers are known. When the characteristics of the object are changed, the traditional PID control should be adjusted by empirical knowledge. This may result in a worse performance by the system. In this article, a new method to tune PID parameters, called the back-propagation network modified by particle swarm optimization, is proposed. This algorithm combines conventional PID control with a back propagation neural network (BPNN) and particle swarm optimization (PSO). This method is demonstrated in the engine idling-speed control problem. The proposed method provides considerable performance benefits compared with a traditional controller in this simulation.     

A self-adaptive data analysis for fault diagnosis of an automotive air-conditioner blower

This paper presents a fault diagnosis system for an automotive air-conditioner blower based on a noise emission signal using a self-adaptive data analysis technique. The proposed diagnosis system consists of feature extraction using the empirical mode decomposition (EMD) method and fault classification using the artificial neural network technique. The EMD method has been developed quite recently to adaptively decompose the non-stationary and non-linear signals. It sifts the complex signal of time series without losing its original properties and then obtains some useful intrinsic mode function (IMF) components. Calculating the energy of each component can reduce the computation dimensions and enhance classification performance. These energy features of various fault conditions are used as inputs to train the artificial neural network. In the fault classification, the probabilistic neural network (PNN) is used to verify the performance of the proposed system and compare with the traditional technique, back-propagation neural network (BPNN). The experimental results indicated the proposed technique performed well for quickly and accurately estimating fault conditions.     

Nondestructive measurement of internal quality attributes of apple fruit by using NIR spectroscopy

Multimedia Tools and Applications - In this paper, a hybrid approach, which combines back propagation neural network (BPNN), generalized regression neural network (GRNN) and particle swarm...     

BP神经网络FPGA实现结构的优化设计

为了实现反向传播（back propagation,BP）神经网络的现场可编程门阵列（field programmable gate array,FPGA）处理速度的提升和资源消耗的降低,提出一种总体设计和关键模块融合优化的BP神经网络的FPGA实现结构。利用定点数据量化和流水线结构,提高系统的处理速度;采用二次方程多段拟合Sigmoid激活函数,降低计算复杂度;通过调整并行转串行模块与激活函数模块的处理顺序,减少了95%的激活函数模块的使用,降低了资源消耗;采用一种网络原始权值读取与更新权值存储交替流水进行的双端口RAM存取方法,以提高数据存取的速度、降低存储资源消耗。经过对硬件优化设计的字符和服装识别实验验证,结果表明,优化后的总逻辑单元使用率为原来的31%。在FPGA中优化结构实现单样本前向传播与反向传播所用时间为24.332?μs,为软件MATLAB实现时间的45.63%,提高了BP神经网络的运算速度。     

基于免疫RBF神经网络的语音情感识别

本文针对语音情感识别中BP神经网络收敛速度慢和正确率低的问题,提出了一种训练径向基函数(RBF)网络的混合算法。在语音情感特征提取的基础上,采用免疫RBF神经网络进行情感识别,同时还训练了一个BP网络进行对比实验,得到了比较理想的识别结果。     

一种通过反馈提高神经网络学习性能的新算法

为了有效提高前向神经网络的学习性能，需要从一个新的角度考虑神经网络的学习训练．基于此，提出了一种基于结果反馈的新算法——FBBP算法．将神经网络输入调整与通常的权值调整的反向传播算法结合起来，通过调整权值和输入矢量值的双重作用来最小化神经网络的误差函数．并通过几个函数逼近和模式分类问题的实例仿真，将FBBP算法与加动量项BP算法、最新的一种加快收敛的权值更新的算法进行了比较，来验证所提出的算法的有效性．实验结果表明，所提出的算法具有训练速度快和泛化能力高的双重优点，是一种非常有效的学习方法．     

Modes decomposition method in fusion with robust random vector functional link network for crude oil price forecasting

This paper proposes a decomposition based method in fusion with the non-iterative approach for crude oil price forecasting. In this approach, the robust random vector functional link network (RVFLN), a non-iterative approach in fusion with the most efficient decomposition technique called variational mode decomposition (VMD) is proposed which is executed with two links — fixed assigned random weights and direct link from input to output, and the iterative learning process is not involved in its functioning which makes it faster in execution as compared to many existing techniques proposed for forecasting. The fusion of VMD and robust RVFLN called VMD-RVFLN is implemented for crude oil price forecasting where the crude oil price series is decomposed using VMD into a linear smoother series by extracting useful information and the decomposed modes pass through the robust RVFLN model which produces the final forecasting values. The analysis performed in the study approves its efficiency and reports improvement in forecasting accuracy and execution time as compared to some of the traditional iterative techniques like BPNN (back propagation neural network), ARIMA (auto-regressive integrated moving average), LSSVR (least squares support vector regression), ANFIS (adaptive neuro-fuzzy inference system), IT2FNN (interval type-2 fuzzy neural network) and RNN (recurrent neural network), etc. However, both ELM and RVFLN without modes decomposition fusion exhibit less execution time at the cost of reduction in prediction accuracy.     

The parameters effect on performance in ANN for hand gesture recognition system

The effectiveness of a neural network function depends on the network architecture and parameters. For discussing the relationship of parameters and performance, this study proposes a novel hand gesture recognition system (HGRS) combining the VICON and the back propagation neural network (BPNN). In this study, different numbers of hidden layer neurons and different numbers of layers were compared for effects on system performance. Too many or too few neurons reduced the recognition rate. Further, the hidden layer was needed for improving the system performance of the system. The training epoch size affects the general ability of the system. If the epoch size is too large, the system “over fit” the training set, and its general ability is impaired. However, an overly small epoch size would impair system recognition. The learning rate and system momentum affect the RMSE of the trained system. A higher learning rate and reduced momentum decrease RMSE.     

An efficient CMAC neural network for stock index forecasting

Stock index forecasting is one of the major activities of financial firms and private investors in making investment decisions. Although many techniques have been developed for predicting stock index, building an efficient stock index forecasting model is still an attractive issue since even the smallest improvement in prediction accuracy can have a positive impact on investments. In this paper, an efficient cerebellar model articulation controller neural network (CAMC NN) is proposed for stock index forecasting. The traditional CAMC NN scheme has been successfully used in robot control due to its advantages of fast learning, reasonable generalization capability and robust noise resistance. But, few studies have been reported in using a CMAC NN scheme for forecasting problems. To improve the forecasting performance, this paper presents an efficient CMAC NN scheme. The proposed CMAC NN scheme employs a high quantization resolution and a large generalization size to reduce generalization error, and uses an efficient and fast hash coding to accelerate many-to-few mappings. The forecasting results and robustness evaluation of the proposed CMAC NN scheme were compared with those of a support vector regression (SVR) and a back-propagation neural network (BPNN). Experimental results from Nikkei 225 and Taiwan Stock Exchange Capitalization Weighted Stock Index (TAIEX) closing indexes show that the performance of the proposed CMAC NN scheme was superior to the SVR and BPNN models.     

Parallel Training of An Improved Neural Network for Text Categorization

This paper studies parallel training of an improved neural network for text categorization. With the explosive growth on the amount of digital information available on the Internet, text categorization problem has become more and more important, especially when millions of mobile devices are now connecting to the Internet. Improved back-propagation neural network (IBPNN) is an efficient approach for classification problems which overcomes the limitations of traditional BPNN. In this paper, we utilize parallel computing to speedup the neural network training process of IBPNN. The parallel IBNPP algorithm for text categorization is implemented on a Sun Cluster with 34 nodes (processors). The communication time and speedup for the parallel IBPNN versus various number of nodes are studied. Experiments are conducted on various data sets and the results show that the parallel IBPNN together with SVD technique achieves fast computational speed and high text categorization correctness.     

Forecasting foreign exchange rates with an improved back-propagation learning algorithm with adaptive smoothing momentum terms

The slow convergence of back-propagation neural network (BPNN) has become a challenge in data-mining and knowledge discovery applications due to the drawbacks of the gradient descent (GD) optimization method, which is widely adopted in BPNN learning. To solve this problem, some standard optimization techniques such as conjugate-gradient and Newton method have been proposed to improve the convergence rate of BP learning algorithm. This paper presents a heuristic method that adds an adaptive smoothing momentum term to original BP learning algorithm to speedup the convergence. In this improved BP learning algorithm, adaptive smoothing technique is used to adjust the momentums of weight updating formula automatically in terms of “3 σ limits theory.” Using the adaptive smoothing momentum terms, the improved BP learning algorithm can make the network training and convergence process faster, and the network’s generalization performance stronger than the standard BP learning algorithm can do. In order to verify the effectiveness of the proposed BP learning algorithm, three typical foreign exchange rates, British pound (GBP), Euro (EUR), and Japanese yen (JPY), are chosen as the forecasting targets for illustration purpose. Experimental results from homogeneous algorithm comparisons reveal that the proposed BP learning algorithm outperforms the other comparable BP algorithms in performance and convergence rate. Furthermore, empirical results from heterogeneous model comparisons also show the effectiveness of the proposed BP learning algorithm.     

主成分和BP神经网络在粮食产量预测中的组合应用

粮食产量的变动受到多种因素的共同影响,各因素之间往往具有十分复杂的非线性关系,传统的预测方法大多无法反映这种变化规律而影响了预测的准确性.BP神经网络模型具有很好的非线性逼近能力,对中国粮食产量能实现比较准确的预测;主成分分析可以对具有模糊关联的变量数据进行降维,其与BP神经网络的组合能优化模型的网络结构,提高预测精度.实证结果表明,组合模型预测结果的精度提高了3%,网络训练的收敛速度和效率也得到不同程度的改善.     

An adaptive local linear optimized radial basis functional neural network model for financial time series prediction

For financial time series, the generation of error bars on the point of prediction is important in order to estimate the corresponding risk. In recent years, optimization techniques-driven artificial intelligence has been used to make time series approaches more systematic and improve forecasting performance. This paper presents a local linear radial basis functional neural network (LLRBFNN) model for classifying finance data from Yahoo Inc. The LLRBFNN model is learned by using the hybrid technique of backpropagation and recursive least square algorithm. The LLRBFNN model uses a local linear model in between the hidden layer and the output layer in contrast to the weights connected from hidden layer to output layer in typical neural network models. The obtained prediction result is compared with multilayer perceptron and radial basis functional neural network with the parameters being trained by gradient descent learning method. The proposed technique provides a lower mean squared error and thus can be considered as superior to other models. The technique is also tested on linear data, i.e., diabetic data, to confirm the validity of the result obtained from the experiment.