A hybrid neural network model for noisy data regression

A hybrid neural network model, based on the fusion of fuzzy adaptive resonance theory (FA ART) and the general regression neural network (GRNN), is proposed in this paper. Both FA and the GRNN are incremental learning systems and are very fast in network training. The proposed hybrid model, denoted as GRNNFA, is able to retain these advantages and, at the same time, to reduce the computational requirements in calculating and storing information of the kernels. A clustering version of the GRNN is designed with data compression by FA for noise removal. An adaptive gradient-based kernel width optimization algorithm has also been devised. Convergence of the gradient descent algorithm can be accelerated by the geometric incremental growth of the updating factor. A series of experiments with four benchmark datasets have been conducted to assess and compare effectiveness of GRNNFA with other approaches. The GRNNFA model is also employed in a novel application task for predicting the evacuation time of patrons at typical karaoke centers in Hong Kong in the event of fire. The results positively demonstrate the applicability of GRNNFA in noisy data regression problems.     

Prediction measurement with mean acceptable error for proper inconsistency in noisy weldability prediction data

Due to the complex nature of the welding process, the data used to construct prediction models often contain a significant amount of inconsistency. In general, this type of inconsistent data is treated as noise in the literature. However, for the weldability prediction, the inconsistency, which we describe as proper-inconsistency, may not be eliminated since the inconsistent data can help extract additional information about the process. This paper discusses that, in the presence of proper-inconsistency, it is inappropriate to perform the same approach generally employed with machine learning algorithms, in terms of the model construction and prediction measurement. Due to the numerical characteristics of proper-inconsistency, it is likely to achieve vague prediction results from the prediction model with the traditional prediction performance measures. In this paper, we propose a new prediction performance measure called mean acceptable error (MACE), which measures the performance of prediction models constructed with the presence of proper-inconsistency. This paper presents experimental results with real weldability prediction data, and we examine the prediction performance of k-nearest neighbor (kNN) and generalized regression neural network (GRNN) measured by MACE and the different characteristics of data in relation to MACE, kNN, and GRNN. The results indicate that using a smaller k on properly-inconsistent data increases the prediction performance measured by MACE. Also, the prediction performance on the correct data increases, while the effect of properly-inconsistent data decreases with the measurement of MACE.     

基于混合改进GSO与GRNN并行集成学习模型

针对萤火虫群优化算法(GSO)不稳定、收敛速度较慢与收敛精度较低等问题和广义回归神经网络(GRNN)的网络结构导致预测误差的特性,提出基于混合改进萤火虫群算法与广义回归神经网络并行集成学习模型,应用于雾霾预测.首先构建融合多种搜索策略的混合改进萤火虫群优化算法(HIGSO),并使用标准测试函数验证算法性能.然后结合HIGSO与引入扰动因子的GRNN模型,建立并行集成学习模型,并通过UCI标准数据集验证模型的有效性与可行性.最后将模型应用于北京、上海和广州地区的雾霾预测,进一步验证模型在雾霾预测中的性能.     

时变过程在线辨识的即时递推核学习方法研究

为了及时跟踪非线性化工过程的时变特性, 提出即时递推核学习 (Kernel learning, KL)的在线辨识方法. 针对待预测的新样本点, 采用即时学习 (Just-in-time kernel learning, JITL)策略, 通过构造累积相似度因子, 选择与其相似的样本集建立核学习辨识模型. 为避免传统即时学习对每个待预测点都重新建模的繁琐, 利用两个临近时刻相似样本集的异同点, 采用递推方法有效添加新样本, 并删减旧模型的样本, 以快速建立新即时模型. 通过一时变连续搅拌釜式反应过程的在线辨识, 表明了所提出方法在保证计算效率的同时, 较传统递推核学习方法提高了辨识的准确程度, 能更好地辨识时变过程.     

Local learning regularization networks for localized regression

Local learning algorithms use a neighborhood of training data close to a given testing query point in order to learn the local parameters and create on-the-fly a local model specifically designed for this query point. The local approach delivers breakthrough performance in many application domains. This paper considers local learning versions of regularization networks (RN) and investigates several options for improving their online prediction performance, both in accuracy and speed. First, we exploit the interplay between locally optimized and globally optimized hyper-parameters (regularization parameter and kernel width) each new predictor needs to optimize online. There is a substantial reduction of the operation cost in the case we use two globally optimized hyper-parameters that are common to all local models. We also demonstrate that this global optimization of the two hyper-parameters produces more accurate models than the other cases that locally optimize online either the regularization parameter, or the kernel width, or both. Then by comparing Eigenvalue decomposition (EVD) with Cholesky decomposition specifically for the local learning training and testing phases, we also reveal that the Cholesky-based implementations are faster that their EVD counterparts for all the training cases. While EVD is suitable for validating cost-effectively several regularization parameters, Cholesky should be preferred when validating several neighborhood sizes (the number of k-nearest neighbors) as well as when the local network operates online. Then, we exploit parallelism in a multi-core system for these local computations demonstrating that the execution times are further reduced. Finally, although the use of pre-computed stored local models instead of the online learning local models is even faster, this option deteriorates the performance. Apparently, there is a substantial gain in waiting for a testing point to arrive before building a local model, and hence the online local learning RNs are more accurate than their pre-computed stored local models. To support all these findings, we also present extensive experimental results and comparisons on several benchmark datasets.

     

组合核相关向量机的软件可靠性预测模型

可靠性作为衡量软件质量的一种重要特性,对软件管理具有重要的意义。针对单一核函数的缺陷,提出一种组合核函数相关向量机的软件可靠性预测模型。首先对当前软件可靠性研究现状进行分析,然后采用组合核函数相关向量机对训练集进行学习和建模,最后通过具体实例对模型的预测性能进行分析。结果表明,本模型获得了理想的软件可靠性预测结果,且其预测性能要优于单一核函数模型,在软件可靠性预测中有重要的应用价值。     

面向非平衡类问题的k近邻分类算法

针对k近邻（kNN）方法不能很好地解决非平衡类问题,提出一种新的面向非平衡类问题的k近邻分类算法。与传统k近邻方法不同,在学习阶段,该算法首先使用划分算法（如K-Means）将多数类数据集划分为多个簇,然后将每个簇与少数类数据集合并成一个新的训练集用于训练一个k近邻模型,即该算法构建了一个包含多个k近邻模型的分类器库。在预测阶段,使用划分算法（如K-Means）从分类器库中选择一个模型用于预测样本类别。通过这种方法,提出的算法有效地保证了k近邻模型既能有效发现数据局部特征,又能充分考虑数据的非平衡性对分类器性能的影响。另外,该算法也有效地提升了k近邻的预测效率。为了进一步提高该算法的性能,将合成少数类过抽样技术（SMOTE）应用到该算法中。KEEL数据集上的实验结果表明,即使对采用随机划分策略划分的多数类数据集,所提算法也能有效地提高k近邻方法在评价指标recall、g-mean、f-measure和AUC上的泛化性能;另外,过抽样技术能进一步提高该算法在非平衡类问题上的性能,并明显优于其他高级非平衡类处理方法。     

Online Multiple Kernel Classification

Although both online learning and kernel learning have been studied extensively in machine learning, there is limited effort in addressing the intersecting research problems of these two important topics. As an attempt to fill the gap, we address a new research problem, termed Online Multiple Kernel Classification (OMKC), which learns a kernel-based prediction function by selecting a subset of predefined kernel functions in an online learning fashion. OMKC is in general more challenging than typical online learning because both the kernel classifiers and the subset of selected kernels are unknown, and more importantly the solutions to the kernel classifiers and their combination weights are correlated. The proposed algorithms are based on the fusion of two online learning algorithms, i.e., the Perceptron algorithm that learns a classifier for a given kernel, and the Hedge algorithm that combines classifiers by linear weights. We develop stochastic selection strategies that randomly select a subset of kernels for combination and model updating, thus improving the learning efficiency. Our empirical study with 15 data sets shows promising performance of the proposed algorithms for OMKC in both learning efficiency and prediction accuracy.     

Generalized regression neural networks in time-varying environment

The current state of knowledge regarding nonstationary processes is significantly poorer then in the case of stationary signals. In many applications, signals are treated as stationary only because in this way it is easier to analyze them; in fact, they are nonstationary. Nonstationary processes are undoubtedly more difficult to analyze and their diversity makes application of universal tools impossible. In this paper we propose a new class of generalized regression neural networks working in nonstationary environment. The generalized regession neural networks (GRNN) studied in this paper are able to follow changes of the best model, i.e., time-varying regression functions. The novelty is summarized as follows: 1) We present adaptive GRNN tracking time-varying regression functions. 2) We prove convergence of the GRNN based on general learning theorems presented in Section IV. 3) We design in detail special GRNN based on the Parzen and orthogonal series kernels. In each case we precise conditions ensuring convergence of the GRNN to the best models described by regression function. 4) We investigate speed of convergence of the GRNN and compare performance of specific structures based on the Parzen kernel and orthogonal series kernel. 5) We study various nonstationarities (multiplicative, additive, "scale change," "movable argument") and design in each case the GRNN based on the Parzen kernel and orthogonal series kernel.     

Modelling of diesel engine performance using advanced machine learning methods under scarce and exponential data set

Traditional methods on creating diesel engine models include the analytical methods like multi-zone models and the intelligent based models like artificial neural network (ANN) based models. However, those analytical models require excessive assumptions while those ANN models have many drawbacks such as the tendency to overfitting and the difficulties to determine the optimal network structure. In this paper, several emerging advanced machine learning techniques, including least squares support vector machine (LS-SVM), relevance vector machine (RVM), basic extreme learning machine (ELM) and kernel based ELM, are newly applied to the modelling of diesel engine performance. Experiments were carried out to collect sample data for model training and verification. Limited by the experiment conditions, only 24 sample data sets were acquired, resulting in data scarcity. Six-fold cross-validation is therefore adopted to address this issue. Some of the sample data are also found to suffer from the problem of data exponentiality, where the engine performance output grows up exponentially along the engine speed and engine torque. This seriously deteriorates the prediction accuracy. Thus, logarithmic transformation of dependent variables is utilized to pre-process the data. Besides, a hybrid of leave-one-out cross-validation and Bayesian inference is, for the first time, proposed for the selection of hyperparameters of kernel based ELM. A comparison among the advanced machine learning techniques, along with two traditional types of ANN models, namely back propagation neural network (BPNN) and radial basis function neural network (RBFNN), is conducted. The model evaluation is made based on the time complexity, space complexity, and prediction accuracy. The evaluation results show that kernel based ELM with the logarithmic transformation and hybrid inference is far better than basic ELM, LS-SVM, RVM, BPNN and RBFNN, in terms of prediction accuracy and training time.     

An improved kernel-based incremental extreme learning machine with fixed budget for nonstationary time series prediction

In order to curb the model expansion of the kernel learning methods and adapt the nonlinear dynamics in the process of the nonstationary time series online prediction, a new online sequential learning algorithm with sparse update and adaptive regularization scheme is proposed based on kernel-based incremental extreme learning machine (KB-IELM). For online sparsification, a new method is presented to select sparse dictionary based on the instantaneous information measure. This method utilizes a pruning strategy, which can prune the least “significant” centers, and preserves the important ones by online minimizing the redundancy of dictionary. For adaptive regularization scheme, a new objective function is constructed based on basic ELM model. New model has different structural risks in different nonlinear regions. At each training step, new added sample could be assigned optimal regularization factor by optimization procedure. Performance comparisons of the proposed method with other existing online sequential learning methods are presented using artificial and real-word nonstationary time series data. The results indicate that the proposed method can achieve higher prediction accuracy, better generalization performance and stability.

     

A hybrid model through the fusion of type-2 fuzzy logic systems and extreme learning machines for modelling permeability prediction

Extreme learning machines (ELM), as a learning tool, have gained popularity due to its unique characteristics and performance. However, the generalisation capability of ELM often depends on the nature of the dataset, particularly on whether uncertainty is present in the dataset or not. In order to reduce the effects of uncertainties in ELM prediction and improve its generalisation ability, this paper proposes a hybrid system through a combination of type-2 fuzzy logic systems (type-2 FLS) and ELM; thereafter the hybrid system was applied to model permeability of carbonate reservoir. Type-2 FLS has been chosen to be a precursor to ELM in order to better handle uncertainties existing in datasets beyond the capability of type-1 fuzzy logic systems. The type-2 FLS is used to first handle uncertainties in reservoir data so that its final output is then passed to the ELM for training and then final prediction is done using the unseen testing dataset. Comparative studies have been carried out to compare the performance of the proposed T2-ELM hybrid system with each of the constituent type-2 FLS and ELM, and also artificial neural network (ANN) and support Vector machines (SVM) using five different industrial reservoir data. Empirical results show that the proposed T2-ELM hybrid system outperformed each of type-2 FLS and ELM, as the two constituent models, in all cases, with the improvement made to the ELM performance far higher against that of type-2 FLS that had a closer performance to the hybrid since it is already noted for being able to model uncertainties. The proposed hybrid also outperformed ANN and SVM models considered.     

多稀疏回声状态网络预测模型

针对单回声状态网络难以充分描述数据信息的问题,提出多稀疏回声状态网络预测模型.通过对相关回声状态网络的组合权值及由相关样本得到的基函数的权值同时进行学习,获得优化的多个稀疏回声状态网络组合模型.所提模型不同于双稀疏相关向量机等多核学习模型,它不需要选择特定的核函数及相应的核参数.因此,该模型不但能更好的描述数据信息,避免了双稀疏相关向量机及其他多核学习中核函数及其参数不易选择的问题.同时,所提模型不需要采用交叉验证的方式确定回声状态网络的谱半径和稀疏度,只需确定相应的区间.本文通过两组标杆数据和一组实际数据仿真实验,与传统回声状态网络方法相比,验证了所提模型具有更好的预测性能.     

A new model-free adaptive sliding controller for active suspension system

Active suspension systems are designed to provide better ride comfort and handling capability in the automotive industry. Since the active suspension system has nonlinear and time-varying characteristics, it is difficult to establish an accurate dynamic model for designing a model-based controller. Here, a functional approximation (FA) based adaptive sliding controller with fuzzy compensation is proposed for an active suspension system. The FA technique is employed to represent the unknown functions, which releases the model-based requirement of the sliding mode control. In addition, a fuzzy control scheme with online learning ability is employed to compensate for the modeling error of the FA with finite number of terms for reducing the implementation difficulty. To guarantee the control system stability, the update laws of the coefficients in the approximation function and the fuzzy tuning parameters are derived from the Lyapunov theorem. The proposed controller is employed on a quarter-car active suspension system. The simulation results and experimental results show that the proposed controller can suppress the oscillation amplitude of the sprung mass effectively. To evaluate the performance improvement of inducing a fuzzy compensator in this FA adaptive controller, the dynamic responses of the proposed hybrid controller are compared with those of FA-based adaptive sliding controller only.     

面向短时地铁客流量预测的混合深度学习模型

城市交通客流量精准预测是智能交通系统的重要环节,是有效管控交通、规划最佳出行线路的关键。目前城市交通客流量短时预测研究主要集中在利用深度学习模型进行时空特征的提取,忽略了对模型优化的研究。针对短时地铁客流量预测存在的问题,提出一种混合深度学习模型ResGRU_Metro,将卷积神经网络、残差单元和门控循环单元相结合,捕获流量数据的时空特征。针对深度学习模型常用的损失函数难以对交通客流量峰值进行精准预测的问题,引入面向短时交通流量预测的加权平方误差,根据交通客流量的大小为预测误差赋予不同权重,并加大对交通客流量峰值处误差的惩罚,使神经网络在反向传播时更加关注峰值处的预测和误差,从而提升交通客流量峰值的预测精度。此外,通过耦合天气、空气质量等外部因子,改善模型的整体预测性能,增强模型的稳定性。实验结果表明,相比LR、PSVR、CNN等典型的预测模型,ResGRU_Metro模型有更高的预测精度,能够准确预测交通客流量的峰值。     

Data preprocessing using hybrid general regression neural networks and particle swarm optimization for remote terminal units

Data corruption in SCADA systems refers to errors that occur during acquisition, processing, or transmission, introducing unintended changes to the original data. In SCADA-based power systems, the data gathered by remote terminal units (RTUs) is subject to data corruption due to noise interference or lack of calibration. In this study, an effective approach based on the fusion of the general regression neural network (GRNN) and the particle swarm optimization (PSO) technique is employed to deal with errors in RTU data. The proposed hybrid model, denoted as GRNN-PSO, is able to handle noisy data in a fast speed, which makes it feasible for practical applications. Experimental results show the GRNN-PSO model has better performance in removing the unintended changes to the original data compared with existing methods.     

A hybrid model of bald eagle search and relevance vector machine for dam safety monitoring using long-term temperature

Accurate simulation of temperature effect is a major challenge for computational (intelligent) prediction models used for monitoring health of high concrete dams, especially in regions with long freezing periods and distinct seasons, occasional extreme weather. A Hydrostatic-Temperature _long-term-Time (HT_LT) model was proposed for better temperature effect simulation using long-term measured environment temperatures and explored the influence of temperatures data sets of different time lengths on dam displacement prediction accuracy with the traditional Hydrostatic-Season-Time model as control. The Bald Eagle Search algorithm was coupled with the Relevance Vector Machine to form a novel hybrid model (BES-RVM) for predicting concrete gravity dam displacement response and comparison of nonlinear mapping capability between different kernel functions was conducted. Further optimized by Successive Projections Algorithm (SPA) for feature selection, sensitive features were selected from the proposed HT_LT variable sets to improve the prediction model’s accuracy. The prediction model was experimented on a real concrete dam with results showing that the BES-RVM model gave excellent prediction performance. Using the HT_LT variable sets significantly reduced the prediction errors and the best performed result came from variables of the two-year long temperatures data. The SPA optimized BES-RVM model largely brought down the dimension and the collinearity of the input variables and further improved the prediction performance.     

Soft computing methods applied to train station parking in urban rail transit

This paper presents three models - a linear model, a generalized regression neural network (GRNN) and an adaptive network based fuzzy inference system (ANFIS) - to estimate the train station parking (TSP) error in urban rail transit. We also develop some statistical indices to evaluate the reliability of controlling parking errors in a certain range. By comparing modeling errors, the subtractive clustering method other than grid partition method is chosen to generate an initial fuzzy system for ANFIS. Then, the collected TSP data from two railway stations are employed to identify the parameters of the proposed three models. The three models can make the average parking errors under an acceptable error, and tuning the parameters of the models is effective in dynamically reducing parking errors. Experiments in two stations indicate that, among the three models, (1) the linear model ranks the third in training and the second in testing, nevertheless, it can meet the required reliability for two stations, (2) the GRNN based model achieves the best performance in training, but the poorest one in testing due to overfitting, resulting in failing to meet the required reliability for the two stations, (3) the ANFIS based model obtains better performance than model 1 both in training and testing. After analyzing parking error characteristics and developing a parking strategy, finally, we confirm the effectiveness of the proposed ANFIS model in the real-world application.     

Predicting lightning outages of transmission lines using generalized regression neural network

Lightning is the major cause of transmission line outages, which can result in large area blackouts of power systems. One effective method to prevent catastrophic consequences is to predict lightning outages before they occur. The abundance of recorded lightning and lightning outage data in power system makes it possible to predict lightning outages of transmission lines. This paper proposes an artificially intelligent algorithm using general regression neural networks (GRNN) to predict lightning outages of transmission lines. First, the data that can be obtained from the operation and management system of a power company are analyzed, and the features that can be used as input parameters of GRNN are extracted. The prediction model based on GRNN is then built to perform lightning outage prediction. Finally, the effectiveness of the proposed method is validated by comparing it with (Back Propagation) BP and (Radial Basis Function) RBF neural networks using actual lightning data and lightning outage data. The simulation results show that the proposed method provides much better prediction performance.     

基于改进极端学习机的网络流量预测

为了提高网络流量的预测精度,针对极端学习机的训练样本选择问题,提出一种改进极端学习机的网络流量预测模型（IELM）。根据最优延迟时间和嵌入维数对网络流量重构,建立网络学习样本,将学习样本输入到改进极端学习机进行训练,随新样本加入而逐步求解网络的权值,以提高学习速度,引入cholesky分解方法提高模型的泛化能力,采用具体网络流量数据进行了仿真测试。结果表明,IELM不仅可以获得较传统网络流量预测模型更高的精度,并且大幅度减少了计算时间,提高了建模效率,可以较好地满足网络流量预测要求。