Robust high performance reinforcement learning through weighted k-nearest neighbors

The aim of this paper is to present (jointly) a series of robust high performance (award winning) implementations of reinforcement learning algorithms based on temporal-difference learning and weighted k- nearest neighbors for linear function approximation. These algorithms, named kNN‐TD(λ) methods, where rigorously tested at the Second and Third Annual Reinforcement Learning Competitions (RLC2008 and RCL2009) held in Helsinki and Montreal respectively, where the kNN‐TD(λ) method (JAMH team) won in the PolyAthlon 2008 domain, obtained the second place in 2009 and also the second place in the Mountain-Car 2008 domain showing that it is one of the state of the art general purpose reinforcement learning implementations. These algorithms are able to learn quickly, to generalize properly over continuous state spaces and also to be robust to a high degree of environmental noise. Furthermore, we describe a derivation of kNN‐TD(λ) algorithm for problems where the use of continuous actions have clear advantages over the use of fine grained discrete actions: the Ex〈a〉 reinforcement learning algorithm.     

Noisy data elimination using mutual k-nearest neighbor for classification mining

k nearest neighbor (kNN) is an effective and powerful lazy learning algorithm, notwithstanding its easy-to-implement. However, its performance heavily relies on the quality of training data. Due to many complex real-applications, noises coming from various possible sources are often prevalent in large scale databases. How to eliminate anomalies and improve the quality of data is still a challenge. To alleviate this problem, in this paper we propose a new anomaly removal and learning algorithm under the framework of kNN. The primary characteristic of our method is that the evidence of removing anomalies and predicting class labels of unseen instances is mutual nearest neighbors, rather than k nearest neighbors. The advantage is that pseudo nearest neighbors can be identified and will not be taken into account during the prediction process. Consequently, the final learning result is more creditable. An extensive comparative experimental analysis carried out on UCI datasets provided empirical evidence of the effectiveness of the proposed method for enhancing the performance of the k-NN rule.     

A comparison of machine learning methods for cutting parameters prediction in high speed turning process

Support vector machines are arguably one of the most successful methods for data classification, but when using them in regression problems, literature suggests that their performance is no longer state-of-the-art. This paper compares performances of three machine learning methods for the prediction of independent output cutting parameters in a high speed turning process. Observed parameters were the surface roughness (Ra), cutting force \((F_{c})\), and tool lifetime (T). For the modelling, support vector regression (SVR), polynomial (quadratic) regression, and artificial neural network (ANN) were used. In this research, polynomial regression has outperformed SVR and ANN in the case of \(F_{c}\) and Ra prediction, while ANN had the best performance in the case of T, but also the worst performance in the case of \(F_{c}\) and Ra. The study has also shown that in SVR, the polynomial kernel has outperformed linear kernel and RBF kernel. In addition, there was no significant difference in performance between SVR and polynomial regression for prediction of all three output machining parameters.     

Instance-Based Regression by Partitioning Feature Projections

A new instance-based learning method is presented for regression problems with high-dimensional data. As an instance-based approach, the conventional method, KNN, is very popular for classification. Although KNN performs well on classification tasks, it does not perform as well on regression problems. We have developed a new instance-based method, called Regression by Partitioning Feature Projections (RPFP) which is designed to meet the requirement for a lazy method that achieves high levels of accuracy on regression problems. RPFP gives better performance than well-known eager approaches found in machine learning and statistics such as MARS, rule-based regression, and regression tree induction systems. The most important property of RPFP is that it is a projection-based approach that can handle interactions. We show that it outperforms existing eager or lazy approaches on many domains when there are many missing values in the training data.     

Modeling glass-forming ability of bulk metallic glasses using computational intelligent techniques

Modeling the glass-forming ability (GFA) of bulk metallic glasses (BMGs) is one of the hot issues ever since bulk metallic glasses (BMGs) are discovered. It is very useful for the development of new BMGs for various engineering applications, if GFA criterion modeled precisely. In this paper, we have proposed support vector regression (SVR), artificial neural network (ANN), general regression neural network (GRNN), and multiple linear regression (MLR) based computational intelligent (CI) techniques that model the maximum section thickness (D_max) parameter for glass forming alloys. For this study, a reasonable large number of BMGs alloys are collected from the current literature of material science. CI models are developed using three thermal characteristics of glass forming alloys i.e., glass transition temperature (T_g), the onset crystallization temperature (T_x), and liquidus temperature (T_l). The R²-values of GRNN, SVR, ANN, and MLR models are computed to be 0.5779, 0.5606, 0.4879, and 0.2611 for 349 BMGs alloys, respectively. We have investigated that GRNN model is performing better than SVR, ANN, and MLR models. The performance of proposed models is compared to the existing physical modeling and statistical modeling based techniques. In this study, we have investigated that proposed CI approaches are more accurate in modeling the experimental D_max than the conventional GFA criteria of BMGs alloys.     

Parameter-insensitive kernel in extreme learning for non-linear support vector regression

Support vector regression (SVR) is a state-of-the-art method for regression which uses the ε‐sensitive loss and produces sparse models. However, non-linear SVRs are difficult to tune because of the additional kernel parameter. In this paper, a new parameter-insensitive kernel inspired from extreme learning is used for non-linear SVR. Hence, the practitioner has only two meta-parameters to optimise. The proposed approach reduces significantly the computational complexity yet experiments show that it yields performances that are very close from the state-of-the-art. Unlike previous works which rely on Monte-Carlo approximation to estimate the kernel, this work also shows that the proposed kernel has an analytic form which is computationally easier to evaluate.     

A new design of evolutionary hybrid optimization of SVR model in predicting the blast-induced ground vibration

This study aims to identify the suitability of hybridizing the firefly algorithm (FA), genetic algorithm (GA), and particle swarm optimization (PSO) with two well-known data-driven models of support vector regression (SVR) and artificial neural network (ANN) to predict blast-induced ground vibration. Here, these combinations are abbreviated using FA–SVR, PSO–SVR, GA–SVR, FA–ANN, PSO–ANN, and GA–ANN models. In addition, a modified FA (MFA) combined with SVR model is also proposed in this study, namely, MFA–SVR. The feasibility of the proposed models is examined using a case study, located in Johor, Malaysia. Then, to provide an objective assessment of performances of the predictive models, their results were compared based on several well known and popular statistical criteria. According to the results, the MFA–SVR with the coefficient of determination (R²) of 0.984 and root mean square error (RMSE) of 0.614 was more accurate model to predict PPV than the PSO–SVR with R² = 0.977 and RMSE = 0.725, the FA–SVR with R² = 0.964 and RMSE = 0.923, the GA–SVR with R² = 0.957 and RMSE = 1.016, the GA–ANN with R² = 0.936 and RMSE = 1.252, the FA–ANN with R² = 0.925 and RMSE = 1.368, and the PSO–ANN with R² = 0.924 and RMSE = 1.366. Consequently, the MFA–SVR model can be sufficiently employed in estimating the ground vibration, and has the capacity to generalize.

     

Prediction of rainfall time series using modular soft computingmethods

In this paper, several soft computing approaches were employed for rainfall prediction. Two aspects were considered to improve the accuracy of rainfall prediction: (1)carrying out a data-preprocessing procedure and (2)adopting a modular modeling method. The proposed preprocessing techniques included moving average (MA) and singular spectrum analysis (SSA). The modular models were composed of local support vectors regression (SVR) models or/and local artificial neural networks (ANN) models. In the process of rainfall forecasting, the ANN was first used to choose data-preprocessing method from MA and SSA. Modular models involved preprocessing the training data into three crisp subsets (low, medium and high levels) according to the magnitudes of the training data, and finally two SVRs were performed in the medium and high-level subsets whereas ANN or SVR was involved in training and predicting the low-level subset. For daily rainfall record, the low-level subset tended to be modeled by the ANN because it was overwhelming in the training data, which is based on the fact that the ANN is very efficient in training large-size samples due to its parallel information processing configuration. Four rainfall time series consisting of two monthly rainfalls and two daily rainfalls from different regions were utilized to evaluate modular models at 1-day, 2-day, and 3-day lead-time with the persistence method and the global ANN as benchmarks. Results showed that the MA was superior to the SSA when they were coupled with the ANN. Comparison results indicated that modular models (referred to as ANN-SVR for daily rainfall simulations and MSVR for monthly rainfall simulations) outperformed other models. The ANN-MA also displayed considerable accuracy in rainfall forecasts compared with the benchmark.     

Regularized versus non-regularized neural network model for prediction of saturated soil-water content on weathered granite soil formation

Modeling unsaturated water flow in soil requires knowledge of the hydraulic properties of soil. However, correlation between soil hydraulic properties such as the relationship between saturated soil-water content θ _s and saturated soil hydraulic conductivity k _s as function of soil depth is in stochastic pattern. On the other hand, soil-water profile process is also believed to be highly non-linear, time varying, spatially distributed, and not easily described by simple models. In this study, the potential of implementing artificial neural network (ANN) model was proposed and investigated to map the soil-water profile in terms of k _s and θ _s with respect to the soil depth d. A regularized neural network (NN) model is developed to overcome the drawbacks of conventional prediction techniques. The use of regularized NN advantaged avoid over-fitting of training data, which was observed as a limitation of classical ANN models. Site experimental data sets on the hydraulic properties of weathered granite soils were collected. These data sets include the observed values of saturated and unsaturated hydraulic conductivities, saturated water contents, and retention curves. The proposed ANN model was examined utilizing 49 records of data collected from field experiments. The results showed that the regularized ANN model has the ability to detect and extract the stochastic behavior of saturated soil-water content with relatively high accuracy.     

A semi-supervised regression model for mixed numerical and categorical variables

In this paper, we develop a semi-supervised regression algorithm to analyze data sets which contain both categorical and numerical attributes. This algorithm partitions the data sets into several clusters and at the same time fits a multivariate regression model to each cluster. This framework allows one to incorporate both multivariate regression models for numerical variables (supervised learning methods) and k-mode clustering algorithms for categorical variables (unsupervised learning methods). The estimates of regression models and k-mode parameters can be obtained simultaneously by minimizing a function which is the weighted sum of the least-square errors in the multivariate regression models and the dissimilarity measures among the categorical variables. Both synthetic and real data sets are presented to demonstrate the effectiveness of the proposed method.     

An efficient CMAC neural network for stock index forecasting

Stock index forecasting is one of the major activities of financial firms and private investors in making investment decisions. Although many techniques have been developed for predicting stock index, building an efficient stock index forecasting model is still an attractive issue since even the smallest improvement in prediction accuracy can have a positive impact on investments. In this paper, an efficient cerebellar model articulation controller neural network (CAMC NN) is proposed for stock index forecasting. The traditional CAMC NN scheme has been successfully used in robot control due to its advantages of fast learning, reasonable generalization capability and robust noise resistance. But, few studies have been reported in using a CMAC NN scheme for forecasting problems. To improve the forecasting performance, this paper presents an efficient CMAC NN scheme. The proposed CMAC NN scheme employs a high quantization resolution and a large generalization size to reduce generalization error, and uses an efficient and fast hash coding to accelerate many-to-few mappings. The forecasting results and robustness evaluation of the proposed CMAC NN scheme were compared with those of a support vector regression (SVR) and a back-propagation neural network (BPNN). Experimental results from Nikkei 225 and Taiwan Stock Exchange Capitalization Weighted Stock Index (TAIEX) closing indexes show that the performance of the proposed CMAC NN scheme was superior to the SVR and BPNN models.     

基于支持向量机回归的麻醉药毒性的QSPR研究

采用支持向量机回归(SVR)方法研究了39个麻醉药毒性的定量构效关系,基于留一法交叉验证的结果,模型的相关系数为0.970。结果表明,所建SVR模型的精度高于逆传播人工神经网络(BPANN)、多元线性回归(MLR)和偏最小二乘法(PLS)所得的结果。     

Nearest-neighbor classifier motivated marginal discriminant projections for face recognition

Marginal Fisher analysis (MFA) is a representative margin-based learning algorithm for face recognition. A major problem in MFA is how to select appropriate parameters, k ₁ and k ₂, to construct the respective intrinsic and penalty graphs. In this paper, we propose a novel method called nearest-neighbor (NN) classifier motivated marginal discriminant projections (NN-MDP). Motivated by the NN classifier, NN-MDP seeks a few projection vectors to prevent data samples from being wrongly categorized. Like MFA, NN-MDP can characterize the compactness and separability of samples simultaneously. Moreover, in contrast to MFA, NN-MDP can actively construct the intrinsic graph and penalty graph without unknown parameters. Experimental results on the ORL, Yale, and FERET face databases show that NN-MDP not only avoids the intractability, and high expense of neighborhood parameter selection, but is also more applicable to face recognition with NN classifier than other methods.     

Locally linear reconstruction for instance-based learning

Instance-based learning (IBL), so called memory-based reasoning (MBR), is a commonly used non-parametric learning algorithm. k-nearest neighbor (k-NN) learning is the most popular realization of IBL. Due to its usability and adaptability, k-NN has been successfully applied to a wide range of applications. However, in practice, one has to set important model parameters only empirically: the number of neighbors (k) and weights to those neighbors. In this paper, we propose structured ways to set these parameters, based on locally linear reconstruction (LLR). We then employed sequential minimal optimization (SMO) for solving quadratic programming step involved in LLR for classification to reduce the computational complexity. Experimental results from 11 classification and eight regression tasks were promising enough to merit further investigation: not only did LLR outperform the conventional weight allocation methods without much additional computational cost, but also LLR was found to be robust to the change of k.     

Forest attribute imputation using machine-learning methods and ASTER data: comparison of k-NN,SVR and random forest regression algorithms

In a small case study of mixed hardwood Hyrcanian forests of Iran, three non-parametric methods, namely k-nearest neighbour (k-NN), support vector machine regression (SVR) and tree regression based on random forest (RF), were used in plot-level estimation of volume/ha, basal area/ha and stems/ha using field inventory and Advanced Spaceborne Thermal Emission and Reflection Radiometer (ASTER) data. Relevant pre-processing and processing steps were applied to the ASTER data for geometric and atmospheric correction and for enhancing quantitative forest parameters. After collecting terrestrial information on trees in the 101 samples, the volume, basal area and tree number per hectare were calculated in each plot. In the k-NN implementation using different distance measures and k, the cross-validation method was used to find the best distance measure and optimal k. In SVR, the best regularized parameters of four kernel types were obtained using leave-one-out cross-validation. RF was implemented using a bootstrap learning method with regularized parameters for decision tree model and stopping. The validity of performances was examined using unused test samples by absolute and relative root mean square error (RMSE) and bias metrics. In volume/ha estimation, the results showed that all the three algorithms had similar performances. However, SVR and RF produced better results than k-NN with relative RMSE values of 28.54, 25.86 and 26.86 (m³ ha^–1), respectively, using k-NN, SVR and RF algorithms, but RF could generate unbiased estimation. In basal area/ha and stems/ha estimation, the implementation results of RF showed that RF was slightly superior in relative RMSE (18.39, 20.64) to SVR (19.35, 22.09) and k-NN (20.20, 21.53), but k-NN could generate unbiased estimation compared with the other two algorithms used.     

SVOIS: Support Vector Oriented Instance Selection for text classification

Automatic text classification is usually based on models constructed through learning from training examples. However, as the size of text document repositories grows rapidly, the storage requirements and computational cost of model learning is becoming ever higher. Instance selection is one solution to overcoming this limitation. The aim is to reduce the amount of data by filtering out noisy data from a given training dataset. A number of instance selection algorithms have been proposed in the literature, such as ENN, IB3, ICF, and DROP3. However, all of these methods have been developed for the k-nearest neighbor (k-NN) classifier. In addition, their performance has not been examined over the text classification domain where the dimensionality of the dataset is usually very high. The support vector machines (SVM) are core text classification techniques. In this study, a novel instance selection method, called Support Vector Oriented Instance Selection (SVOIS), is proposed. First of all, a regression plane in the original feature space is identified by utilizing a threshold distance between the given training instances and their class centers. Then, another threshold distance, between the identified data (forming the regression plane) and the regression plane, is used to decide on the support vectors for the selected instances. The experimental results based on the TechTC-100 dataset show the superior performance of SVOIS over other state-of-the-art algorithms. In particular, using SVOIS to select text documents allows the k-NN and SVM classifiers perform better than without instance selection.     

Performance evaluation of hybrid FFA-ANFIS and GA-ANFIS models to predict particle size distribution of a muck-pile after blasting

Accurately predicting the particle size distribution of a muck-pile after blasting is always an important subject for mining industry. Adaptive neuro-fuzzy inference system (ANFIS) has emerged as a synergic intelligent system. The main contribution of this paper is to optimize the premise and consequent parameters of ANFIS by firefly algorithm (FFA) and genetic algorithm (GA). To the best of our knowledge, no research has been published that assesses FFA and GA with ANFIS for fragmentation prediction and no research has tested the efficiency of these models to predict the fragmentation in different time scales as of yet. To show the effectiveness of the proposed ANFIS-FFA and ANFIS-GA models, their modelling accuracy has been compared with ANFIS, support vector regression (SVR) and artificial neural network (ANN). Intelligence predictions of fragmentation by ANFIS-FFA, ANFIS-GA, ANFIS, SVR and ANN are compared with observed values of fragmentation available in 88 blasting event of two quarry mines, Iran. According to the results, both ANFIS-FFA and ANFIS-GA prediction models performed satisfactorily; however, the lowest root mean square error (RMSE) and the highest correlation of determination (R²) values were obtained from ANFIS-GA model. The values of R² and RMSE obtained from ANFIS-GA, ANFIS-FFA, ANFIS, SVR and ANN models were equal to (0.989, 0.974), (0.981, 1.249), (0.956, 1.591), (0.924, 2.016) and (0.948, 2.554), respectively. Consequently, the proposed ANFIS-GA model has the potential to be used for predicting aims on other fields.

     

Simulation and prediction of hydrological processes based on firefly algorithm with deep learning and support vector for regression

ABSTRACT

Hydrological processes are hard to accurately simulate and predict because of various natural and human influences. In order to improve the simulation and prediction accuracy of the hydrological process, the firefly algorithm with deep learning (DLFA) was used in this study to optimise the parameters of support vector for regression (SVR) automatically, and a prediction model was established based on DLFA and SVR. The hydrological process of Huangfuchuan in Fugu County, Shanxi Province was taken as the research object to verify the performance of the prediction model, and the results were compared with those by the other six prediction models. The experimental results showed that the proposed prediction model achieved improved prediction performance compared with the other six models.     

Using kNN model for automatic text categorization

An investigation is conducted on two well-known similarity-based learning approaches to text categorization: the k-nearest neighbors (kNN) classifier and the Rocchio classifier. After identifying the weakness and strength of each technique, a new classifier called the kNN model-based classifier (kNN Model) is proposed. It combines the strength of both kNN and Rocchio. A text categorization prototype, which implements kNN Model along with kNN and Rocchio, is described. An experimental evaluation of different methods is carried out on two common document corpora: the 20-newsgroup collection and the ModApte version of the Reuters-21578 collection of news stories. The experimental results show that the proposed kNN model-based method outperforms the kNN and Rocchio classifiers, and is therefore a good alternative for kNN and Rocchio in some application areas. This work was partly supported by the European Commission project ICONS, project no. IST-2001-32429.     

Effective protocols for kNN search on broadcast multi-dimensional index trees

In a wireless mobile environment, data broadcasting provides an efficient way to disseminate data. Via data broadcasting, a server can provide location-based services to a large client population in a wireless environment. Among different location-based services, the k nearest neighbors (kNN) search is important and is used to find the k closest objects to a given point. However, the kNN search in a broadcast environment is particularly challenging due to the sequential access to the data on a broadcast channel. We propose efficient protocols for the kNN search on a broadcast R-tree, which is a popular multi-dimensional index tree, in a wireless broadcast environment in terms of latency and tuning time as well as memory usage. We investigate how a server schedules the broadcast and provide the corresponding kNN search algorithms at the mobile clients. One of our kNN search protocols further allows a kNN search to start at an arbitrary time instance and it can skip the waiting time for the beginning of a broadcast cycle, thereby reducing the latency. The experimental results validate that our mechanisms achieve the objectives.