Fuzzy rule-based support vector regression system

In this paper,we design a fuzzy rule-based support vector regression system.The proposed system utilizes the advantages of fuzzy model and support vector regression to extract support vectors to generate fuzzy if-then rules from the training data set.Based on the first-order linear Tagaki-Sugeno (TS) model,the structure of rules is identified by the support vector regression and then the consequent parameters of rules are tuned by the global least squares method.Our model is applied to the real world regression task.The simulation results gives promising performances in terms of a set of fuzzy rules,which can be easily interpreted by humans.     

Fuzzy nile-based support vector regression system

In this paper, we design a fuzzy rule-based support vector regression system. The proposed system utilizes the advantages of fuzzy model and support vector regression to extract support vectors to generate fuzzy if-then rules from the training data set. Based on the first-order hnear Tagaki-Sugeno （TS） model, the structure of rules is identified by the support vector regression and then the consequent parameters of rules are tuned by the global least squares method. Our model is applied to the real world regression task. The simulation results gives promising performances in terms of a set of fuzzy hales, which can be easily interpreted by humans.     

Prediction of silicon content in hot metal using support vector regression based on chaos particle swarm optimization

The prediction of silicon content in hot metal has been a major study subject as one of the most important means for the monitoring state in ferrous metallurgy industry. A prediction model of silicon content is established based on the support vector regression (SVR) whose optimal parameters are selected by chaos particle swarm optimization. The data of the model are collected from No. 3 BF in Panzhihua Iron and Steel Group Co. of China. The results show that the proposed prediction model has better prediction results than neural network trained by chaos particle swarm optimization and least squares support vector regression, the percentage of samples whose absolute prediction errors are less than 0.03 when predicting silicon content by the proposed model is higher than 90%, it indicates that the prediction precision can meet the requirement of practical production.     

Cycle-breaking acceleration for support vector regression

Support vector regression (SVR) is a powerful tool in modeling and prediction tasks with widespread application in many areas. The most representative algorithms to train SVR models are Shevade et al.'s Modification 2 and Lin's WSS1 and WSS2 methods in the LIBSVM library. Both are variants of standard SMO in which the updating pairs selected are those that most violate the Karush-Kuhn-Tucker optimality conditions, to which LIBSVM adds a heuristic to improve the decrease in the objective function. In this paper, and after presenting a simple derivation of the updating procedure based on a greedy maximization of the gain in the objective function, we show how cycle-breaking techniques that accelerate the convergence of support vector machines (SVM) in classification can also be applied under this framework, resulting in significantly improved training times for SVR.     

Rotation-invariant texture image retrieval using particle swarm optimization and support vector regression

This paper presents a novel rotation-invariant texture image retrieval using particle swarm optimization (PSO) and support vector regression (SVR), which is called the RTIRPS method. It respectively employs log-polar mapping (LPM) combined with fast Fourier transformation (FFT), Gabor filter, and Zernike moment to extract three kinds of rotation-invariant features from gray-level images. Subsequently, the PSO algorithm is utilized to optimize the RTIRPS method. Experimental results demonstrate that the RTIRPS method can achieve satisfying results and outperform the existing well-known rotation-invariant image retrieval methods under considerations here. Also, in order to reduce calculation complexity for image feature matching, the RTIRPS method employs the SVR to construct an efficient scheme for the image retrieval.     

Comprehensive learning particle swarm optimization based memetic algorithm for model selection in short-term load forecasting using support vector regression

BackgroundShort-term load forecasting is an important issue that has been widely explored and examined with respect to the operation of power systems and commercial transactions in electricity markets. Of the existing forecasting models, support vector regression (SVR) has attracted much attention. While model selection, including feature selection and parameter optimization, plays an important role in short-term load forecasting using SVR, most previous studies have considered feature selection and parameter optimization as two separate tasks, which is detrimental to prediction performance.ObjectiveBy evolving feature selection and parameter optimization simultaneously, the main aims of this study are to make practitioners aware of the benefits of applying unified model selection in STLF using SVR and to provide one solution for model selection in the framework of memetic algorithm (MA).MethodsThis study proposes a comprehensive learning particle swarm optimization (CLPSO)-based memetic algorithm (CLPSO-MA) that evolves feature selection and parameter optimization simultaneously. In the proposed CLPSO-MA algorithm, CLPSO is applied to explore the solution space, while a problem-specific local search is proposed for conducting individual learning, thereby enhancing the exploitation of CLPSO.ResultsCompared with other well-established counterparts, benefits of the proposed unified model selection problem and the proposed CLPSO-MA for model selection are verified using two real-world electricity load datasets, which indicates the SVR equipped with CLPSO-MA can be a promising alternative for short-term load forecasting.     

A modified support vector regression: Integrated selection of training subset and model

In recent years, support vector regression (SVR) has become an emerging and popular forecasting technique in the field of machine learning. However, it is subjected to the model selection and learning complexity O(K * N³), especially for a massive data set (N is the size of training dataset, and K is the number of search). How to simultaneously reduce K and N can give us insight and inspiration on designing an effective and accurate selection algorithm. To this end, this paper tries to integrate the selection of training subset and model for SVR, and proposes a nested particle swarm optimization (NPSO) by inheriting the model selection of the existing training subset based SVR (TS-SVR). This nested algorithm is achieved by adaptively and periodically estimating the search region of the optimal parameter setting for TS-SVR. Complex SVR, involving large-scale training data, can be seen as extensions of TS-SVRs, yielding a nested sequence of TS-SVRs with increasing sample size. The uniform design idea is transplanted to the above modeling process, and the convergence for the proposed model is proofed. By using two artificial regression problems, Boston housing and electric load in New South Wales as empirical data, the proposed approach is compared with the standard ones, the APSO-OTS-SVR, and other existing approaches. Empirical results show that the proposed approach not only can select proper training subset and parameter, but also has better generalization performance and fewer processing time.     

融合概率分布和单调性的支持向量回归算法

传统支持向量回归是单纯基于样本数据的输入输出值建模,仅使用样本数据信息,未充分利用其他已知信息,模型泛化能力不强.为了进一步提高其性能,提出一种融合概率分布和单调性先验知识的支持向量回归算法.首先将对偶二次规划问题简化为线性规划问题,在求解时,加入与拉格朗日乘子相关的单调性约束条件;通过粒子群算法优化惩罚参数和核参数,优化目标包括四阶矩估计表示的输出样本概率分布特性.实验结果表明,融合这两部分信息的模型,能使预测值较好地满足训练样本隐含的概率分布特性及已知的单调性,既提高了预测精度,又增加了模型的可解释性.     

Parallel randomized sampling for support vector machine (SVM) and support vector regression (SVR)

A parallel randomized support vector machine (PRSVM) and a parallel randomized support vector regression (PRSVR) algorithm based on a randomized sampling technique are proposed in this paper. The proposed PRSVM and PRSVR have four major advantages over previous methods. (1) We prove that the proposed algorithms achieve an average convergence rate that is so far the fastest bounded convergence rate, among all SVM decomposition training algorithms to the best of our knowledge. The fast average convergence bound is achieved by a unique priority based sampling mechanism. (2) Unlike previous work (Provably fast training algorithm for support vector machines, 2001) the proposed algorithms work for general linear-nonseparable SVM and general non-linear SVR problems. This improvement is achieved by modeling new LP-type problems based on Karush–Kuhn–Tucker optimality conditions. (3) The proposed algorithms are the first parallel version of randomized sampling algorithms for SVM and SVR. Both the analytical convergence bound and the numerical results in a real application show that the proposed algorithm has good scalability. (4) We present demonstrations of the algorithms based on both synthetic data and data obtained from a real word application. Performance comparisons with SVM^light show that the proposed algorithms may be efficiently implemented.     

Exploiting multi-scale support vector regression for image compression

Unlike traditional neural networks that require predefined topology of the network, support vector regression (SVR) approach can model the data within the given level of accuracy with only a small subset of the training data, which are called support vectors (SVs). This property of sparsity has been exploited as the basis for image compression. In this paper, for still image compression, we propose a multi-scale support vector regression (MS-SVR) approach, which can model the images with steep variations and smooth variations very well resulting in good performance. We test our proposed MS-SVR based algorithm on some standard images. The experimental results verify that the proposed MS-SVR achieves better performance than standard SVR. And in a wide range of compression ratio, MS-SVR is very close to JPEG in terms of peak signal-to-noise ratio (PSNR) but exhibits better subjective quality. Furthermore, MS-SVR even outperforms JPEG on both PSNR and subjective quality when the compression ratio is higher enough, for example 25:1 for Lena image. Even when compared with JPEG-2000, the results show greatly similar trend as those in JPEG experiments, except that the compression ratio is a bit higher where our proposed MS-SVR will outperform JPEG-2000.     

Particle swarm optimizer with two differential mutation

In this article, a particle swarm optimization algorithm with two differential mutation (PSOTD) is proposed. In PSOTD, a novel structure with two swarms and two layers (bottom layer and top layer) is designed. The top layer consists of all the personal best particles, and the bottom layer consists of all the particles. We divide the particles in the top layer into two sub-swarms. Two different differential mutation operations with two different control parameters are employed in order to breed the particles in the top layer. Thus, one sub-swarm has a good exploration capability, and the other sub-swarm has a good exploitation capability. Obviously, since the top layer leads the bottom layer, the bottom particles achieve a good trade-off between exploration and exploitation. Under the searching structure, PSO enhances the global search capability and search efficiency. In order to test the performance of PSOTD, 44 benchmark functions widely adopted in the literature are used. The experimental results demonstrate that the proposed PSOTD outperforms most of the other tested variants of the PSO in terms of both solution quality and efficiency.     

Nonlinear model predictive control with relevance vector regression and particle swarm optimization

In this paper, a nonlinear model predictive control strategy which utilizes a probabilistic sparse kernel learning technique called relevance vector regression （RVR） and particle swarm optimization with controllable random exploration velocity （PSO-CREV） is applied to a catalytic continuous stirred tank reactor （CSTR） process. An accurate reliable nonlinear model is first identified by RVR with a radial basis function （RBF） kernel and then the optimization of control sequence is speeded up by PSO-CREV. Additional stochastic behavior in PSO-CREV is omitted for faster convergence of nonlinear optimization. An improved system performance is guaranteed by an accurate sparse predictive model and an efficient and fast optimization algorithm. To compare the performance, model predictive control （MPC） using a deterministic sparse kernel learning technique called Least squares support vector machines （LS-SVM） regression is done on a CSTR. Relevance vector regression shows improved tracking performance with very less computation time which is much essential for real time control.     

Real-time prediction of order flowtimes using support vector regression

In a make-to-order production system, a due date must be assigned to new orders that arrive dynamically, which requires predicting the order flowtime in real-time. This study develops a support vector regression model for real-time flowtime prediction in multi-resource, multi-product systems. Several combinations of kernel and loss functions are examined, and results indicate that the linear kernel and the ε$\epsilon$-insensitive loss function yield the best generalization performance. The prediction error of the support vector regression model for three different multi-resource systems of varying complexity is compared to that of classic time series models (exponential smoothing and moving average) and to a feedforward artificial neural network. Results show that the support vector regression model has lower flowtime prediction error and is more robust. More accurately predicting flowtime using support vector regression will improve due-date performance and reduce expenses in make-to-order production environments.     

基于支持向量机的污水处理软测量算法的研究

针对污水处理过程中生化需氧量BOD难以实时在线测量的问题,建立了用于预估BOD的支持向量机（SVM）的软测量模型。考虑到该支持向量机模型的测量精度取决于其两个参数C、σ能否获得最优值,采用遗传算法和粒子群优化算法,实现对这两个参数的寻优。仿真结果表明：该软测量模型的测量精度较高,可用于污水处理厂对BOD进行在线测量。     

Software reliability prediction model based on support vector regression with improved estimation of distribution algorithms

Software reliability prediction plays a very important role in the analysis of software quality and balance of software cost. The data during software lifecycle is used to analyze and predict software reliability. However, predicting the variability of software reliability with time is very difficult. Recently, support vector regression (SVR) has been widely applied to solve nonlinear predicting problems in many fields and has obtained good performance in many situations; however it is still difficult to optimize SVR's parameters. Previously, some optimization algorithms have been used to find better parameters of SVR, but these existing algorithms usually are not fully satisfactory. In this paper, we first improve estimation of distribution algorithms (EDA) in order to maintain the diversity of the population, and then a hybrid improved estimation of distribution algorithms (IEDA) and SVR model, called IEDA-SVR model, is proposed. IEDA is used to optimize parameters of SVR, and IEDA-SVR model is used to predict software reliability. We compare IEDA-SVR model with other software reliability models using real software failure datasets. The experimental results show that the IEDA-SVR model has better prediction performance than the other models.     

Nonlinear system identification using memetic differential evolution trained neural networks

Several gradient-based approaches such as back propagation (BP) and Levenberg Marquardt (LM) methods have been developed for training the neural network (NN) based systems. But, for multimodal cost functions these procedures may lead to local minima, therefore, the evolutionary algorithms (EAs) based procedures are considered as promising alternatives. In this paper we focus on a memetic algorithm based approach for training the multilayer perceptron NN applied to nonlinear system identification. The proposed memetic algorithm is an alternative to gradient search methods, such as back-propagation and back-propagation with momentum which has inherent limitations of many local optima. Here we have proposed the identification of a nonlinear system using memetic differential evolution (DE) algorithm and compared the results with other six algorithms such as Back-propagation (BP), Genetic Algorithm (GA), Particle Swarm Optimization (PSO), Differential Evolution (DE), Genetic Algorithm Back-propagation (GABP), Particle Swarm Optimization combined with Back-propagation (PSOBP). In the proposed system identification scheme, we have exploited DE to be hybridized with the back propagation algorithm, i.e. differential evolution back-propagation (DEBP) where the local search BP algorithm is used as an operator to DE. These algorithms have been tested on a standard benchmark problem for nonlinear system identification to prove their efficacy. First examples shows the comparison of different algorithms which proves that the proposed DEBP is having better identification capability in comparison to other. In example 2 good behavior of the identification method is tested on an one degree of freedom (1DOF) experimental aerodynamic test rig, a twin rotor multi-input-multi-output system (TRMS), finally it is applied to Box and Jenkins Gas furnace benchmark identification problem and its efficacy has been tested through correlation analysis.     

A novel support vector regression for data set with outliers

Support vector machine (SVM) is sensitive to the outliers, which reduces its generalization ability. This paper presents a novel support vector regression (SVR) together with fuzzification theory, inconsistency matrix and neighbors match operator to address this critical issue. Fuzzification method is exploited to assign similarities on the input space and on the output response to each pair of training samples respectively. The inconsistency matrix is used to calculate the weights of input variables, followed by searching outliers through a novel neighborhood matching algorithm and then eliminating them. Finally, the processed data is sent to the original SVR, and the prediction results are acquired. A simulation example and three real-world applications demonstrate the proposed method for data set with outliers.     

Parametric identification of seismic isolators using differential evolution and particle swarm optimization

The objective of a base isolation system is to decouple the building from the damaging components of the earthquake by placing isolators between the superstructure and the foundation. The correct identification of these devices is, therefore, a critical step towards reliable simulations of base-isolated systems subjected to dynamic ground motion. In this perspective, the parametric identification of seismic isolators from experimental dynamic tests is here addressed. In doing so, the focus is on identifying Bouc–Wen model parameters by means of particle swarm optimization and differential evolution. This paper is especially concerned with the assessment of these non-classical parametric identification techniques using a standardized experimental protocol to set out the dynamic loading conditions. A critical review of the obtained outputs demonstrates that particle swarm optimization and differential evolution can be effectively exploited for the parametric identification of seismic isolators.     

A multiple criteria active learning method for support vector regression

This paper presents a novel active learning method developed in the framework of ε-insensitive support vector regression (SVR) for the solution of regression problems with small size initial training data. The proposed active learning method selects iteratively the most informative as well as representative unlabeled samples to be included in the training set by jointly evaluating three criteria: (i) relevancy, (ii) diversity, and (iii) density of samples. All three criteria are implemented according to the SVR properties and are applied in two clustering-based consecutive steps. In the first step, a novel measure to select the most relevant samples that have high probability to be located either outside or on the boundary of the ε-tube of SVR is defined. To this end, initially a clustering method is applied to all unlabeled samples together with the training samples that are inside the ε-tube (those that are not support vectors, i.e., non-SVs); then the clusters with non-SVs are eliminated. The unlabeled samples in the remaining clusters are considered as the most relevant patterns. In the second step, a novel measure to select diverse samples among the relevant patterns from the high density regions in the feature space is defined to better model the SVR learning function. To this end, initially clusters with the highest density of samples are chosen to identify the highest density regions in the feature space. Then, the sample from each selected cluster that is associated with the portion of feature space having the highest density (i.e., the most representative of the underlying distribution of samples contained in the related cluster) is selected to be included in the training set. In this way diverse samples taken from high density regions are efficiently identified. Experimental results obtained on four different data sets show the robustness of the proposed technique particularly when a small-size initial training set are available.