支持向量回归方法用于1,4,2-二氮磷杂环戊-5-(硫)酮类化合物除草活性的QSPR研究

采用支持向量回归方法研究了1,4,2-二氮磷杂环戊-5-(硫)酮类化合物除草活性的QSAR。基于留一法交叉验证的结果,比较了支持向量机回归(SVR)与几种常用建模方法对于该类化合物除草活性的预测精度。研究表明:所建SVR模型的精度高于逆传播人工神经网络(BPANN)、多元线性回归和偏最小二乘(PLS)所得结果。     

Reduction to Least-Squares Estimates in Multiple Fuzzy Regression Analysis

In this paper, we deal with the problem of least-squares multiple regression with fuzzy data. The regression coefficients are assumed to be real (crisp). A formula for solving the regression coefficients in one-variable models is derived. If each independent variable is effective (i.e., its corresponding regression coefficient is nonzero), the multiple regression problem can be replaced with a 0-1 programming problem. Its optimal solution is easily computed. Finally, we also propose effective algorithms to compute the regression coefficients in a general case.      

连续超松弛支持向量机回归算法应用研究

支持向量回归问题的研究,对函数拟合(回归逼近)具有重要的理论和应用意义.借鉴分类问题的有效算法,将其推广到回归问题中来,针对用于分类问题的SOR支持向量机有效算法,提出了SORR支持向量回归算法.在若干不同维数的数据集上,对SORR算法、ASVR算法和LibSVM算法进行数值试验,并进行比较分析.数值实验结果表明,SORR算法是有效的,与当前流行的支持向量机回归算法相比,在回归精度和学习速度上都有一定的优势.     

Training nu-support vector regression: theory and algorithms

We discuss the relation between epsilon-support vector regression (epsilon-SVR) and nu-support vector regression (nu-SVR). In particular, we focus on properties that are different from those of C-support vector classification (C-SVC) and nu-support vector classification (nu-SVC). We then discuss some issues that do not occur in the case of classification: the possible range of epsilon and the scaling of target values. A practical decomposition method for nu-SVR is implemented, and computational experiments are conducted. We show some interesting numerical observations specific to regression.     

岭参数优化的ELM岭回归学习算法

针对ELM(extreme learning machine,极限学习机)学习算法可能存在的解的奇异问题,提出了岭参数优化的ELM岭回归学习算法(ELMRR).该算法利用岭回归方法代替原有的线性回归算法,以均方根误差为性能指标,采用粒子群优化算法确定最佳岭参数.为了验证该方法的有效性,对函数回归和分类问题进行仿真实验分析,结果表明该方法改善了ELM的预测性能且克服了传统岭回归算法岭参数难以确定的缺点.     

A meta-heuristic framework for forecasting household electricity consumption

It may be difficult to model household electricity consumption with conventional methods such as regression due to seasonal and monthly changes. This paper illustrates a flexible integrated meta-heuristic framework based on Artificial Neural Network (ANN) Multi Layer Perceptron (MLP), conventional regression and design of experiment (DOE) for forecasting household electricity consumption. Previous studies base their verification by the difference in error estimation, whereas this study uses various error estimation methods and design of experiment (DOE). Moreover, DOE is based on analysis of variance (ANOVA) and Duncan Multiple Range Test (DMRT). Furthermore, actual data is compared with ANN MLP and conventional regression model through ANOVA. If the null hypothesis is accepted, DMRT is used to select either ANN MLP or conventional regression. However, if the null hypothesis is accepted then the proposed framework selects either the MLP or regression model based on the average of Minimum Absolute Percentage Error (MAPE), Mean Square Error (MSE) and Mean Absolute Error (MAE). The significance of this study is the integration of ANN MLP, conventional regression and DOE for flexible modeling and improved processing, development and testing of household electricity consumption. Some of the previous studies assume that ANN MLP provide better estimation and others estimate electricity consumptions based on the conventional regression approach. However, this study presents a flexible integrated framework to locate the best model based on the actual data. Moreover, it would provide more reliable and precise forecasting for policy makers. To show the applicability and superiority of the integrated approach, annual household electricity consumption in Iran from 1974 to 2003 was collected for processing, training and testing purpose.     

On cluster-wise fuzzy regression analysis

Since Tanaka et al. (1982) proposed a study of linear regression analysis with a fuzzy model, fuzzy regression analysis has been widely studied and applied in a variety of substantive areas. Regression analysis in the case of heterogeneity of observations is commonly presented in practice. The authors' main goal is to apply fuzzy clustering techniques to fuzzy regression analysis. Fuzzy clustering is used to overcome the heterogeneous problem in the fuzzy regression model. They present the cluster-wise fuzzy regression analysis in two approaches: the two-stage weighted fuzzy regression and the one-stage generalized fuzzy regression. The two-stage procedure extends the results of Jajuga (1986) and Diamond (1988). The one-stage approach is created by embedding fuzzy clusterings into the fuzzy regression model fitting at each step of procedure. This kind of embedding in the one-stage procedure is more effective since the structure of regression line shape encountered in the data set is taken into account at each iteration of the algorithm. Numerical results give evidence that the one-stage procedure can be highly recommended in cluster-wise fuzzy regression analysis.     

Coupling machine learning,tree-based and statistical models with cellular automata to simulate urban growth

This paper compares six land use change (LUC) models, including artificial neural networks (ANNs), support vector regression (SVR), random forest (RF), classification and regression trees (CART), logistic regression (LR), and multivariate adaptive regression splines (MARS). These models were used to simulate urban growth in the megacity of Tehran Metropolitan Area (TMA). These LUC models were integrated with cellular automata (CA) and validated using a variety of goodness-of-fit metrics. The results showed that the percent correct metrics (PCMs) varied between 54.6% for LR and 59.6% for MARS, while the area under curve (AUC) ranged from 67.6% for LR to 74.7% for ANNs. The results also showed a considerable difference between the spatial patterns within the error maps. The results of this comparative study will enable decision makers and scholars to better understand the performance of the models when reducing the number of misses and false alarms is a priority.     

A method for regularization of evolutionary polynomial regression

While many applications require models that have no acceptable linear approximation, the simpler nonlinear models are defined by polynomials. The use of genetic algorithms to find polynomial models from data is known as evolutionary polynomial regression (EPR). This paper introduces evolutionary polynomial regression with regularization, an algorithm extending EPR with a regularization term to control polynomial complexity. The article also describes a set of experiences to compare both flavors of EPR against other methods including linear regression, regression trees and support vector regression. These experiments show that evolutionary polynomial regression with regularization is able to achieve better fitting and needs less computation time than plain EPR.     

Aggregate a posteriori linear regression adaptation

We present a new discriminative linear regression adaptation algorithm for hidden Markov model (HMM) based speech recognition. The cluster-dependent regression matrices are estimated from speaker-specific adaptation data through maximizing the aggregate a posteriori probability, which can be expressed in a form of classification error function adopting the logarithm of posterior distribution as the discriminant function. Accordingly, the aggregate a posteriori linear regression (AAPLR) is developed for discriminative adaptation where the classification errors of adaptation data are minimized. Because the prior distribution of regression matrix is involved, AAPLR is geared with the Bayesian learning capability. We demonstrate that the difference between AAPLR discriminative adaptation and maximum a posteriori linear regression (MAPLR) adaptation is due to the treatment of the evidence. Different from minimum classification error linear regression (MCELR), AAPLR has closed-form solution to fulfil rapid adaptation. Experimental results reveal that AAPLR speaker adaptation does improve speech recognition performance with moderate computational cost compared to maximum likelihood linear regression (MLLR), MAPLR, MCELR and conditional maximum likelihood linear regression (CMLLR). These results are verified for supervised adaptation as well as unsupervised adaptation for different numbers of adaptation data.     

基于ELM的改进CART决策树回归算法

为提高CART(ClassificationAndRegressionTree)决策树回归算法的准确性,提出一种基于ELM(Extreme Learning Machine)的改进CART决策树回归算法——ELM-CART算法.所提算法主要是在CART回归树创建过程中,在每个叶节点使用极限学习机建模,可以得到真正意义上...     

Estimation of individual prediction reliability using the local sensitivity analysis

For a given prediction model, some predictions may be reliable while others may be unreliable. The average accuracy of the system cannot provide the reliability estimate for a single particular prediction. The measure of individual prediction reliability can be important information in risk-sensitive applications of machine learning (e.g. medicine, engineering, business). We define empirical measures for estimation of prediction accuracy in regression. Presented measures are based on sensitivity analysis of regression models. They estimate reliability for each individual regression prediction in contrast to the average prediction reliability of the given regression model. We study the empirical sensitivity properties of five regression models (linear regression, locally weighted regression, regression trees, neural networks, and support vector machines) and the relation between reliability measures and distribution of learning examples with prediction errors for all five regression models. We show that the suggested methodology is appropriate only for the three studied models: regression trees, neural networks, and support vector machines, and test the proposed estimates with these three models. The results of our experiments on 48 data sets indicate significant correlations of the proposed measures with the prediction error.     

Enrollment Management Model: Artificial Neural Networks versus Logistic Regression

This paper presents an enrollment management model by applying artificial neural network (ANN). The aim of the research, which has been presented in this paper, is to show that ANNs are more successful in predicting than the classical statistical method – regression analysis (logistic regression). Both predictive models, no matter whether they are based on ANNs or logistic regression, offer satisfactory predictive results, and they can offer support in the decision-making process. However, the model based on neural networks shows certain advantages. ANNs demand understanding of functional connection between independent and dependent variables in order to evaluate the model. Also, they adapt easily to related independent variables, without the appearance of the problem of multicollinearity. In contrast to logistic regression, neural networks can recognize the appearance of nonlinearity and interactions in input data, and they can react on time.     

最小方差支撑向量回归

基于支撑向量回归(SVR)可以通过构建支撑向量机分类问题实现的基本思想,推广最小类方差支撑向量机(MCVSVMs)于回归估计,提出了最小方差支撑向量回归(MVSVR)算法.该方法继承了MCVSVMs鲁棒性和泛化能力强的优点,分析了MVSVR和标准SVR之间的关系,讨论了在散度矩阵奇异情况下该方法的求解问题,同时也讨论了MVSVR的非线性情况.实验表明,该方法是可行的,且表现出了更强的泛化能力.     

Parse-matrix evolution for symbolic regression

Data-driven model is highly desirable for industrial data analysis in case the experimental model structure is unknown or wrong, or the concerned system has changed. Symbolic regression is a useful method to construct the data-driven model (regression equation). Existing algorithms for symbolic regression such as genetic programming and grammatical evolution are difficult to use due to their special target programming language (i.e., LISP) or additional function parsing process. In this paper, a new evolutionary algorithm, parse-matrix evolution (PME), for symbolic regression is proposed. A chromosome in PME is a parse-matrix with integer entries. The mapping process from the chromosome to the regression equation is based on a mapping table. PME can easily be implemented in any programming language and free to control. Furthermore, it does not need any additional function parsing process. Numerical results show that PME can solve the symbolic regression problems effectively.     

Lagrange支持向量回归机算法研究

支持向量回归机问题的研究远没有像支持向量机问题成熟完善,支持向量回归机对函数拟合(回归逼近)具有重要的理论和应用意义.借鉴分类问题的有效算法,将其推广到回归问题中来,针对Lagrange支持向量机(LSVM)算法,提出了有效的Lagrange支持向量回归机(LSVR)算法,在若干不同维数的数据集上,对LSVR算法、ASVR算法和LibSVM算法进行数值试验,并进行比较分析.数值试验表明LSVR算法是有效的,与当前流行的求解支持向量回归机的算法相比,在时间和正确度上都有一定的优势.     

Reduced Rank Kernel Ridge Regression

Ridge regression is a classical statistical technique that attempts to address the bias-variance trade-off in the design of linear regression models. A reformulation of ridge regression in dual variables permits a non-linear form of ridge regression via the well-known kernel trick. Unfortunately, unlike support vector regression models, the resulting kernel expansion is typically fully dense. In this paper, we introduce a reduced rank kernel ridge regression (RRKRR) algorithm, capable of generating an optimally sparse kernel expansion that is functionally identical to that resulting from conventional kernel ridge regression (KRR). The proposed method is demonstrated to out-perform an alternative sparse kernel ridge regression algorithm on the Motorcycle and Boston Housing benchmarks.     

聚类再回归方法在机场噪声时间序列预测中的应用

对于机场噪声的预测,针对绘制等值线方法预测成本高和误差较大的缺点,以及分类再回归 方法中分类时缺乏可指导性标准的问题,本文提出了基于支持向量机的先聚类、再回归的时间序列的预测方法。对机场噪声时间序列的先聚类再回归方法,采用常用k均值划分算法,利用聚类特点,将样本限定在同一类的范围内,再对同类样本进行回归预测。Housing及Laser generated data数据集上的实验表明,采用先聚类再回归方法得到的拟合值比直接回归方法得到的拟合值要精确。将该方法应用到北京某机场实测数据中,并与其他预测模型进行对比,准确度明显优于其他预测方法。     

基于时间序列的模糊支持向量回归

在分析了模糊支持向量回归的基础上,给出了一种基于时间序列分析的模糊支持向量回归方法TS-FSVR,并通过粒子群优化方法对模型中的参数进行了优化。并将该方法成功地应用到城市旅游环境承载力的评估系统中,取得了较好的实验结果。     

Bayesian analysis of empirical software engineering cost models

Many parametric software estimation models have evolved in the last two decades (L.H. Putnam and W. Myers, 1992; C. Jones, 1997; R.M. Park et al., 1992). Almost all of these parametric models have been empirically calibrated to actual data from completed software projects. The most commonly used technique for empirical calibration has been the popular classical multiple regression approach. As discussed in the paper, the multiple regression approach imposes a few assumptions frequently violated by software engineering datasets. The paper illustrates the problems faced by the multiple regression approach during the calibration of one of the popular software engineering cost models, COCOMO II. It describes the use of a pragmatic 10 percent weighted average approach that was used for the first publicly available calibrated version (S. Chulani et al., 1998). It then moves on to show how a more sophisticated Bayesian approach can be used to alleviate some of the problems faced by multiple regression. It compares and contrasts the two empirical approaches, and concludes that the Bayesian approach was better and more robust than the multiple regression approach