Testing linear independence in linear models with interval-valued data

Testing methods are introduced in order to determine whether there is some ‘linear’ relationship between imprecise predictor and response variables in a regression analysis. The variables are assumed to be interval-valued. Within this context, the variables are formalized as compact convex random sets, and an interval arithmetic-based linear model is considered. Then, a suitable equivalence for the hypothesis of linear independence in this model is obtained in terms of the mid-spread representations of the interval-valued variables. That is, in terms of some moments of random variables. Methods are constructed to test this equivalent hypothesis; in particular, the one based on bootstrap techniques will be applicable in a wide setting. The methodology is illustrated by means of a real-life example, and some simulation studies are considered to compare techniques in this framework.     

Constrained predictive pole-placement control with linear models

Predictive pole-placement (PPP) control is a continuous-time MPC using a particular set of basis functions leading to pole-placement behaviour in the unconstrained case. This paper presents two modified versions of the PPP controller which are each shown to have desirable stability properties when controlling systems with input, output and state constraints.     

Parse-matrix evolution for symbolic regression

Data-driven model is highly desirable for industrial data analysis in case the experimental model structure is unknown or wrong, or the concerned system has changed. Symbolic regression is a useful method to construct the data-driven model (regression equation). Existing algorithms for symbolic regression such as genetic programming and grammatical evolution are difficult to use due to their special target programming language (i.e., LISP) or additional function parsing process. In this paper, a new evolutionary algorithm, parse-matrix evolution (PME), for symbolic regression is proposed. A chromosome in PME is a parse-matrix with integer entries. The mapping process from the chromosome to the regression equation is based on a mapping table. PME can easily be implemented in any programming language and free to control. Furthermore, it does not need any additional function parsing process. Numerical results show that PME can solve the symbolic regression problems effectively.     

基于空间矢量变换的模型约束变形

基于空间矢量线性变换知识,提出了一种新的多边形网格模型变形方法。在三维模型空间选择一点作为约束源,并设置此约束源的影响半径。通过计算约束源与三维模型网格面片顶点之间的距离来确定待变形局部顶点区域。把空间矢量的线性变换应用到网格面片顶点变形所需的向量函数,依此函数直接精确地计算出网格面片顶点的新位置,从而实现模型的变形。通过对算法原理的进一步分析扩展了该变形方法的应用范围。     

Polynomial regression interval-valued fuzzy systems

In recent years, the type-2 fuzzy sets theory has been used to model and minimize the effects of uncertainties in rule-base fuzzy logic system (FLS). In order to make the type-2 FLS reasonable and reliable, a new simple and novel statistical method to decide interval-valued fuzzy membership functions and probability type reduce reasoning method for the interval-valued FLS are developed. We have implemented the proposed non-linear (polynomial regression) statistical interval-valued type-2 FLS to perform smart washing machine control. The results show that our quadratic statistical method is more robust to design a reliable type-2 FLS and also can be extend to polynomial model.     

Collinearity detection in linear regression models

Multicollinearity can seriously affect least-squares parameter estimates. Many methods have been suggested to determine those parameters most involved. This paper, beginning with the contributions of Belsley, Kuh, and Welsch (1980) and Belsley (1991), forges a new direction. A decomposition of the variable space allows the near dependencies to be isolated in one sub-space. And this, in turn, allows a corresponding decomposition of the main statistics, as well as a new one proposed here, to provide better information on the structure of the collinear relations.     

Centre and Range method for fitting a linear regression model to symbolic interval data

This paper introduces a new approach to fitting a linear regression model to symbolic interval data. Each example of the learning set is described by a feature vector, for which each feature value is an interval. The new method fits a linear regression model on the mid-points and ranges of the interval values assumed by the variables in the learning set. The prediction of the lower and upper bounds of the interval value of the dependent variable is accomplished from its mid-point and range, which are estimated from the fitted linear regression model applied to the mid-point and range of each interval value of the independent variables. The assessment of the proposed prediction method is based on the estimation of the average behaviour of both the root mean square error and the square of the correlation coefficient in the framework of a Monte Carlo experiment. Finally, the approaches presented in this paper are applied to a real data set and their performance is compared.     

Nonlinear regression applied to interval-valued data

This paper introduces a nonlinear regression model to interval-valued data. The method extends the classical nonlinear regression model in order to manage interval-valued datasets. The parameter estimates of the nonlinear model considers some optimization algorithms aiming to identify which one presents the best accuracy and precision in the prediction task. A detailed prediction performance study comparing the proposed nonlinear method and other linear regression methods for interval variables is presented based on K-fold cross-validation scheme with synthetic interval-valued datasets generated on a Monte Carlo framework. Moreover, two suitable real interval-valued datasets are considered to illustrate the usefulness and the performance of the approaches presented in this paper. The results suggested that the use of the nonlinear method is suitable for real datasets, as well as in the Monte Carlo simulation study.     

Robust likelihood inference for regression parameters in partially linear models

A robust likelihood approach is proposed for inference about regression parameters in partially-linear models. More specifically, normality is adopted as the working model and is properly corrected to accomplish the objective. Knowledge about the true underlying random mechanism is not required for the proposed method. Simulations and illustrative examples demonstrate the usefulness of the proposed robust likelihood method, even in irregular situations caused by the components of the nonparametric smooth function in partially-linear models.     

Meta-heuristic algorithms for parameter estimation of semi-parametric linear regression models

Consider the semi-parametric linear regression model Y=β^′X+ε, where ε has an unknown distribution F₀. The semi-parametric MLE of β under this set-up is called the generalized semi-parametric MLE(GSMLE). Although the GSML estimation of the linear regression model is statistically appealing, it has never been attempted due to difficulties with obtaining the GSML estimates of β and F until recent work on linear regression for complete data and for right-censored data by Yu and Wong [2003a. Asymptotic properties of the generalized semi-parametric MLE in linear regression. Statistica Sinica 13, 311-326; 2003b. Semi-parametric MLE in simple linear regression analysis with interval-censored data. Commun. Statist.—Simulation Comput. 32, 147-164; 2003c. The semi-parametric MLE in linear regression with right censored data. J. Statist. Comput. Simul. 73, 833-848]. However, after obtaining all candidates, their algorithm simply does an exhaustive search to find the GSML estimators. In this paper, it is shown that Yu and Wong's algorithm leads to the so-called dimension disaster. Based on their idea, a simulated annealing algorithm for finding semi-parametric MLE is proposed along with techniques to reduce computations. Experimental results show that the new algorithm runs much faster for multiple linear regression models while keeping the nice features of Yu and Wong's original one.     

Robust recursive identification of multidimensional linear regression models

Stochastic adaptive estimation and control algorithms involving recursive prediction estimates have guaranteed convergence rates when the noise is not ‘too’ coloured, as when a positive-real condition on the noise mode is satisfied. Moreover, the whiter the noise environment the more robust are the algorithms. This paper shows that for linear regression signal models, the suitable introduction of while noise into the estimation algorithm can make it more robust without compromising on convergence rates. Indeed, there are guaranteed attractive convergence rates independent of the process noise colour. No positive-real condition is imposed on the noise model.     

Hoeffding bound based evolutionary algorithm for symbolic regression

In symbolic regression area, it is difficult for evolutionary algorithms to construct a regression model when the number of sample points is very large. Much time will be spent in calculating the fitness of the individuals and in selecting the best individuals within the population. Hoeffding bound is a probability bound for sums of independent random variables. As a statistical result, it can be used to exactly decide how many samples are necessary for choosing i individuals from a population in evolutionary algorithms without calculating the fitness completely. This paper presents a Hoeffding bound based evolutionary algorithm (HEA) for regression or approximation problems when the number of the given learning samples is very large. In HEA, the original fitness function is used in every k generations to update the approximate fitness obtained by Hoeffding bound. The parameter 1?δ is the probability of correctly selecting i best individuals from population P, which can be tuned to avoid an unstable evolution process caused by a large discrepancy between the approximate model and the original fitness function. The major advantage of the proposed HEA algorithm is that it can guarantee that the solution discovered has performance matching what would be discovered with a traditional genetic programming (GP) selection operator with a determinate probability and the running time can be reduced largely. We examine the performance of the proposed algorithm with several regression problems and the results indicate that with the similar accuracy, the HEA algorithm can find the solution more efficiently than tradition EA. It is very useful for regression problems with large number of training samples.     

Constrained stabilization problems for linear plants

Linear systems with magnitude and rate constraints on both the state and control variables are considered. For such systems, semi-global and global constrained stabilization problems are formulated when state feedback controllers are used. Necessary and sufficient conditions for the solvability of the formulated problems are developed. Moreover, design methodologies for such constrained stabilization problems are presented. An important aspect of our development here is a taxonomy of constraints to show clearly for what type of constraints what can or cannot be achieved.     

Genetic algorithms for outlier detection and variable selection in linear regression models

This article addresses some problems in outlier detection and variable selection in linear regression models. First, in outlier detection there are problems known as smearing and masking. Smearing means that one outlier makes another, non-outlier observation appear as an outlier, and masking that one outlier prevents another one from being detected. Detecting outliers one by one may therefore give misleading results. In this article a genetic algorithm is presented which considers different possible groupings of the data into outlier and non-outlier observations. In this way all outliers are detected at the same time. Second, it is known that outlier detection and variable selection can influence each other, and that different results may be obtained, depending on the order in which these two tasks are performed. It may therefore be useful to consider these tasks simultaneously, and a genetic algorithm for a simultaneous outlier detection and variable selection is suggested. Two real data sets are used to illustrate the algorithms, which are shown to work well. In addition, the scalability of the algorithms is considered with an experiment using generated data.I would like to thank Dr Tero Aittokallio and an anonymous referee for useful comments.     

Local polynomial estimation in partial linear regression models under dependence

A regression model whose regression function is the sum of a linear and a nonparametric component is presented. The design is random and the response and explanatory variables satisfy mixing conditions. A new local polynomial type estimator for the nonparametric component of the model is proposed and its asymptotic normality is obtained. Specifically, this estimator works on a prewhitening transformation of the dependent variable, and the results show that it is asymptotically more efficient than the conventional estimator (which works on the original dependent variable) when the errors of the model are autocorrelated. A simulation study and an application to a real data set give promising results.     

A study of partial F tests for multiple linear regression models

Partial F tests play a central role in model selections in multiple linear regression models. This paper studies the partial F tests from the view point of simultaneous confidence bands. It first shows that there is a simultaneous confidence band associated naturally with a partial F test. This confidence band provides more information than the partial F test and the partial F test can be regarded as a side product of the confidence band. This view point of confidence bands also leads to insights of the major weakness of the partial F tests, that is, a partial F test requires implicitly that the linear regression model holds over the entire range of the covariates in concern. Improved tests are proposed and they are induced by simultaneous confidence bands over restricted regions of the covariates. Power comparisons between the partial F tests and the new tests have been carried out to assess when the new tests are more or less powerful than the partial F tests. Computer programmes have been developed for easy implements of these new confidence band based inferential methods. An illustrative example is provided.     

Constrained linear quadratic regulation

The paper is a contribution to the theory of the infinite-horizon linear quadratic regulator (LQR) problem subject to inequality constraints on the inputs and states, extending an approach first proposed by Sznaier and Damborg (1987). A solution algorithm is presented, which requires solving a finite number of finite-dimensional positive definite quadratic programs. The constrained LQR outlined does not feature the undesirable mismatch between open-loop and closed-loop nominal system trajectories, which is present in the other popular forms of model predictive control (MPC) that can be implemented with a finite quadratic programming algorithm. The constrained LQR is shown to be both optimal and stabilizing. The solution algorithm is guaranteed to terminate in finite time with a computational cost that has a reasonable upper bound compared to the minimal cost for computing the optimal solution. Inherent to the approach is the removal of a tuning parameter, the control horizon, which is present in other MPC approaches and for which no reliable tuning guidelines are available. Two examples are presented that compare constrained LQR and two other popular forms of MPC. The examples demonstrate that constrained LQR achieves significantly better performance than the other forms of MPC on some plants, and the computational cost is not prohibitive for online implementation     

Simultaneous selection of variables and smoothing parameters in structured additive regression models

In recent years, considerable research has been devoted to developing complex regression models that can deal simultaneously with nonlinear covariate effects and time trends, unit- or cluster specific heterogeneity, spatial heterogeneity and complex interactions between covariates of different types. Much less effort, however, has been devoted to model and variable selection. The paper develops a methodology for the simultaneous selection of variables and the degree of smoothness in regression models with a structured additive predictor. These models are quite general, containing additive (mixed) models, geoadditive models and varying coefficient models as special cases. This approach allows one to decide whether a particular covariate enters the model linearly or nonlinearly or is removed from the model. Moreover, it is possible to decide whether a spatial or cluster specific effect should be incorporated into the model to cope with spatial or cluster specific heterogeneity. Particular emphasis is also placed on selecting complex interactions between covariates and effects of different types. A new penalty for two-dimensional smoothing is proposed, that allows for ANOVA-type decompositions into main effects and an interaction effect without explicitly specifying the main effects. The penalty is an additive combination of other penalties. Fast algorithms and software are developed that allow one to even handle situations with many covariate effects and observations. The algorithms are related to backfitting and Markov chain Monte Carlo techniques, which divide the problem in a divide and conquer strategy into smaller pieces. Confidence intervals taking model uncertainty into account are based on the bootstrap in combination with MCMC techniques.     

Simultaneous selection of variables and smoothing parameters in structured additive regression models

In recent years, considerable research has been devoted to developing complex regression models that can deal simultaneously with nonlinear covariate effects and time trends, unit- or cluster specific heterogeneity, spatial heterogeneity and complex interactions between covariates of different types. Much less effort, however, has been devoted to model and variable selection. The paper develops a methodology for the simultaneous selection of variables and the degree of smoothness in regression models with a structured additive predictor. These models are quite general, containing additive (mixed) models, geoadditive models and varying coefficient models as special cases. This approach allows one to decide whether a particular covariate enters the model linearly or nonlinearly or is removed from the model. Moreover, it is possible to decide whether a spatial or cluster specific effect should be incorporated into the model to cope with spatial or cluster specific heterogeneity. Particular emphasis is also placed on selecting complex interactions between covariates and effects of different types. A new penalty for two-dimensional smoothing is proposed, that allows for ANOVA-type decompositions into main effects and an interaction effect without explicitly specifying the main effects. The penalty is an additive combination of other penalties. Fast algorithms and software are developed that allow one to even handle situations with many covariate effects and observations. The algorithms are related to backfitting and Markov chain Monte Carlo techniques, which divide the problem in a divide and conquer strategy into smaller pieces. Confidence intervals taking model uncertainty into account are based on the bootstrap in combination with MCMC techniques.