递推局部线性回归估计及其应用

在非参数统计中,局部多项式回归是重要的工具,然而以往研究的算法基本都是非递推的.本文研究递推的局部线性回归估计及其应用.首先推导出递推算法,给出了回归函数及其导函数的非参数估计.在一定的条件下,证明了算法的强一致性.并且通过仿真例子研究了它在非线性条件异方差模型的回归函数估计和非线性ARX(nonlinear autoregressive system with exogenous inputs,NARX)系统辨识中的应用.     

Optimal self-tuning filtering, prediction, and smoothing for discrete multivariable processes

An algorithm is proposed for self-tuning optimal fixed-lag smoothing or filtering for linear discrete-time multivariable processes. Az-transfer function solution to the discrete multivariable estimation problem is first presented. This solution involves spectral factorization of polynomial matrices and assumes knowledge of the process parameters and the noise statistics. The assumption is then made that the signal-generating process and noise statistics are unknown. The problem is reformulated so that the model is in an innovations signal form, and implicit self-tuning estimation algorithms are proposed. The parameters of the innovation model of the process can be estimated using an extended Kalman filter or, alternatively, extended recursive least squares. These estimated parameters are used directly in the calculation of the predicted, smoothed, or filtered estimates. The approach is an attempt to generalize the work of Hagander and Wittenmark.     

Testing the link when the index is semiparametric—a comparative study

Generalized structured models are popular in applied statistics. They can circumvent the curse of dimensionality and provide results that are easy to interpret. However, there are two major concerns that need to be addressed before they are applied. Firstly, the credibility of the specified structure, such as additivity, and secondly, the specification of the link function need to be assessed. The focus is on the latter issue. In many cases it is feasible to estimate a nonparametric link, but the effort is often not justified. In contrast parametric links enable the use of likelihood-based estimates, which are asymptotically efficient, and which perform excellently in practice, particularly for small samples. Several statistics for testing the credibility of parametric link specifications are introduced. Estimation and implementation are discussed, and the performance of the statistics is compared in an intensive simulation study. Applications to real data are also described.     

Testing the link when the index is semiparametric—a comparative study

Generalized structured models are popular in applied statistics. They can circumvent the curse of dimensionality and provide results that are easy to interpret. However, there are two major concerns that need to be addressed before they are applied. Firstly, the credibility of the specified structure, such as additivity, and secondly, the specification of the link function need to be assessed. The focus is on the latter issue. In many cases it is feasible to estimate a nonparametric link, but the effort is often not justified. In contrast parametric links enable the use of likelihood-based estimates, which are asymptotically efficient, and which perform excellently in practice, particularly for small samples. Several statistics for testing the credibility of parametric link specifications are introduced. Estimation and implementation are discussed, and the performance of the statistics is compared in an intensive simulation study. Applications to real data are also described.     

Bayesian analysis of semiparametric reproductive dispersion mixed-effects models

Semiparametric reproductive dispersion mixed-effects model (SPRDMM) is an extension of the reproductive dispersion model and the semiparametric mixed model, and it includes many commonly encountered models as its special cases. A Bayesian procedure is developed for analyzing SPRDMMs on the basis of P-spline estimates of nonparametric components. A hybrid algorithm combining the Gibbs sampler and the Metropolis-Hastings algorithm is used to simultaneously obtain the Bayesian estimates of unknown parameters, smoothing function and random effects, as well as their standard error estimates. The Bayes factor for model comparison is employed to select better approximation of the smoothing function via path sampling. Several simulation studies and a real example are used to illustrate the proposed methodologies.     

A Fusion Kalman Filter and UFIR Estimator Using the Influence Function Method

In this paper,the Kalman filter(KF)and the unbiased finite impulse response(UFIR)filter are fused in the discrete-time state-space to improve robustness against uncertainties.To avoid the problem where fusion filters may give up some advantages of UFIR filters by fusing based on noise statistics,we attempt to find a way to fuse without using noise statistics.The fusion filtering algorithm is derived using the influence function that provides a quantified measure for disturbances on the resulting filtering outputs and is termed as an influence finite impulse response(IFIR)filter.The main advantage of the proposed method is that the noise statistics of process noise and measurement noise are no longer required in the fusion process,showing that a critical feature of the UFIR filter is inherited.One numerical example and a practice-oriented case are given to illustrate the effectiveness of the proposed method.It is shown that the IFIR filter has adaptive performance and can automatically switch from the Kalman estimate to the UFIR estimates according to operating conditions.Moreover,the proposed method can reduce the effects of optimal horizon length on the UFIR estimate and can give the state estimates of best accuracy among all the compared methods.     

Automatic methods of useful signals extraction from noise background under conditions of nonparametric uncertainty

Three basic techniques for random signals processing are under study: the problems of filtration, interpolation, and prediction. The last advances (including those of the author) in finding smoothing parameter (bandwidth) in the problems of nonparametric kernel estimation of unknown probability densities and their derivatives made it possible to advance further in the theory of the nonparametric estimation of signals with unknown distribution. This progress gave rise to the evolution of automatic methods for signals extraction from noise under the conditions of nonparametric uncertainty. The word “automatic” is understood in the sense that the suggested methods for processing signals depend only on the observable sample. In the article, by the simple example of the additive model, the comparison is made of the nonparametric procedures for the signals processing with the known optimal processing procedures obtained at the complete statistical information about the signals and noise distributions. The results of computer modeling show that the accuracy of nonparametric signals estimates insignificantly gives up to the accuracy of optimal estimates.     

Study on semiparametric Wilcoxon fuzzy neural networks

Fuzzy neural network (FNN) has long been recognized as an efficient and powerful learning machine for general machine learning problems. Recently, Wilcoxon fuzzy neural network (WFNN), which generalizes the rank-based Wilcoxon approach for linear parametric regression problems to nonparametric neural network, was proposed aiming at improving robustness against outliers. FNN and WFNN are nonparametric models in the sense that they put no restrictions, except possibly smoothness, on the functional form of the regression function. However, they may be difficult to interpret and, even worse, yield poor estimates with high computational cost when the number of predictor variables is large. To overcome this drawback, semiparametric models have been proposed in statistical regression theory. A semiparametric model keeps the easy interpretability of its parametric part and retains the flexibility of its nonparametric part. Based on this, semiparametric FNN and semiparametric WFNN will be proposed in this paper. The learning rules are based on the backfitting procedure frequently used in semiparametric regression. Simulation results show that the semiparametric models perform better than their nonparametric counterparts.     

Latent features in similarity judgments: a nonparametric bayesian approach

One of the central problems in cognitive science is determining the mental representations that underlie human inferences. Solutions to this problem often rely on the analysis of subjective similarity judgments, on the assumption that recognizing likenesses between people, objects, and events is crucial to everyday inference. One such solution is provided by the additive clustering model, which is widely used to infer the features of a set of stimuli from their similarities, on the assumption that similarity is a weighted linear function of common features. Existing approaches for implementing additive clustering often lack a complete framework for statistical inference, particularly with respect to choosing the number of features. To address these problems, this article develops a fully Bayesian formulation of the additive clustering model, using methods from nonparametric Bayesian statistics to allow the number of features to vary. We use this to explore several approaches to parameter estimation, showing that the nonparametric Bayesian approach provides a straightforward way to obtain estimates of both the number of features and their importance.     

A nonparametric Bayesian approach toward robot learning by demonstration

In the past years, many authors have considered application of machine learning methodologies to effect robot learning by demonstration. Gaussian mixture regression (GMR) is one of the most successful methodologies used for this purpose. A major limitation of GMR models concerns automatic selection of the proper number of model states, i.e., the number of model component densities. Existing methods, including likelihood- or entropy-based criteria, usually tend to yield noisy model size estimates while imposing heavy computational requirements. Recently, Dirichlet process (infinite) mixture models have emerged in the cornerstone of nonparametric Bayesian statistics as promising candidates for clustering applications where the number of clusters is unknown a priori. Under this motivation, to resolve the aforementioned issues of GMR-based methods for robot learning by demonstration, in this paper we introduce a nonparametric Bayesian formulation for the GMR model, the Dirichlet process GMR model. We derive an efficient variational Bayesian inference algorithm for the proposed model, and we experimentally investigate its efficacy as a robot learning by demonstration methodology, considering a number of demanding robot learning by demonstration scenarios.     

Statistical Inferences for Generalized Pareto Distribution Based on Interior Penalty Function Algorithm and Bootstrap Methods and Applications in Analyzing Stock Data

This paper studies the application of extreme value statistics (EVS) theory on analysis for stock data, based on interior penalty function algorithm and Bootstrap methods. The generalized Pareto distribution (GPD) models are considered in analyzing the closing price data of Shanghai stock market. The maximum likelihood estimates (MLEs) are obtained by using the interior penalty function algorithm. Correspondingly, the bias and standard errors of MLEs, and the hypothesis test on the shape parameter are concerned through Bootstrap methods. Some simulations are performed to demonstrate the efficacy of parameter estimation and the power of the test. The estimates of the tail index in this paper are compared with those obtained via classical methods. At last, the model is diagnosed by numerical and graphical methods and the Value-at-Risk (VaR) is estimated.     

Online Noise Estimation Using Stochastic-Gain HMM for Speech Enhancement

We propose a noise estimation algorithm for single-channel noise suppression in dynamic noisy environments. A stochastic-gain hidden Markov model (SG-HMM) is used to model the statistics of nonstationary noise with time-varying energy. The noise model is adaptive and the model parameters are estimated online from noisy observations using a recursive estimation algorithm. The parameter estimation is derived for the maximum-likelihood criterion and the algorithm is based on the recursive expectation maximization (EM) framework. The proposed method facilitates continuous adaptation to changes of both noise spectral shapes and noise energy levels, e.g., due to movement of the noise source. Using the estimated noise model, we also develop an estimator of the noise power spectral density (PSD) based on recursive averaging of estimated noise sample spectra. We demonstrate that the proposed scheme achieves more accurate estimates of the noise model and noise PSD, and as part of a speech enhancement system facilitates a lower level of residual noise.     

Adaptive sequential estimation with unknown noise statistics

Sequential estimators are derived for suboptimal adaptive estimation of the unknown a priori state and observation noise statistics simultaneously with the system state. First- and second-order moments of the noise processes are estimated based on state and observation noise samples generated in the Kalman filter algorithm. A limited memory algorithm is developed for adaptive correction of the a priori statistics which are intended to compensate for time-varying model errors. The algorithm provides improved state estimates at little computational expense when applied to an orbit determination problem for a near-earth satellite with significant modeling errors.     

A comparative study of nonparametric estimation in Weibull regression: A penalized likelihood approach

The Weibull distribution is popularly used to model lifetime distributions in many areas of applied statistics. This paper employs a penalized likelihood method to estimate the shape parameter and an unknown regression function simultaneously in a nonparametric Weibull regression. Four methods were considered: two cross-validation methods, a corrected Akaike information criterion, and a Bayesian information criterion. Each method was evaluated based on shape parameter estimation as well as selecting the smoothing parameter in a penalized likelihood model through a simulation study. Adapting a lower-dimensional approximation and deriving confidence intervals from Bayes models of the penalized likelihood, the comparative performances of methods using both censored and uncensored data were examined for various censoring rates. The methods are applied to a real data example of lung cancer.     

A local neuro-fuzzy network for high-dimensional models and optimization

In this paper a new iterative construction algorithm for local model networks is presented. The algorithm is focussed on building models with sparsely distributed data as they occur in engine optimization processes.The validity function of each local model is fitted to the available data using statistical criteria along with regularization and thus allowing an arbitrary orientation and extent in the input space. Local models are consecutively placed into those regions of the input space where the model error is still large thus guaranteeing maximal improvement through each new local model. The orientation and extent of each validity function are also adapted to the available training data such that the determination of the local regression parameters is a well-posed problem. The regularization of the model can be controlled in a distinct manner using only two user-defined parameters. In order to assess the quality of the obtained model, the algorithm also provides accurate model statistics. Different examples illustrate the efficiency of the proposed algorithm.One illustrative example shows how local models are adapted to the shape of the target function in the presence of noise. A second example shows results obtained with measurement databases of IC-engines.     

On robustness to noise of least squares based adaptive control

Current engineering practice for adaptive control schemes is to base the design on globally convergent schemes for simple plant models. An important class of such schemes uses least squares estimation of assumed simple input-output models and constructs the controller using the parameter estimates. This paper studies the robustness of such schemes to the presence of unmodelled plant coloured noise. Such noise is sometimes an adequate model for unmodelled plant dynamics.The theory of the paper makes a connection between the least squares parameter error equations and those associated with extended least squares using a posteriori noise estimates for which there are known global convergence results. For the case of adaptive minimum variance control of minimum phase plants, this connection permits stronger convergence results than those hitherto derived from the theory of extended least squares based on a priori noise estimates.     

Confidence intervals of the hazard rate function for discrete distributions using mixtures

The statistical models and methods for lifetime data mainly deal with continuous nonnegative lifetime distributions. However, discrete lifetimes arise in various common situations where either the clock time is not the best scale for measuring lifetime or the lifetime is measured discretely. In most settings involving lifetime data, the population under study is not homogenous. Mixture models, in particular mixtures of discrete distributions, provide a natural answer to this problem. Nonparametric mixtures of power series distributions are considered, as for instance nonparametric mixtures of Poisson laws or nonparametric mixtures of geometric laws. The mixing distribution is estimated by nonparametric maximum likelihood (NPML). Next, the NPML estimator is used to build estimates and confidence intervals for the hazard rate function of the discrete lifetime distribution. To improve the performance of the confidence intervals, a bootstrap procedure is considered where the estimated mixture is used for resampling. Various bootstrap confidence intervals are investigated and compared to the confidence intervals obtained directly from the NPML estimates.     

Regression analysis of belief functions on interval-valued variables: comparative studies

In real engineering, the observations of process variables are usually imprecise, uncertain, or both. In such cases, the general process modeling approaches cannot be implemented. In this paper, we investigate on the parametric and nonparametric evidential regression of imprecise and uncertain data, represented as belief function on interval-valued variables. The parametric evidential regression includes both multiple linear and nonlinear evidential regression models. The nonlinear evidential regression model is derived by introducing kernel function into the multiple linear evidential regression model. The parametric evidential regression models are identified by using evidential EM algorithm, an evidential extension of the EM algorithm. In the nonparametric evidential regression, the prediction for a given input vector is computed using a nonparametric, instance-based approach: the training samples in the neighborhood of the given input vector provide pieces of evidence reflecting the values taken by such input vector, these pieces of evidence are combined to form the prediction. Some unreliable sensor experiments are designed to validate the performances of the proposed parametric and nonparametric evidential regression models. With comparative studies, we get some interesting results.     

Analysis of quantization noise and state estimation with quantized measurements

The approximate correction of the additive white noise model in quantized Kalman filter is investigated under certain conditions. The probability density function of the error of quantized measurements is analyzed theoretically and experimentally. The analysis is based on the probability theory and nonparametric density estimation technique, respectively. The approximator of probability density function of quantized measurement noise is given. The numerical results of nonparametric density estimation algorithm demonstrate that the theoretical conclusion is reasonable. Based on the analysis of quantization noise, a novel algorithm for state estimation with quantized measurements also is proposed. The algorithm is based on the least-squares estimator and unscented transform. By least-squares estimator, the effective information is extracted from the quantized measurements. Also, using the information to update the estimated state can give a better estimation under the influence of quantization. The root mean square error (RMSE) of the proposed algorithm is compared with the RMSE of the existing methods for a typical tracking scenario in wireless sensor networks systems. Simulations provide a strong evidence that this tracking algorithm could indeed give us a more precise estimated result.     

On the Consistency of Instantaneous Rigid Motion Estimation

Instantaneous camera motion estimation is an important research topic in computer vision. Although in theory more than five points uniquely determine the solution in an ideal situation, in practice one can usually obtain better estimates by using more image velocity measurements because of the noise present in the velocity measurements. However, the usefulness of using a large number of observations has never been analyzed in detail. In this paper, we formulate this problem in the statistical estimation framework. We show that under certain noise models, consistency of motion estimation can be established: that is, arbitrarily accurate estimates of motion parameters are possible with more and more observations. This claim does not simply follow from the general consistency result for maximum likelihood estimates. Some experiments will be provided to verify our theory. Our analysis and experiments also indicate that many previously proposed algorithms are inconsistent under even very simple noise models.