Determining Sample Characteristics and Their Asymptotic Linear-Regression Properties Estimated Using Inequality Constraints

Linear regression parameters are estimated using inequality constraints. A sample estimate of the noise variance is proposed and its consistency is proved. A sample estimate of the matrix of mean-square errors in estimates of regression parameters is considered. Its consistency is proved for rather general assumptions on the noise distribution law. __________ Translated from Kibernetika i Sistemnyi Analiz, No. 3, pp. 153–165, May–June 2005.     

Estimating the matrix of root-mean-square errors of estimates of linear regression parameters for an arbitrary number of regressors and three inequality constraints

The problem of estimating linear regression parameters is considered with regard to three inequality constraints. Formulas are proposed for calculating a sample estimate for the matrix of root-mean-square error estimates of regression parameters and noise variance. __________ Translated from Kibernetika i Sistemnyi Analiz, No. 3, pp. 42–60, May–June 2006.     

Extended stochastic gradient identification algorithms for Hammerstein–Wiener ARMAX systems

An extended stochastic gradient algorithm is developed to estimate the parameters of Hammerstein–Wiener ARMAX models. The basic idea is to replace the unmeasurable noise terms in the information vector of the pseudo-linear regression identification model with the corresponding noise estimates which are computed by the obtained parameter estimates. The obtained parameter estimates of the identification model include the product terms of the parameters of the original systems. Two methods of separating the parameter estimates of the original parameters from the product terms are discussed: the average method and the singular value decomposition method. To improve the identification accuracy, an extended stochastic gradient algorithm with a forgetting factor is presented. The simulation results indicate that the parameter estimation errors become small by introducing the forgetting factor.     

Search for the upper bound of Bayesian estimates of the parameter in an exponential failure model with two known quantiles of an a priori distribution function

A fast exact algorithm of searching for the upper bound of Bayesian estimates for the parameter of the exponential distribution under the condition that an a priori distribution belongs to the class of all distribution functions with two equal quantiles. This problem arises in analyzing the sensitivity of Bayesian estimates to the choice of an a priori distribution in an exponential failure model. __________ Translated from Kibernetika i Sistemnyi Analiz, No. 1, pp. 90–102, January–February 2007.     

On the convergence of least squares estimates in white noise

The problem of convergence of least squares (LS) estimates in a stochastic linear regression model with white noise is considered. It is well known that if the parameter estimates are known to converge, the convergence analysis for many adaptive systems can be rendered considerably less arduous. For an important case where the regression vector is a measurable function of the observations and the noise is Gaussian, it has been shown, by using a Bayesian embedding argument, that the LS estimates converge almost surely for almost all true parameters in the parameter space except for a zero-measure set. However, nothing can be said about a particular given system, which is usually the objective. It has long been conjectured that such a “bad” zero measure set in the parameter space does not actually exist. A conclusive answer to this important question is provided and it is shown that the set can indeed exist. This then shows that to provide conclusive convergence results for stochastic adaptive systems, it is necessary to resort to a sample pathwise analysis instead of the Bayesian embedding approach     

Simultaneous parameter estimation and segmentation of Gibbs randomfields using simulated annealing

An adaptive segmentation algorithm is developed which simultaneously estimates the parameters of the underlying Gibbs random field (GRF)and segments the noisy image corrupted by additive independent Gaussian noise. The algorithm, which aims at obtaining the maximum a posteriori (MAP) segmentation is a simulated annealing algorithm that is interrupted at regular intervals for estimating the GRF parameters. Maximum-likelihood (ML) estimates of the parameters based on the current segmentation are used to obtain the next segmentation. It is proven that the parameter estimates and the segmentations converge in distribution to the ML estimate of the parameters and the MAP segmentation with those parameter estimates, respectively. Due to computational difficulties, however, only an approximate version of the algorithm is implemented. The approximate algorithm is applied on several two- and four-region images with different noise levels and with first-order and second-order neighborhoods     

A New Approach to Unit Root Testing

A novel simulation based approach to unit root testing is proposed in this paper. The test is constructed from the distinct orders in probability of the OLS parameter estimates obtained from a spurious and an unbalanced regression, respectively. While the parameter estimate from a regression of two integrated and uncorrelated time series is of order O _p (1), the estimate is of order O _p (T ⁻¹) if the dependent variable is stationary. The test statistic is constructed as an interquantile range from the empirical distribution obtained from regressing the standardized data sufficiently often on controlled random walks. GLS detrending (Elliott et al., Econometrica 64(4):813–836, 1996) and spectral density variance estimators (Perron and Ng, Econom Theory 14(5):560–603, 1998) are applied to account for deterministic terms and residual autocorrelation in the data. A Monte Carlo study confirms that the proposed test has favorable empirical size properties and is powerful in local-to-unity neighborhoods. As an empirical illustration, we test the purchasing power parity hypothesis for a sample of G7 economies.     

Experimental and analytic comparison of the accuracy of different estimates of parameters in a linear regression model

We consider LS-, LAD-, R-, M-, S-, LMS-, LTS-, MM-, and HBR-estimates for the parameters of a linear regression model with unknown noise distribution. With computer modeling for medium sized samples, we compare the accuracy of the considered estimates for the most popular probability distributions of noise in a regression model. For different noise distributions, we analytically compute asymptotic efficiencies of LS-, LAD-, R-, M-, S-, and LTS- estimates. We give recommendations for practical applications of these methods for different noise distributions in the model. We show examples on real datasets that support the advantages of robust estimates.     

Consistency of the extended gradient identification algorithm for multi-input multi-output systems with moving average noises

The consistency of identification algorithms for systems with colored noises is a main topic in system identification. This paper focuses on the extended stochastic gradient (ESG) identification algorithm for the multivariable linear systems with moving average noises. By integrating the noise regression terms and the noise model parameters into the information matrix and the parameter vector, and based on the gradient search principle, the ESG algorithm is presented. The unknown noise terms in the information matrix are replaced with their estimates. The convergence analysis shows that the parameter estimation error converges to zero under a persistent excitation condition. Two simulation examples are given to illustrate the effectiveness of the algorithm.     

Nonparametric estimate of almost periodic signals

The asymptotic properties of least-squares estimates of almost periodic signals under random noise are investigated. The consistency and strong consistency of the estimates are proved. __________ Translated from Kibernetika i Sistemnyi Analiz, No. 3, pp. 57–63, May–June 2007.     

Least squares estimation of nonstationary covariance parameters in linear systems

Least squares estimation techniques are employed to overcome previous difficulties encountered in accurately estimating the state and measurement noise covariance parameters in linear stochastic systems. In the past accurate and rapidly converging covariance parameter estimates have been achieved with complex estimation algorithms only after specifying the statistical nature of the noise in the system and constraining the time variation of the covariance parameters. Weighted least squares estimation allows these restrictions to be removed while achieving near optimal accuracy using a filter on the same order of complexity as a Kalman filter. Allowing the covariance parameters to vary in as general a manner in time as the state in a linear discrete time stochastic system, and assuming that a Kalman filter is applied to this system using incorrect knowledge of the a priori statistics, it is shown how a covariance system is developed similar to the original system. Unbiased least squares estimates of the covariance parameters and of the original state are obtained without the necessity of specifying the distribution on the noise in either system. The accuracy of these estimates approaches optimal accuracy with increasing measurements when adaptive Kalman filters are applied to each system.     

Multicast key distribution schemes based on Stinson-Van Trung designs

The efficiency characteristics of multicast key distribution schemes based on (v, b, r, λ)-designs are investigated. The stability and connectivity of such designs are estimated. These estimates generalize and enhance the well-known estimates for the efficiency characteristics of multicast key distribution schemes based on incomplete block designs. __________ Translated from Kibernetika i Sistemnyi Analiz, No. 3, pp. 100–111, May–June 2007.     

Consistent parameter estimation in multi-input multi-output discrete systems

An algebraic approach to the determination of consistent parameter estimates of multiple-input multiple-output (MIMO) discrete systems for general type input sequences is developed. It is shown that a special canonical structure for the system matrices exist such that the parameter estimation equations decouple into p independent subsystem equations. This results in a significant reduction in the dimension of the data matrix and allows simple algebraic equations to be derived relating system parameters, least-squares parameter estimates, and noise parameters. It is shown that the solution of these equations yields consistent parameter estimates which have rapid initial convergence towards the correct parameter values.     

Automatic methods of useful signals extraction from noise background under conditions of nonparametric uncertainty

Three basic techniques for random signals processing are under study: the problems of filtration, interpolation, and prediction. The last advances (including those of the author) in finding smoothing parameter (bandwidth) in the problems of nonparametric kernel estimation of unknown probability densities and their derivatives made it possible to advance further in the theory of the nonparametric estimation of signals with unknown distribution. This progress gave rise to the evolution of automatic methods for signals extraction from noise under the conditions of nonparametric uncertainty. The word “automatic” is understood in the sense that the suggested methods for processing signals depend only on the observable sample. In the article, by the simple example of the additive model, the comparison is made of the nonparametric procedures for the signals processing with the known optimal processing procedures obtained at the complete statistical information about the signals and noise distributions. The results of computer modeling show that the accuracy of nonparametric signals estimates insignificantly gives up to the accuracy of optimal estimates.     

Parameter identification of damping models in multibody dynamic simulation of mechanical systems

To reduce vibration and noise, a damping mechanism is often required in mechanical systems. Many types of dampers are currently used. In this paper, several typical damping models, i.e., structural damping, frictional damping, and viscoelastic damping, are illustrated, and their parameters are identified for multibody dynamic simulation. Linear damping, widely adopted for structural damping, is applied to beam deflection. Quadratic damping including air resistance is applied to plate deflection. To model stick phenomenon in mechanical dampers, a STV (stick-transition velocity) model was first introduced. To identify parameters, an optimization process is applied to the damping parameters. A new MSTV (modified stick-transition velocity) model is proposed for a friction damper. A modified Kelvin–Voight model is suggested for a rubber bushing model used in vehicle dynamics, and its parameters are identified. A modified Bouc–Wen model is also proposed; it includes the hysteretic behavior of an elastomer, and optimized results with parameter identification are compared to test results.     

Tolerance design of manipulator parameters using design of experiment approach

A robotic arm must manipulate objects with high accuracy and repeatability to perform precise tasks. There are many factors that cause variations in performance and they referred as noise factors. A probabilistic approach has been used to model the effects of noise factors and an experimental design technique has been adopted to select optimal tolerance of kinematic and dynamic parameters for minimal performance variations. The control and noise factor arrays are employed to identify statistically significant parameters and their interactions. The performance measures like signal to noise ratio and reliability have been utilized and results are validated by Monte Carlo simulations. The proposed design of experiment methodology requires minimal computations. The tolerance design methodology of manipulator is illustrated by 2-DOF revolute–revolute planar manipulator following cubic and quintic trajectory to perform a task. The statistical analysis of simulated performances is carried out using analysis of variance technique, which showed that statistically significant parameters are independent of trajectory. The individual parameter tolerance sensitivity has also been carried out.     

Enhanced parameter estimation methods for noisy SPECT data

Functional imaging with PET and SPECT is capable of visualizing subtle changes in physiological function in vivo, which aids in the early diagnosis of disease. Quantitative functional parameters are usually derived by curve fitting the dynamic data of a functional imaging study. However, the intrinsic high level of noise and low signal to noise ratio can lead to instability in the parameter estimation and give rise to non-physiological parameter estimates. Clustering techniques have been applied to improve signal to noise ratio and the reliability of parametric image generation, but these may enhance partial volume effects (PVE) and result in biased estimates for small structures. Therefore, a systematic study was performed using computer simulations of SPECT data and the generalized linear least square algorithm (GLLS) to evaluate the ability of three proposed enhanced methods and a clustering-aided method to improve the reliability of parametric image generation. The results demonstrate that clustering with sufficient cluster numbers ameliorated PVE and provided noise-insensitive parameter estimates. The enhanced GLLS method with a prior volume of distribution and bootstrap Monte Carlo resampling improved the reliability of the curve fitting, and is thus suitable for application to noisy SPECT data.     

Consistent recursive parameter estimation of partial differential equation models

This article introduces a new residual-based recursive parameter estimation algorithm for linear partial differential equations. The main idea is to replace the unmeasurable noise variables by estimates of the noise and to compute recursively both the model parameters and the noise estimates. It is proven that under some mild assumptions the estimated parameters converge to the true values with probability one. Numerical examples that demonstrate the effectiveness of the proposed approach are also provided.     

Minimum absolute deviation estimation in regression analysis

The use of ordinary least squares estimators (OLS) in regression analysis is widespread. The OLS estimates are, however, very sensitive to the presence of large disturbances. As an alternative to the OLS estimator, the minimum absolute deviation estimator (MAD) is studied. The purpose of this study is, first, to determine the effect of error distributions, with progressively heavier tails starting from the normal distribution and ending with the Cauchy distribution, on the performance of the MAD estimates and the OLS estimates. This provides a framework for when to choose the MAD estimator over the OLS estimator. Second, the effect of some of the other parameters in regression analysis, namely, the unknown parameter vector, the multicollinearity between the independent variables, and the size of the sample on the relative performance of the MAD and OLS estimators is investigated. Some guidelines regarding the choice of the MAD estimator in regression analysis are provided.     

Log-modified Weibull regression models with censored data: Sensitivity and residual analysis

This paper proposes a regression model considering the modified Weibull distribution. This distribution can be used to model bathtub-shaped failure rate functions. Assuming censored data, we consider maximum likelihood and Jackknife estimators for the parameters of the model. We derive the appropriate matrices for assessing local influence on the parameter estimates under different perturbation schemes and we also present some ways to perform global influence. Besides, for different parameter settings, sample sizes and censoring percentages, various simulations are performed and the empirical distribution of the modified deviance residual is displayed and compared with the standard normal distribution. These studies suggest that the residual analysis usually performed in normal linear regression models can be straightforwardly extended for a martingale-type residual in log-modified Weibull regression models with censored data. Finally, we analyze a real data set under log-modified Weibull regression models. A diagnostic analysis and a model checking based on the modified deviance residual are performed to select appropriate models.