Blind Minimax Estimation

We consider the linear regression problem of estimating an unknown, deterministic parameter vector based on measurements corrupted by colored Gaussian noise. We present and analyze blind minimax estimators (BMEs), which consist of a bounded parameter set minimax estimator, whose parameter set is itself estimated from measurements. Thus, our approach does not require any prior assumption or knowledge, and the proposed estimator can be applied to any linear regression problem. We demonstrate analytically that the BMEs strictly dominate the least-squares (LS) estimator, i.e., they achieve lower mean-squared error (MSE) for any value of the parameter vector. Both Stein's estimator and its positive-part correction can be derived within the blind minimax framework. Furthermore, our approach can be readily extended to a wider class of estimation problems than Stein's estimator, which is defined only for white noise and nontransformed measurements. We show through simulations that the BMEs generally outperform previous extensions of Stein's technique.     

Minimax estimation of deterministic parameters in linear models with a random model matrix

We consider the problem of estimating an unknown deterministic parameter vector in a linear model with a random model matrix, with known second-order statistics. We first seek the linear estimator that minimizes the worst-case mean-squared error (MSE) across all parameter vectors whose (possibly weighted) norm is bounded above. We show that the minimax MSE estimator can be found by solving a semidefinite programming problem and develop necessary and sufficient optimality conditions on the minimax MSE estimator. Using these conditions, we derive closed-form expressions for the minimax MSE estimator in some special cases. We then demonstrate, through examples, that the minimax MSE estimator can improve the performance over both a Baysian approach and a least-squares method. We then consider the case in which the norm of the parameter vector is also bounded below. Since the minimax MSE approach cannot account for a nonzero lower bound, we consider, in this case, a minimax regret method in which we seek the estimator that minimizes the worst-case difference between the MSE attainable using a linear estimator that does not know the parameter vector, and the optimal MSE attained using a linear estimator that knows the parameter vector. For analytical tractability, we restrict our attention to the scalar case and develop a closed-form expression for the minimax regret estimator.     

Minimax estimation of unknown deterministic signals in colorednoise

The estimation of a deterministic signal corrupted by random noise is considered. The strategy is to find a linear noncausal estimator which minimizes the maximum mean square error over an a priori set of signals. This signal set is specified in terms of frequency/energy constraints via the discrete Fourier transform. Exact filter expressions are given for the case of additive white noise. For the case of additive colored noise possessing a continuous power spectral density, a suboptimal filter is derived whose asymptotic performance is optimal. Asymptotic expressions for the minimax estimator error are developed for both cases. The minimax filter is applied to random data and is shown to solve asymptotically a certain worst-case Wiener filter problem     

Minimax MSE-ratio estimation with signal covariance uncertainties

In continuation to an earlier work, we further consider the problem of robust estimation of a random vector (or signal), with an uncertain covariance matrix, that is observed through a known linear transformation and corrupted by additive noise with a known covariance matrix. While, in the earlier work, we developed and proposed a competitive minimax approach of minimizing the worst-case mean-squared error (MSE) difference regret criterion, here, we study, in the same spirit, the minimum worst-case MSE ratio regret criterion, namely, the worst-case ratio (rather than difference) between the MSE attainable using a linear estimator, ignorant of the exact signal covariance, and the minimum MSE (MMSE) attainable by optimum linear estimation with a known signal covariance. We present the optimal linear estimator, under this criterion, in two ways: The first is as a solution to a certain semidefinite programming (SDP) problem, and the second is as an expression that is of closed form up to a single parameter whose value can be found by a simple line search procedure. We then show that the linear minimax ratio regret estimator can also be interpreted as the MMSE estimator that minimizes the MSE for a certain choice of signal covariance that depends on the uncertainty region. We demonstrate that in applications, the proposed minimax MSE ratio regret approach may outperform the well-known minimax MSE approach, the minimax MSE difference regret approach, and the "plug-in" approach, where in the latter, one uses the MMSE estimator with an estimated covariance matrix replacing the true unknown covariance.     

Minimax MSE estimation of deterministic parameters with noise covariance uncertainties

In this paper, a minimax mean-squared error (MSE) estimator is developed for estimating an unknown deterministic parameter vector in a linear model, subject to noise covariance uncertainties. The estimator is designed to minimize the worst-case MSE across all norm-bounded parameter vectors, and all noise covariance matrices, in a given region of uncertainty. The minimax estimator is shown to have the same form as the estimator that minimizes the worst-case MSE over all norm-bounded vectors for a least-favorable choice of the noise covariance matrix. An example demonstrating the performance advantage of the minimax MSE approach over the least-squares and weighted least-squares methods is presented.     

Robust mean-squared error estimation in the presence of model uncertainties

We consider the problem of estimating an unknown parameter vector x in a linear model that may be subject to uncertainties, where the vector x is known to satisfy a weighted norm constraint. We first assume that the model is known exactly and seek the linear estimator that minimizes the worst-case mean-squared error (MSE) across all possible values of x. We show that for an arbitrary choice of weighting, the optimal minimax MSE estimator can be formulated as a solution to a semidefinite programming problem (SDP), which can be solved very efficiently. We then develop a closed form expression for the minimax MSE estimator for a broad class of weighting matrices and show that it coincides with the shrunken estimator of Mayer and Willke, with a specific choice of shrinkage factor that explicitly takes the prior information into account. Next, we consider the case in which the model matrix is subject to uncertainties and seek the robust linear estimator that minimizes the worst-case MSE across all possible values of x and all possible values of the model matrix. As we show, the robust minimax MSE estimator can also be formulated as a solution to an SDP. Finally, we demonstrate through several examples that the minimax MSE estimator can significantly increase the performance over the conventional least-squares estimator, and when the model matrix is subject to uncertainties, the robust minimax MSE estimator can lead to a considerable improvement in performance over the minimax MSE estimator.     

Robust weighted likelihood estimation of exponential parameters

The problem of estimating the parameter of an exponential distribution when a proportion of the observations are outliers is quite important to reliability applications. The method of weighted likelihood is applied to this problem, and a robust estimator of the exponential parameter is proposed. Interestingly, the proposed estimator is an /spl alpha/-trimmed mean type estimator. The large-sample robustness properties of the new estimator are examined. Further, a Monte Carlo simulation study is conducted showing that the proposed estimator is, under a wide range of contaminated exponential models, more efficient than the usual maximum likelihood estimator in the sense of having a smaller risk, a measure combining bias & variability. An application of the method to a data set on the failure times of throttles is presented.     

A Minimax Chebyshev Estimator for Bounded Error Estimation

We develop a nonlinear minimax estimator for the classical linear regression model assuming that the true parameter vector lies in an intersection of ellipsoids. We seek an estimate that minimizes the worst-case estimation error over the given parameter set. Since this problem is intractable, we approximate it using semidefinite relaxation, and refer to the resulting estimate as the relaxed Chebyshev center (RCC). We show that the RCC is unique and feasible, meaning it is consistent with the prior information. We then prove that the constrained least-squares (CLS) estimate for this problem can also be obtained as a relaxation of the Chebyshev center, that is looser than the RCC. Finally, we demonstrate through simulations that the RCC can significantly improve the estimation error over the CLS method.     

Comparing between estimation approaches: admissible and dominating linear estimators

We treat the problem of evaluating the performance of linear estimators for estimating a deterministic parameter vector x in a linear regression model, with the mean-squared error (MSE) as the performance measure. Since the MSE depends on the unknown vector x, a direct comparison between estimators is a difficult problem. Here, we consider a framework for examining the MSE of different linear estimation approaches based on the concepts of admissible and dominating estimators. We develop a general procedure for determining whether or not a linear estimator is MSE admissible, and for constructing an estimator strictly dominating a given inadmissible method so that its MSE is smaller for all x. In particular, we show that both problems can be addressed in a unified manner for arbitrary constraint sets on x by considering a certain convex optimization problem. We then demonstrate the details of our method for the case in which x is constrained to an ellipsoidal set and for unrestricted choices of x. As a by-product of our results, we derive a closed-form solution for the minimax MSE estimator on an ellipsoid, which is valid for arbitrary model parameters, as long as the signal-to-noise-ratio exceeds a certain threshold.     

Shrinkage estimation of scale parameter of the extreme-valuedistribution

The author studies the usual preliminary test estimator of the scale parameter of the extreme-value distribution in censored samples. The optimum levels of significance and their corresponding critical values for the preliminary test are obtained based on the minimax regret criterion. A preliminary test shrinkage estimator that is smoother than the usual preliminary test estimator is proposed as well. The optimum values of shrinkage coefficients for the preliminary test shrinkage estimator are obtained, and are also based on the minimax regret criterion. Comparison of these two estimators shows that if the mean square error is a criterion of goodness of estimation then the preliminary test shrinkage estimator is better than the usual preliminary test estimator     

Linear minimax regret estimation of deterministic parameters with bounded data uncertainties

We develop a new linear estimator for estimating an unknown parameter vector x in a linear model in the presence of bounded data uncertainties. The estimator is designed to minimize the worst-case regret over all bounded data vectors, namely, the worst-case difference between the mean-squared error (MSE) attainable using a linear estimator that does not know the true parameters x and the optimal MSE attained using a linear estimator that knows x. We demonstrate through several examples that the minimax regret estimator can significantly increase the performance over the conventional least-squares estimator, as well as several other least-squares alternatives.     

Bayesian Estimation of Parameters of Life Distributions and Reliability from Type II Censored Samples

The Bayesian approach to reliability estimation from Type II censored samples is discussed here with emphasis on obtaining natural conjugate prior distributions. The underlying sampling distribution from which the censored samples are drawn follows a generalized life model (GLM) which includes a model proposed by Epstein and Sobel, Weibull, exponential, and Rayleigh distributions as special cases. Results are given for the Type II asymptotic distribution of largest values, Pareto, and Limited distribution. The natural conjugate prior, Bayes estimate for the generalized scale parameter, posterior risk, Bayes risk and Bayes estimate of the reliability function were derived for the distributions studied. In every case the natural conjugate prior is a 2-parameter family which provides a wide range of possible prior knowledge. Conjugate diffuse priors were derived. A diffuse prior, also called a quasi-pdf, is not a pdf because its integral is not unity. It represents roughly an informationless prior state of knowledge. The proper choice of the parameter for the diffuse prior leads to maximum likelihood, classical uniform minimum-variance unbiased estimator, and an admissible biased estimator with minimum mean square error as the generalized Bayes estimate. A feature of the GLM is the increasing function g(·) with possible applications in accelerated testing. KG(·) is a s-complete s-sufficient statistic for ?, and KG(·)/m is a maximum likelihood estimate for ?. Similar results were obtained for the Pareto, Type II asymptotic distribution of extremes, Pareto (associated with Pearl-Reed growth distribution) and others.     

Filtering for bimodal systems: the case of unknown switching statistics

This paper considers the problem of state estimation for discrete-time systems whose dynamics randomly switches between two linear stochastic behaviors (bimodal systems). The novelty of this paper is that no statistical information on the switching process is assumed available for the filter design. Two different approaches are here proposed to solve the estimation problem in these conditions. One method is based on a combined use of stochastic singular systems and of the minimax filtering theory, while the other relies on the maximum entropy principle. Based on these approaches two filtering algorithms are derived, whose features are theoretically and numerically compared. Some attention has been devoted to the study of the asymptotic properties of both the filters.     

Linear Minimax Filtering of a Stationary Random Process under the Condition of the Interval Fuzziness in the State Matrix of the System with a Restricted Variance

The equations for the transition functions of optimal minimax filters are derived under the condition of the interval fuzziness of the linear dynamic system with the parametric uncertain assignment of only the state matrix in the context of Hurwitz stability. The method by which the required filter is synthesized as the suboptimal minimax filter of the stable and unstable first-order systems with the given degree of stability is discussed.     

A Scaling Parameter Approach to Delay-Dependent State Estimation of Delayed Neural Networks

This brief is concerned with studying the delay-dependent state estimation problem of recurrent neural networks with time-varying delay. The neuron activation function is more general than the sigmoid functions, and the time-varying delay is allowed to vary fast with time. A scaling parameter based approach is proposed, and a delay-dependent criterion is derived under which the resulting error system is globally asymptotically stable. It is shown that the design of a proper state estimator is directly accomplished by means of the feasibility of a linear matrix inequality. Thanks to the introduction of a scaling parameter, the developed result can efficiently be applied to chaotic delayed neural networks.      

Estimation of Constrained Parameters With Guaranteed MSE Improvement

We address the problem of estimating an unknown parameter vector x in a linear model y=Cx+v subject to the a priori information that the true parameter vector x belongs to a known convex polytope X. The proposed estimator has the parametrized structure of the maximum a posteriori probability (MAP) estimator with prior Gaussian distribution, whose mean and covariance parameters are suitably designed via a linear matrix inequality approach so as to guarantee, for any xisinX, an improvement of the mean-squared error (MSE) matrix over the least-squares (LS) estimator. It is shown that this approach outperforms existing "superefficient" estimators for constrained parameters based on different parametrized structures and/or shapes of the parameter membership region X     

A game theory approach to constrained minimax state estimation

This paper presents a game theory approach to the constrained state estimation of linear discrete time dynamic systems. In the application of state estimators, there is often known model or signal information that is either ignored or dealt with heuristically. For example, constraints on the state values (which may be based on physical considerations) are often neglected because they do not easily fit into the structure of the state estimator. This paper develops a method for incorporating state equality constraints into a minimax state estimator. The algorithm is demonstrated on a simple vehicle tracking simulation.     

Minimax emission computed tomography using high-resolutionanatomical side information and B-spline models

In this paper a minimax methodology is presented for combining information from two imaging modalities having different intrinsic spatial resolutions. The focus application is emission computed tomography (ECT), a low-resolution modality for reconstruction of radionuclide tracer density, when supplemented by high-resolution anatomical boundary information extracted from a magnetic resonance image (MRI) of the same imaging volume. The MRI boundary within the two-dimensional (2-D) slice of interest is parameterized by a closed planar curve. The Cramer-Rao (CR) lower bound is used to analyze estimation errors for different boundary shapes. Under a spatially inhomogeneous Gibbs field model for the tracer density a representation for the minimax MRI-enhanced tracer density estimator is obtained. It is shown that the estimator is asymptotically equivalent to a penalized maximum likelihood (PML) estimator with resolution-selective Gibbs penalty. Quantitative comparisons are presented using the iterative space alternating generalized expectation maximization (SAGE-FM) algorithm to implement the PML estimator with and without minimax weight averaging     

Stabilization of Multidimensional Systems Using Local State Estimators

A design method for stabilizing a multidimensional system isproposed. The deign method is developed from the viewpoint ofLyapunov stability and it is based on two Riccati like matrixinequalities those ensure the stabilizability and the detectabilityof a multidimensional system in a strong sense. The arrangementscheme of the stabilizer is set up as a combination of plantand local state estimator, which imitates the well-known compensatordesign for 1-D systems incorporated with full order state observer.     

Approximate MLEs for the location and scale parameters of thehalf-logistic distribution with type-II right-censoring

For the half-logistic distribution the maximum likelihood method does not provide an explicit estimator for the scale parameter based on either complete or right-censored samples. The authors provide a simple method of deriving an explicit estimator by approximating the likelihood function. The bias and variance of this estimator are studied, and it is shown that this estimator is as efficient as the best linear unbiased estimator. An example to illustrate the method is presented