Application of extreme value theory to level estimation innonlinearly distorted hidden Markov models

This paper is concerned with the application of extreme value theory (EVT) to the state level estimation problem for discrete-time, finite-state Markov chains hidden in additive colored noise and subjected to unknown nonlinear distortion. If the nonlinear distortion affects only those observations with small magnitudes or those that lie outside a finite interval, we show that the level estimation problem can be reduced to a curve fitting problem with a unique global minimum. Compared with optimum maximum likelihood estimation algorithms, the developed level estimation algorithms are computationally inexpensive and are not affected by the unknown nonlinearity as long as the extreme values of observations are not distorted. This work has been motivated by unknown deadzone and saturation nonlinearities introduced by sensors in data measurement systems. We illustrate the effectiveness of the new EVT-based level estimation algorithms with computer simulations     

Robust detection using extreme-value theory

The use of extreme-value theory (EVT) in the detection of a binary signal in additive, but statistically unknown, noise is considered. It is shown that the optimum threshold and the probability of error of the system can be accurately estimated by using EVT to obtain properties of the initial probability density functions on their "tails." Both constant signals and slowly fading signals are considered. In the case of a fading signal, the detector becomes adaptive. Detection of the constant signal, both with and without an initial learning period, is studied by computer simulation.     

Some Graphical Techniques for Estimating Weibull Confidence Intervals

Many methods for estimating the parameter and percentile statistical confidence intervals for the Weibull and Gumbel (extreme value) distributions have been described in the literature. Most of these methods depend on extensive computer programs, require reference to tables which do not cover all sample sizes of interest and/or are not widely available. This paper describes a semi-empirical technique which permits rapid estimation of the 2-sided 90% statistical confidence intervals for the Weibull or Gumbel distribution parameters, as well as for the 1, 5, 10 percentiles. The estimates can be obtained for type II censoring and sample sizes to 25. The statistical confidence intervals calculated using this method are not exact, but are very good approximations and are useful to engineers who do not have ready access to programs or lengthy tables, or who require quick estimates. If more accurate statistical confidence intervals are required, then the more complicated methods described in the references should be used.     

Data-Aided Symbol Timing Estimation for Linear Modulation

Timing phase estimation (TPE) plays the key role in feedforward (FF) symbol timing.For reasons of performance often data-aided (DA) methods are preferred. But frequently, they turn out to be critical with respect to theimplementation and the spectrum efficiency (due to the required overhead).This paper presents three DA TPE methods for quadrature pulse amplitude modulation (PAM). In spite of their very low complexity, these methods closelyapproach the theoretical limit for timing estimation with respect to the estimation variance, even at low SNR. This enables power efficient transmission.Further, they employ a CAZAC (constant amplitude, zero auto-correlation)sequence as training-sequence (TS), or a sequence with similar correlationproperties. Since such sequences are suited for almost all DA receiver tasks, a high spectrum efficiency can be obtained by the use of a single TS. A generalization of the proposed methods for DA TPE withrespect to the choice of the TS is also shown. The presented methods can be applied to noncoherent receivers. They are suited for high data-rate applications, since they can work with an oversampling factor of 2.     

Array signal Processing in the known waveform and steering vector case

The amplitude estimation of a signal that is known only up to an unknown scaling factor, with interference and noise present, is of interest in several applications, including using the emerging quadrupole resonance (QR) technology for explosive detection. In such applications, a sensor array is often deployed for interference suppression. This paper considers the complex amplitude estimation of a known waveform signal whose array response is also known a priori. Two approaches, viz., the Capon and the maximum likelihood (ML) methods, are considered for the signal amplitude estimation in the presence of temporally white but spatially colored interference and noise. We derive closed-form expressions for the expected values and mean-squared errors (MSEs) of the two estimators. A comparative study shows that the ML estimate is unbiased, whereas the Capon estimate is biased downwards for finite data sample lengths. We show that both methods are asymptotically statistically efficient when the number of data samples is large but not when the signal-to-noise ratio (SNR) is high. Furthermore, we consider a more general scenario where the interference and noise are both spatially and temporally correlated. We model the interference and noise vector as a multichannel autoregressive (AR) random process. An alternating least squares (ALS) method for parameter estimation is presented. We show that in most cases, the ALS method is superior to the model-mismatched ML (M/sup 3/L) method, which ignores the temporal correlation of the interference and noise.     

Globally convergent algorithms for estimating generalized gamma distributions in fast signal and image processing

Many applications in real-time signal, image, and video processing require automatic algorithms for rapid characterizations of signals and images through fast estimation of their underlying statistical distributions. We present fast and globally convergent algorithms for estimating the three-parameter generalized gamma distribution (G Gamma D). The proposed method is based on novel scale-independent shape estimation (SISE) equations. We show that the SISE equations have a unique global root in their semi-infinite domains and the probability that the sample SISE equations have a unique global root tends to one. The consistency of the global root, its scale, and index shape estimators is obtained. Furthermore, we establish that, with probability tending to one, Newton-Raphson (NR) algorithms for solving the sample SISE equations converge globally to the unique root from any initial value in its given domain. In contrast to existing methods, another remarkable novelty is that the sample SISE equations are completely independent of gamma and polygamma functions and involve only elementary mathematical operations, making the algorithms well suited for real-time both hardware and software implementations. The SISE estimators also allow the maximum likelihood (ML) ratio procedure to be carried out for testing the generalized Gaussian distribution (GGD) versus the G Gamma D. Finally, the fast global convergence and accuracy of our algorithms for finite samples are demonstrated by both simulation studies and real image analysis.     

Maximum likelihood estimation of the number, directions and strengths of point radio sources from variable baseline interferometer data

The optimum estimation of the number, directions, and strengths of multiple point radio sources is considered when the mutual coherence function of the sources' radiation is spatially sampled atMbaselines by a variable baseline correlation interferometer. The measurements are corrupted by the effects of additive background noise (including receiver noise) and a finite correlation time. Statistically approached, the problem is considered as a combination of parameter estimation and goodness of fit with the maximum likelihood (ML) principle being the basic criterion used. First the measurements' probability density function is derived, assuming the sources' number is known. Then the ML estimator (MLE) of the sources' parameters is obtained. The MLE's asymptotic optimum performance (unbiasedness with minimum variance) is then shown to be achieved when the number of measurements exceeds the number of sources by a threshold that is small (or zero) for most signal-to-noise ratios of interest. Next the number of sources is estimated according to a likelihood probability that measures the tenability of the MLE associated with every possible number of sources with respect to the measurements. The ML number-parameter estimation theory is then put into the form of an efficient algorithm which proves to be superior when compared to other processing methods such as Fourier maps.     

Robust parameter-estimation using the bootstrap method for the2-parameter Weibull distribution

This paper proposes bootstrap robust estimation methods for the Weibull parameters; it applies bootstrap estimators of order statistics to the parametric estimation procedure. Estimates of the Weibull parameters are equivalent to the estimates using the extreme value distribution. Therefore, the bootstrap estimators of order statistics for the parameters of the extreme value distribution are examined. Accuracy and robustness for outliers are examined by Monte Carlo experiments which indicate adequate efficiency of the proposed reliability estimators for data with some outliers     

Estimation of the energy spectrums of reflections in pulse Doppler weather radars. Part 2. Extreme performance

This is the second paper in a series of papers dedicated to the peculiarities of estimation of the continuous energy spectra of random processes of different nature, which are determined by their samples at discrete moments of time. In this paper we analyze extreme performance of the reconstruction of continuous energy spectra, in particular, the ones of interperiod fluctuations of reflections from meteorological objects in pulse Doppler weather radars under the hypothetical conditions of a priori known covariance matrix of the analyzed processes. The reasons, which cause known disadvantages of classical (nonparametric) spectral estimation (SE) methods for energy spectrums shape reconstruction, are discussed. We have considered known and suggested criteria, using which the extreme performance of classical SE methods and parametric superresolution ones has been quantitatively compared. It has been demonstrated that the extreme performance of SE methods contains important but not comprehensive information. In order to choose a SE method appropriate for operation under real-world conditions, this information should be used together with information on a corresponding method’s adaptive performance under a priori unknown statistical characteristics of input effects.     

利用特征极值比的盲信道阶数估计方法

确定性辨识方法是盲信道辨识的主流方法,然而确定性方法性能受信道阶数估计的严重影响。本文针对大多数信道阶数估计算法在坏信道条件下失效问题,分析子空间方法中噪声子空间矢量构成特殊矩阵的奇异性与信道阶数之间的关系,对该特殊矩阵最大特征值最小特征值的变化情况进行对比分析,利用特征极值的比值来反映信号子空间到噪声子空间的变化情况,从而提出特征极值比定理。针对观测数据有限且含噪声的实际应用条件,提出一种盲信道阶数估计算法,该算法以不同信道阶数的特征极值比作为参数构造目标函数,得到在真实信道阶数处目标函数取全局最大值,同时对该算法进行了复杂度分析。最后针对两种常用仿真信道参数对算法进行了验证,结果表明,在短数据和低信噪比条件下,本文算法能以较高的估计概率得到好信道和坏信道的有效阶数。      

The reconstruction of a band-limited function and its Fouriertransform from a finite number of samples at arbitrary locations bysingular value decomposition

A method for the stable interpolation of a bandlimited function known at sample instants with arbitrary locations in the presence of noise is given. Singular value decomposition is used to provide a series expansion that, in contrast to the method of sampling functions, permits simple identification of vectors in the minimum-norm space poorly represented in the sample values. Three methods, Miller regularization, least squares estimation, and maximum a posteriori estimation, are given for obtaining regularized reconstructions when noise is present. The singular value decomposition (SVD) method is used to interrelate these methods. Examples illustrating the technique are given     

Novelty detection using extreme value statistics

Extreme value theory is a branch of statistics that concerns the distribution of data of unusually low or high value, i.e. in the tails of some distribution. These extremal points are important in many applications as they represent the outlying regions of normal events against which we may wish to define abnormal events. In the context of density modelling, novelty detection or radial-basis function systems, points that lie outside of the range of expected extreme values may be flagged as outliers. There has been interest in the area of novelty detection, but decisions as to whether a point is an outlier or not tend to be made on the basis of exceeding some (heuristic) threshold. It is shown that a more principled approach may be taken using extreme value statistics     

Novelty Detection with Multivariate Extreme Value Statistics

Novelty detection, or one-class classification, aims to determine if data are “normal” with respect to some model of normality constructed using examples of normal system behaviour. If that model is composed of generative probability distributions, the extent of “normality” in the data space can be described using Extreme Value Theory (EVT), a branch of statistics concerned with describing the tails of distributions. This paper demonstrates that existing approaches to the use of EVT for novelty detection are appropriate only for univariate, unimodal problems. We generalise the use of EVT for novelty detection to the analysis of data with multivariate, multimodal distributions, allowing a principled approach to the analysis of high-dimensional data to be taken. Examples are provided using vital-sign data obtained from a large clinical study of patients in a high-dependency hospital ward.     

Extreme Value Characteristics of Distributions of Cumulative Processes

The paper introduces the concept of a cumulative stochastic process and derives the general mathematical expression of the distribution corresponding to such processes when they can be assumed to be Markovian. The behaviour of such a distribution in correspondence to accumulation functions of the type u(t) = atb and u(t) = l ln(l + t) is explored. It is shown how the exponential, Weibull, gamma, normal and lognormal distributions are particular cases of the general distribution. Next, the characteristics of the extreme values of n independent observations coming from such a general distribution are investigated. The central characteristics of the extreme values distributions are related to the hazard rate of the initial distribution. In particular, a simple method for relating the modal smallest value and the modal largest value to the sample size using the asymptotic expression of the hazard rate is given. The tail characteristics of the extreme values distributions are investigated numerically or analytically. The mathematical findings are applied to the volume effect on the failure probability of materials.     

The linear random process

A survey of the theory and applications of the class of random processes is presented. Topics discussed include the law of large numbers, covariance estimation, and the relationship of linear to normal processes. Various applications of the linear process to problems of communication theory are considered. These include prediction, signal extraction, and detection, using the linear process as a model for the signal or noise. Several exmples illustrating certain aspects of linear processes are given.     

Estimation of generalized mixtures and its application in imagesegmentation

We introduce the notion of a generalized mixture and propose some methods for estimating it, along with applications to unsupervised statistical image segmentation. A distribution mixture is said to be "generalized" when the exact nature of the components is not known, but each belongs to a finite known set of families of distributions. For instance, we can consider a mixture of three distributions, each being exponential or Gaussian. The problem of estimating such a mixture contains thus a new difficulty: we have to label each of three components (there are eight possibilities). We show that the classical mixture estimation algorithms-expectation-maximization (EM), stochastic EM (SEM), and iterative conditional estimation (ICE)-can be adapted to such situations once as we dispose of a method of recognition of each component separately. That is, when we know that a sample proceeds from one family of the set considered, we have a decision rule for what family it belongs to. Considering the Pearson system, which is a set of eight families, the decision rule above is defined by the use of "skewness" and "kurtosis". The different algorithms so obtained are then applied to the problem of unsupervised Bayesian image segmentation, We propose the adaptive versions of SEM, EM, and ICE in the case of "blind", i.e., "pixel by pixel", segmentation. "Global" segmentation methods require modeling by hidden random Markov fields, and we propose adaptations of two traditional parameter estimation algorithms: Gibbsian EM (GEM) and ICE allowing the estimation of generalized mixtures corresponding to Pearson's system. The efficiency of different methods is compared via numerical studies, and the results of unsupervised segmentation of three real radar images by different methods are presented.     

Interval estimation from censored and masked system-failure data

The author addresses confidence interval (CI) estimation in a competing risk (or multiple failure mode) framework where sample data are singly time-censored on the right and partially masked. A three-component series system with exponentially-distributed component-failure times is considered in order to represent cases involving full as well as partial masking. The approximate CIs considered are based on: asymptotic-normal theory for maximum likelihood estimators; cube-root transformation of the exponential distribution rate parameter; and inverted likelihood ratio tests. The small-sample coverage properties of these approximate CIs are estimated via computer simulation. These results also apply to models where component-failure times are Weibull distributed with known shape parameters     

Blind and semi-blind equalization: methods and algorithms

Channel identification techniques that do not require the use of a training sequence (blind methods), or that can operate with very short training sequence (semiblind methods) are a topic of major concern for modern communication applications. This paper presents a review of channel identification methods that are applicable in this context, with a strong emphasis on second-order subspace-based and maximum likelihood (Ml) estimation schemes. The main focus of the paper is on: (i) providing a clear picture of the principle and theory associated with subspace-based methods in the blind and semi-blind contexts; (ii) describing algorithmic solutions, sometimes based on novel results, that are suitable for carrying out the delicate likelihood optimization task associated withMl estimation.     

Hybrid FM-polynomial phase signal modeling: parameter estimationand Cramer-Rao bounds

Parameter estimation for a class of nonstationary signal models is addressed. The class contains combination of a polynomial-phase signal (PPS) and a frequency-modulated (FM) component of the sinusoidal or hyperbolic type. Such signals appear in radar and sonar applications involving moving targets with vibrating or rotating components. A novel approach is proposed that allows us to decouple estimation of the FM parameters from those of the PPS, relying on properties of the multilag high-order ambiguity function (ml-HAF). The accuracy achievable by any unbiased estimator of the hybrid FM-PPS parameters is investigated by means of the Cramer-Rao lower bounds (CRLBs). Both exact and large sample approximate expressions of the bounds are derived and compared with the performance of the proposed methods based on Monte Carlo simulations     

基于贝叶斯压缩感知的子空间拟合DOA估计方法

经典加权子空间拟合算法需进行多维非线性优化,初始参数的难以设置和较大的计算量限制了其应用。结合压缩感知理论,本文提出了一种基于改进贝叶斯压缩感知的子空间拟合DOA估计新方法。该方法首先通过低复杂度的子空间分解算法PASTd估计信号加权子空间,进而基于入射信号的空域稀疏性,将信号子空间拟合建模为多测量值稀疏重构问题,并应用贝叶斯压缩感知算法进行求解。算法在贝叶斯压缩感知的迭代求解中引入了基于相对阈值判决的基消除机制,加快收敛速度的同时避免了矩阵奇异问题。仿真结果表明本文算法在低信噪比、小快拍情况下空间分辨率优于MUSIC和l1-SVD算法,可直接用于相干源的估计,并对信源数目的估计误差具有较强鲁棒性。