Optimum quantization of order statistics

We present necessary conditions for minimum mean square error (MMSE) quantization of order statistics. These conditions are obtained by observing that in a quantize-sort cascade system, operations of quantization and sorting are interchangeable. We consider MMSE quantization of a single order statistic as well as of a group of order statistics and highlight several interesting properties. Numerical characteristics of order statistic quantizers are presented for Gaussian and uniform distributions, and detailed comparisons are made     

Information properties of order statistics and spacings

We explore properties of the entropy, Kullback-Leibler information, and mutual information for order statistics. The probability integral transformation plays a pivotal role in developing our results. We provide bounds for the entropy of order statistics and some results that relate entropy ordering of order statistics to other well-known orderings of random variables. We show that the discrimination information between order statistics and data distribution, the discrimination information among the order statistics, and the mutual information between order statistics are all distribution free and are computable using the distributions of the order statistics of the samples from the uniform distribution. We also discuss information properties of spacings for uniform and exponential samples and provide a large sample distribution-free result on the entropy of spacings. The results show interesting symmetries of information orderings among order statistics.     

Low-rank estimation of higher order statistics

Low-rank estimators for higher order statistics are considered in this paper. The bias-variance tradeoff is analyzed for low-rank estimators of higher order statistics using a tensor product formulation for the moments and cumulants. In general, the low-rank estimators have a larger bias and smaller variance than the corresponding full-rank estimator, and the mean-squared error can be significantly smaller. This makes the low-rank estimators extremely useful for signal processing algorithms based on sample estimates of the higher order statistics. The low-rank estimators also offer considerable reductions in the computational complexity of such algorithms. The design of subspaces to optimize the tradeoffs between bias, variance, and computation is discussed, and a noisy input, noisy output system identification problem is used to illustrate the results     

Dynamic system identification with order statistics

Systems consisting of linear dynamic and memory-less nonlinear subsystems are identified. The paper deals with systems in which the nonlinear element is followed by a linear element, as well as systems in which the subsystems are connected in parallel. The goal of the identification is to recover the nonlinearity from noisy input-output observations of the whole system; signals interconnecting the elements are not measured. Observed values of the input signal are rearranged in increasing order, and coefficients for the expansion of the nonlinearity in trigonometric series are estimated from the new sequence of observations obtained in this way. Two algorithms are presented, and their mean integrated square error is examined. Conditions for pointwise convergence are also established. For the nonlinearity satisfying the Lipschitz condition, the error converges to zero. The rate of convergence derived for differentiable nonlinear characteristics is insensitive to the roughness of the probability density of the input signal. Results of numerical simulation are also presented     

Motion estimation using higher order statistics

The objective of this paper is to introduce a fourth-order cost function of the displaced frame difference (DFD) capable of estimating motion even for small regions or blocks. Using higher than second-order statistics is appropriate in case the image sequence is severely corrupted by additive Gaussian noise. Some results are presented and compared to those obtained from the mean kurtosis and the mean square error of the DFD.     

Higher order statistics of antenna subset diversity

We use a virtual-branch technique to derive higher order statistics of the output signal-to-noise ratio (SNR) of antenna subset diversity in a multipath fading environment. In particular, closed-form expressions of the cumulants, central moments, skewness, and kurtosis for various antenna subset diversity schemes are derived. These measures characterize the statistical behavior of the output SNR distribution.     

Time delay estimation using higher order statistics

The problem of time delay estimation in spatially correlated Gaussian noises is addressed. New normal equations based upon higher order statistics of the sensor measurements are established, from which a new adaptive algorithm for time delay estimation is proposed. Simulation examples are presented to illustrate the effectiveness of this algorithm     

Color edge detection using vector order statistics

A method is proposed whereby a color image is treated as a vector field and the edge information carried directly by the vectors is exploited. A class of color edge detectors is defined as the minimum over the magnitudes of linear combinations of the sorted vector samples. From this class, a specific edge detector is obtained and its performance characteristics studied. Results of a quantitative evaluation and comparison to other color edge detectors, using Pratt's (1991) figure of merit and an artificially generated test image, are presented. Edge detection results obtained for real color images demonstrate the efficiency of the detector.     

A structure for adaptive order statistics filtering

In applications such as smoothing and enhancement of images, adaptive filtering techniques offer the flexibility needed for good performance with non-stationary observations. Many adaptive schemes can be based on the idea of determining the local statistics of the signal through appropriate tests on the data, to aid in the selection of a filtering procedure that is suited to the data. In the paper, the authors consider decision-directed or data-dependent adaptive filtering schemes that are based on order statistics. A general formulation for such a class of adaptive order statistics filters is presented. Approximate statistical performance analysis, especially in the presence of edges, may be carried out for this entire class of filters. The authors give examples of some existing filters that fit into this framework. The formulation also accommodates filters that employ multiple windows in their operation. To illustrate the potential of this class of multiple window (MW) filters, they construct and analyze simple filters, like the triple window median (TW-MED) and the triple window median of means (TW-MOM) filters, that are shown to yield useful performance. The class of mean-median hybrid (MMH) filters is also presented as a simple example which may be extended to give interesting performance.     

On higher order statistics of the Nakagami-m distribution

This paper proposes a fading model that leads to a formal, but simple method to obtain the exact formula of the Nakagami-m published in 1960 distribution for m=n/2, with n a nonzero integer number. Based on such a model, the joint distribution of the envelope and its time derivative are accomplished, and exact formulas for the level crossing rate (closed-form formula) and for the average fade duration are derived. Simulation curves and exact formulas are checked against each other and a very good agreement between them is attained. Although derived for discrete values of m (m being an integer multiple of 1/2), there are no mathematical constraints for these expressions to be used for any real value of m⩾1/2     

Characterization of sleep spindles using higher order statistics and spectra

This work characterizes the dynamics of sleep spindles, observed in electroencephalogram (EEG) recorded from humans during sleep, using both time and frequency domain methods which depend on higher order statistics and spectra. The time domain method combines the use of second- and third-order correlations to reveal information on the stationarity of periodic spindle rhythms to detect transitions between multiple activities. The frequency domain method, based on normalized spectrum and bispectrum, describes frequency interactions associated with nonlinearities occurring in the observed EEG.     

Detection and classification of spectrally equivalent processesusing higher order statistics

This paper addresses the problem of detecting two spectrally equivalent processes, which are modeled by a noisy AR process and an ARMA process. Higher order statistics (HOS) are shown to be efficient tools for detection. Two HOS-based detectors are derived for the binary hypothesis testing problem (i.e., known signal spectrum): The first detector exploits the asymptotic Gaussianity of the sample estimates of the cumulants. The second detector exploits the singularity of a certain matrix based on HOS. The more general composite hypothesis testing problem (i.e., with unknown signal spectrum) is then considered and a detector proposed. The performances of the different detectors are compared in terms of receiver operating characteristic (ROC) curves. Approximate closed-form expressions are derived for the threshold and the ROCs of the three detectors     

Computationally efficient soft-decision decoding of linear blockcodes based on ordered statistics

Soft-decision decoding of a linear block code using the most reliable basis corresponding to each received word is investigated. Based either on probabilistic properties or on the structure of the code considered, three improvements to the algorithm devised by Fossorier and Lin (see ibid., vol.41, no.9, p.1379-1396, 1995) are presented. These modifications allow large computation savings or significant decoding speedup with little error performance degradation. First, a reduced probabilistic list of codeword candidates is associated with order-i reprocessing of a given code. It results in a large reduction of the maximum number of computations with a very small degradation in performance. Then, a probabilistic stopping criterion is introduced for order-0 reprocessing. This new test significantly decreases the average number of computations when appropriately implemented. Finally, the application of the algorithm to coset decoding is considered for |u|u+v| constructed codes. In addition to the conventional coset decoding, a new adaptive practically optimum coset decoding method is presented where at each reprocessing stage, the number of surviving cosets decreases. Suboptimum closest coset decoding is also investigated. It is shown that two-stage decoding with the algorithm of Fossorier and Lin offers a large variety of choices, since the reprocessing order of each stage can be determined independently     

A new impulse detector based on order statistics

The acquisition or transmission of digital images through sensors or communication channels is often interfered by impulse noise. It is very important to eliminate noise in the images before subsequent processing, such as image segmentation, object recognition, and edge detection. In this letter, a novel impulse noise-detection algorithm is presented, which can remove impulse noise from corrupted images successfully, and at the same time, without eliminating image details. The algorithm is based on the order statistics within a local window. Although our method is low in complexity when compared with some other complicated algorithms, experimental results show that it produces better restored images than many other existing techniques.     

A theorem on the entropy of certain binary sequences and applications--II

In this, the second part of a two-part paper, we apply the result of Part I [9] to three problems in multi-user communication. 1) For the special case of the broadcast channel where both channels are binary symmetric, we establish a converse result that shows the ad hoc scheme suggested by Cover and Bergmans [2], [3l is in fact optimal. 2) For a modified version of a source coding problem with side information of Slepian and Wolf [3], we establish a converse result. 3) We show that the "common information" of a certain pair of dependent random variables is zero.     

A theorem on the entropy of certain binary sequences and applications--I

In this, the first part of a two-part paper, we establish a theorem concerning the entropy of a certain sequence of binary random variables. In the sequel we will apply this result to the solution of three problems in multi-user communication, two of which have been open for some time. Specifically we show the following. LetXandYbe binary randomn-vectors, which are the input and output, respectively, of a binary symmetric channel with "crossover" probabilityp_0. LetH{X}andH{ Y}be the entropies ofXandY, respectively. Then begin{equation} begin{split} frac{1}{n} H{X} geq h(alpha_0), qquad 0 leq alpha_0 &leq 1, Rightarrow \ qquad qquad &qquad frac{1}{n}H{Y} geq h(alpha_0(1 - p_0) + (1 - alpha_0)p_0) end{split} end{equation} whereh(lambda) = -lambda log lambda - (1 - lambda) log(l - lambda), 0 leq lambda leq 1.     

基于样本熵的DNA序列相似性分析

针对传统方法在分析DNA序列相似性方面的不足,提出了一种基于样本熵的DNA序列相似性分析方法.以5种东亚钳蝎神经毒素的基因序列作为分析对象,首先通过DNA序列的图形表示把DNA序列转换为时间序列,然后运用样本熵算法计算出时间序列的样本熵值,将样本熵的互值大小作为分析序列之间相似性的依据,最后将样本熵方法与DTW(Dynamic Time Warping,动态时间弯曲)方法的实验结果进行比较.实验结果表明,样本熵分析方法能有效分析序列之间的相似性,与DTW分析方法相比较,显示出更强的相似性和区别度,可将其进一步应用于生物序列的分析.     

General formula for conditional mean using higher order statistics

A general formula is given for conditional mean, which is the well-known minimum variance estimator, in terms of cross-cumulants and locally optimum nonlinearities. This formula will greatly facilitate the solution of various problems emerging in nonlinear system identification, non-Gaussian estimation theory, and nonlinear signal processing