交叉熵约束的红外图像最小错误阈值分割

针对目标和背景具有相似统计分布的红外图像,经典阈值分割方法仅以某种形式的方差或熵作为准则,未考虑图像的实际特性,分割效果不甚理想。为此,提出了一种基于交叉熵约束的红外图像最小错误阈值分割新方法。首先,引入交叉熵来度量目标和背景统计分布的相似性,交叉熵越小表明分布越相似;然后在交叉熵小于一定值的条件下使分类错误达到最小。交叉熵的约束保证了分割过程适应红外图像实际特性,分类错误最小确保了分割效果的有效性。该方法原理清晰、参数设置简单,在一系列实际图像上的实验结果表明,与现有几种经典阈值分割方法相比,文中方法有效提高了目标和背景具有相似统计分布的红外图像的阈值分割准确率。     

射电成像中电离层扰动的最小熵校正方法

在合成孔径射电成像中,电离层的扰动会在接收信号中引入相位误差,导致图像出现模糊和漂移。本文提出了一种新方法,以校正这种相位误差。使用熵作为衡量射电天文图像质量的指标,熵越小代表图像质量越高,当熵值达到最小时,认为相位误差被校正。相比其他传统方法,本方法仅利用脏图本身就能够校正相位误差。     

二态隐马尔可夫过程熵率的逼近算法

基于熵率上下界收敛性,该文提出了一个算法以计算二态隐马尔可夫过程的熵率.该算法能以任意精度逼近熵率的理论值,且可计算最大偏差.算法的复杂度的对数和误差的对数为线性关系,因此其计算代价是可以接受的.该算法为计算一般隐马尔可夫模型的熵率提供了一种新途径.     

Near-lossless image compression: minimum-entropy, constrained-errorDPCM

A near-lossless image compression scheme is presented. It is essentially a differential pulse code modulation (DPCM) system with a mechanism incorporated to minimize the entropy of the quantized prediction error sequence. With a "near-lossless" criterion of no more than a d gray-level error for each pixel, where d is a small nonnegative integer, trellises describing all allowable quantized prediction error sequences are constructed. A set of "contexts" is defined for the conditioning prediction error model and an algorithm that produces minimum entropy conditioned on the contexts is presented. Finally, experimental results are given.     

A new interpretation on the MMSE as a robust MEE criterion

The minimum mean-square error (MMSE) and minimum error entropy (MEE) are two important criteria in the estimation related problems. The MMSE can be viewed as a robust MEE criterion in the minimax sense, as its minimization is equivalent to minimizing an upper bound (the maximum value) of the error entropy. This note gives a new and more meaningful interpretation on the robustness of MMSE for problems in which there exists uncertainty in the probability model. It is shown that the MMSE estimator imposes an upper bound on error entropy for the true model. The upper bound consists of two terms. The first term quantifies the “MMSE performance” under nominal conditions, and the second term measures the “distance” between the true and nominal models. This robustness property is parallel to that of the risk-sensitive estimation. Illustration examples are included to confirm the robustness of MMSE.     

太赫兹ISAR成像系统误差校正算法

针对非线性系统误差对太赫兹雷达成像质量的影响,提出一种最小熵系统误差校正算法。在实测的太赫兹逆合成孔径雷达成像实验中,非线性误差会对回波相位产生影响,从而使得脉压后的距离像能量分散,进而降低成像质量。经过对误差形式的理论分析,建立一维距离像的相位误差补偿模型,并基于最小熵的优化准则迭代校正此系统误差。实验结果表明,与基于参考点目标的方法相比,所提方法自适应性更强,且具有更好的校正效果。     

The minimum error entropy based robust wireless channel tracking inimpulsive noise

This article concerns the problem of adaptive wireless channel tracking in the non-Gaussian α-stable noise. By assuming a primitive Cauchy distribution for the estimate error and minimizing the entropy of error, we develop the least entropy of error (LEE) based wireless channel tracking algorithm and the second-order LEE (SOLEE) algorithm. Simulation results show that both algorithms are robust to impulsive noise and such robustness can be achieved without any performance loss in the Gaussian noise     

Hessian and Concavity of Mutual Information, Differential Entropy, and Entropy Power in Linear Vector Gaussian Channels

Within the framework of linear vector Gaussian channels with arbitrary signaling, the Jacobian of the minimum mean square error and Fisher information matrices with respect to arbitrary parameters of the system are calculated in this paper. Capitalizing on prior research where the minimum mean square error and Fisher information matrices were linked to information-theoretic quantities through differentiation, the Hessian of the mutual information and the entropy are derived. These expressions are then used to assess the concavity properties of mutual information and entropy under different channel conditions and also to derive a multivariate version of an entropy power inequality due to Costa.     

图像融合的量化评价方法及实验分析

图像融合是图像信息的综合处理,是将同一对象的一个或更多的图像合成在一幅图像中,以便比原来的任何一幅图像更容易地为人所理解。介绍了图像融合的各种评价方法,包括:熵、交叉熵、交互信息量、均方误差、均方根误差、峰值信噪比。本文利用sym4小波变换的方法进行融合,将融合结果用各种量化评价方法进行了比较,结果表明交叉熵与交互信息量是行之有效的评价方法。     

Entropy-constrained scalar quantization and minimum entropy witherror bound by discrete wavelet transforms in image compression

The global maximum of an entropy function with different decision levels for a three-level scalar quantizer performed after a discrete wavelet transform was derived. Herein, we considered the case of entropy-constrained scalar quantization capable of avoiding many compression ratio reductions as the mean squared error was minimized. We also dealt with the problem of minimum entropy with an error bound, which was referred to as the rate distortion function. For generalized Gaussian distributed input signals, the Shannon bound would decrease monotonically when the parameter of distribution γ was to leave from 2. That is Gaussian distributions would contain the highest Shannon bound among the generalized Gaussian distributions. Additionally, we proposed two numerical approaches of the secant and false position methods implemented in real cases to solve the problems of entropy-constrained scalar quantization and minimum entropy with an error bound. The convergence condition of the secant method was also addressed     

Entropy minimization for supervised digital communications channelequalization

This paper investigates the application of error-entropy minimization algorithms to digital communications channel equalization. The pdf of the error between the training sequence and the output of the equalizer is estimated using the Parzen windowing method with a Gaussian kernel, and then, the Renyi's quadratic entropy is minimized using a gradient descent algorithm. By estimating Renyi's entropy over a short sliding window, an online training algorithm is also introduced. Moreover, for a linear equalizer, an orthogonality condition for the minimum entropy solution that leads to an alternative fixed-point iterative minimization method is derived. The performance of linear and nonlinear equalizers trained with entropy and mean square error (MSE) is compared. As expected, the results of training a linear equalizer are very similar for both criteria since, even if the input noise is non-Gaussian, the output filtered noise tends to be Gaussian. On the other hand, for nonlinear channels and using a multilayer perceptron (MLP) as the equalizer, differences between both criteria appear. Specifically, it is shown that the additional information used by the entropy criterion yields a faster convergence in comparison with the MSE     

Uniqueness of a two-step predictor based spectral estimator thatgeneralizes the maximum entropy concept

Given a finite set of autocorrelations, it is well known that maximization of the entropy functional subject to this data leads to a stable autoregressive model. Since maximization of the entropy functional is equivalent to maximization of the minimum mean square error associated with one-step predictors, the problem of obtaining admissible extensions that maximize the k-step minimum-mean-square prediction error subject to the given autocorrelations has been shown to result in stable autoregressive moving-average (ARMA) extensions. The uniqueness of this true generalization of the maximum-entropy extension is proved here by a constructive procedure in the case of two-step predictors     

Quantizing for maximum output entropy (Corresp.)

The entropy at the output of a quantizer is equal to the average mutual information between unquantized and quantized random variables. Thus, for a fixed number of quantization levels, output entropy is a reasonable information-theoretic criterion of quantizer fidelity. It is shown that, for a class of signal distributions, which includes the Gaussian, the quantizers with maximum output entropy (MOE) and minimum average error (MAE) are approximately the same within a multiplicative constant.     

基于小波包分析的激光陀螺信号滤波方法

各种随机噪声是激光陀螺误差的主要来源。为了减少激光陀螺的随机误差,提高其测量精度,介绍了基于小波包分析的滤波方法,研究了小波包分析和滤波的原理、熵标准和阈值函数的选取,比较了选择不同熵标准、阈值函数对激光陀螺信号滤波的效果,并采用Allan方差法分析滤波效果。结果表明基于小波包分析的滤波方法能有效减小随机误差,提高激光陀螺的测量精度。     

Survey and comparative analysis of entropy and relative entropy thresholding techniques

Entropy-based image thresholding has received considerable interest in recent years. Two types of entropy are generally used as thresholding criteria: Shannon's entropy and relative entropy, also known as Kullback-Leibler information distance, where the former measures uncertainty in an information source with an optimal threshold obtained by maximising Shannon's entropy, whereas the latter measures the information discrepancy between two different sources with an optimal threshold obtained by minimising relative entropy. Many thresholding methods have been developed for both criteria and reported in the literature. These two entropy-based thresholding criteria have been investigated and the relationship among entropy and relative entropy thresholding methods has been explored. In particular, a survey and comparative analysis is conducted among several widely used methods that include Pun and Kapur's maximum entropy, Kittler and Illingworth's minimum error thresholding, Pal and Pal's entropy thresholding and Chang et al.'s relative entropy thresholding methods. In order to objectively assess these methods, two measures, uniformity and shape, are used for performance evaluation     

On quantizers for DPCM coding of picture signals

A measure of picture quality for simple element, differentially coded pictures is developed based on certain subjective tests. The measure weights the quantization noise according to its visibility. It is shown that the measure correlates well with the picture quality determined on a standard impairment scale. Optimization of DPCM quantizers is done for this and for the mean-square measure of picture quality. Performance of the following types of quantizers is evaluated in terms of entropy of the quantized output and the picture quality: a) minimum mean-square error quantizers with a fixed number of levels, b) minimum mean-square error quantizers with fixed entropy, c) minimum mean-square subjective distortion quantizers with a fixed number of levels, d) minimum mean-square subjective distortion quantizers with fixed entropy, and e) uniform quantizers. It is concluded that for a fixed number of levels and a fixed word-length coding of the quantizer outputs, the quantizers in c) outperform those in a); and with variable length coding, the quantizers in d) perform better than all of the other quantizers having the same entropy. The sensitivity of the approach to variation of picture content is also investigated.     

A maximum entropy method for lightwave system performance evaluation

A constrained maximum entropy criterion is used to approximate the cumulative distribution function of the photocurrent generated by an avalanche photodiode in response to an incident information-bearing optical signal. The approximate distribution, derived from known moments of the photocurrent, is used to evaluate the probability of error in direct detection lightwave systems. In this application, the results of the maximum entropy method are equivalent to those of a Gauss quadrature rule method. However the maximum entropy method exhibits a relative efficiency in terms of the required number of moments of the photocurrent.     

Lattice quantizers: Entropy reduction by proper tie-handling (Corresp.)

The entropy of a lattice quantizer can be lowered by breaking ties in rounding off in favor of the closest lattice point of highest probability. For a differential pulse-code modulation (DPCM) signal, this means rounding off toward the component of smallest norm. This variant of the algorithm of Conway and Sloane reduces entropy without sacrificing mean-square error.     

Mixing and Non-Mixing Local Minima of the Entropy Contrast for Blind Source Separation

In this paper, both non-mixing and mixing local minima of the entropy are analyzed from the viewpoint of blind source separation (BSS); they correspond respectively to acceptable and spurious solutions of the BSS problem. The contribution of this work is twofold. First, a Taylor development is used to show that the exact output entropy cost function has a non-mixing minimum when this output is proportional to any of the non-Gaussian sources, and not only when the output is proportional to the lowest entropic source. Second, in order to prove that mixing entropy minima exist when the source densities are strongly multimodal, an entropy approximator is proposed. The latter has the major advantage that an error bound can be provided. Even if this approximator (and the associated bound) is used here in the BSS context, it can be applied for estimating the entropy of any random variable with multimodal density     

Research on network analysis method for development ability of big data industry in underdeveloped area

Traditional evaluation methods of industrial development ability were mostly lack of objectivity.An evaluation model was proposed by using a BP neural network based on entropy weight.Evaluation index system of big data industry development ability in underdeveloped areas was established.Taking Guizhou industrial development data as samples,entropy weight method was used to determine expected output and compared with the actual output .The experimental results show that the proposed entropy weight-BP evaluation model can optimize error of using single BP network and improve the accuracy and objectivity of evaluation.