Adaptive blind deconvolution of linear channels using Renyi's entropy with Parzen window estimation

Blind deconvolution of linear channels is a fundamental signal processing problem that has immediate extensions to multiple-channel applications. In this paper, we investigate the suitability of a class of Parzen-window-based entropy estimates, namely Renyi's entropy, as a criterion for blind deconvolution of linear channels. Comparisons between maximum and minimum entropy approaches, as well as the effect of entropy order, equalizer length, sample size, and measurement noise on performance, will be investigated through Monte Carlo simulations. The results indicate that this nonparametric entropy estimation approach outperforms the standard Bell-Sejnowski and normalized kurtosis algorithms in blind deconvolution. In addition, the solutions using Shannon's entropy were not optimal either for super- or sub-Gaussian source densities.     

Using the Penalized Mutual Information Criterion in the Multivariate Edgeworth-Expanded Gaussian Mixture Density for Blind Separation of Convolutive Post-Nonlinear Mixtures

This paper proposes the blind separation of convolutive post-nonlinear (CPNL) mixtures based on the minimization of the penalized mutual information criterion. The proposed algorithm is based on the estimation score function difference (SFD) and the Newton optimization. Compared with the blind source separation of a linear mixture, the separation performance of a nonlinear mixture is strongly related to the accuracy of the score function estimation. Under this framework, the multivariate Edgeworth-expanded Gaussian mixture density is adopted to estimate the SFD, which preserves the higher-order statistical structure of the data as compared to the nonparametric density estimation. Also, the Newton optimization converges faster than the steepest descent gradient. In order to calculate the Hessian matrix, the Taylor expansion of the penalized mutual information criterion is extended to second order. The minimization of the penalized mutual information criterion ensures a priori normalization of the estimated sources, thus avoiding scale indeterminacy. The proposed algorithm has a better performance, and at the same time it speeds up the convergence. Simulations with computer-generated data and synthetic real-world data show the effectiveness of the proposed algorithm.     

基于LTE信号的外辐射源雷达同频基站干扰抑制方法研究

针对基于LTE信号的外辐射源雷达接收信号包含多个同频发射基站的直达波和多径杂波干扰的问题,该文对传统的外辐射源雷达信号处理流程进行了改进,增加了对同频基站干扰的处理步骤,提出了一种基于卷积混合模型的盲源分离算法来抑制同频基站的杂波干扰。假设混合矩阵是一个矢量线性时不变滤波器矩阵,以互信息为代价函数,通过求取互信息的梯度,用最速下降法进行迭代,分离准则是使分离后的信号之间互信息最小化。仿真表明,该文算法能够有效地抑制LTE信号同频发射基站的杂波干扰,为后续的主基站杂波对消处理提供了基础。     

Fractional delay estimation for blind source separation and localization of temporomandibular joint sounds

Temporomandibular joint (TMJ) sound sources are generated from the two joints connecting the lower jaw to the temporal bone. Such sounds are important diagnostic signs in patients suffering from temporomandibular disorder (TMD). In this study, we address the problem of source separation of the TMJ sounds. In particular, we examine patients with only one TMJ generating "clicks". Thereafter, we consider the TMJ sounds recorded from the two auditory canals as mixtures of clicks from the TMD joint and the noise produced by the other healthy/normal TMJ. We next exploit the statistical nonstationary nature of the TMJ signals by employing the degenerate unmixing estimation technique (DUET) algorithm, a time-frequency (T-F) approach to separate the sources. As the DUET algorithm requires the sensors to be closely spaced, which is not satisfied by our recording setup, we have to estimate the delay between the recorded TMJ sounds to perform an alignment of the mixtures. Thus, the proposed extension of DUET enables an essentially arbitrary separation of the sensors. It is also shown that DUET outperforms the convolutive Infomax algorithm in this particular TMJ source separation scenario. The spectra of both separated TMJ sources with our method are comparable to those available in existing literature. Examination of both spectra suggests that the click source has a better audible prominence than the healthy TMJ source. Furthermore, we address the problem of source localization. This can be achieved automatically by detecting the sign of our proposed mutual information estimator which exhibits a maximum at the delay between the two mixtures. As a result, the localized separated TMJ sources can be of great clinical value to dental specialists.     

Learning from Examples with Information Theoretic Criteria

This paper discusses a framework for learning based on information theoretic criteria. A novel algorithm based on Renyi's quadratic entropy is used to train, directly from a data set, linear or nonlinear mappers for entropy maximization or minimization. We provide an intriguing analogy between the computation and an information potential measuring the interactions among the data samples. We also propose two approximations to the Kulback-Leibler divergence based on quadratic distances (Cauchy-Schwartz inequality and Euclidean distance). These distances can still be computed using the information potential. We test the newly proposed distances in blind source separation (unsupervised learning) and in feature extraction for classification (supervised learning). In blind source separation our algorithm is capable of separating instantaneously mixed sources, and for classification the performance of our classifier is comparable to the support vector machines (SVMs).     

An error-entropy minimization algorithm for supervised training ofnonlinear adaptive systems

The paper investigates error-entropy-minimization in adaptive systems training. We prove the equivalence between minimization of error's Renyi (1970) entropy of order α and minimization of a Csiszar (1981) distance measure between the densities of desired and system outputs. A nonparametric estimator for Renyi's entropy is presented, and it is shown that the global minimum of this estimator is the same as the actual entropy. The performance of the error-entropy-minimization criterion is compared with mean-square-error-minimization in the short-term prediction of a chaotic time series and in nonlinear system identification     

Experimental upper bound for the performance of convolutive source separation methods

An important problem in the field of blind source separation (BSS) of real convolutive mixtures is the determination of the role of the demixing filter structure and the criterion/optimization method in limiting separation performance. This issue requires the knowledge of the optimal performance for a given structure, which is unknown for real mixtures. Herein, the authors introduce an experimental upper bound on the separation performance for a class of convolutive blind source separation structures, which can be used to approximate the optimal performance. As opposed to a theoretical upper bound, the experimental upper bound produces an estimate of the optimal separating parameters for each dataset in addition to specifying an upper bound on separation performance. Estimation of the upper bound involves the application of a supervised learning method to the set of observations found by recording the sources one at a time. Using the upper bound, it is demonstrated that structures other than the finite-impulse-response (FIR) structure should be considered for real (convolutive) mixtures, there is still much room for improvement in current convolutive BSS algorithms, and the separation performance of these algorithms is not necessarily limited by local minima.     

一种新型的卷积混和盲信号分离算法

本文提出了一种新的卷积混合盲信号分离算法。首先将卷积混合模型进行等价简化,再用理论推导论证了卷积混合盲分离问题等价于最优化Wiener滤波器问题,给出了一种去相关分离准则函数。最后,对两个源信号卷积混合的情况,用标准自适应LMS算法使分离准则函数达到最小值,从而得到了两个信号混合的分离算法;然后,推广到多个源信号混合得到了相应分离算法。通过计算机仿真试验验证了本算法的有效性和可行性。     

Efficiency-fairness tradeoff in telecommunications networks

Introducing the concept of /spl alpha/-fairness, which allows for a bounded fairness compromise, so that no source is allocated less than a fraction /spl alpha/ of its fair share, this letter studies tradeoffs between efficiency (utilization, throughput or revenue) and fairness in a general telecommunications network with relation to any fairness criterion. We formulate a linear program that finds the optimal bandwidth allocation by maximizing efficiency subject to /spl alpha/-fairness constraints. This leads to what we call an efficiency-fairness curve, which shows the benefit in efficiency as a function of the extent to which fairness is compromised.     

On the Estimation of Differential Entropy From Data Located on Embedded Manifolds

Estimation of the differential entropy from observations of a random variable is of great importance for a wide range of signal processing applications such as source coding, pattern recognition, hypothesis testing, and blind source separation. In this paper, we present a method for estimation of the Shannon differential entropy that accounts for embedded manifolds. The method is based on high-rate quantization theory and forms an extension of the classical nearest-neighbor entropy estimator. The estimator is consistent in the mean square sense and an upper bound on the rate of convergence of the estimator is given. Because of the close connection between compression and Shannon entropy, the proposed method has an advantage over methods estimating the Renyi entropy. Through experiments on uniformly distributed data on known manifolds and real-world speech data we show the accuracy and usefulness of our proposed method.     

卷积混叠信号盲分离

本文讨论了卷积混叠信号盲分离问题，证明了一系列基于高阶谱的判据，指出了盲辨识和盲分离的充分条件，利用判据，发展了若干算法，模拟实验证实了判据的正确性和算法的有效性。     

Mutual information approach to blind separation of stationarysources

This paper presents a unified approach to the problem of blind separation of sources, based on the concept of mutual information. This concept is applied to the whole source sequences as stationary processes and thus provides a universal contrast applicable to both the instantaneous and convolutive mixture cases. For practical implementation, we introduce several degraded forms of this contrast, computable from a finite-dimensional distribution of the reconstructed source processes only. From them, we derive several sets of estimating equations, generalizing those considered earlier     

基于互信息的卷积混合语音时域盲分离算法

本文以互信息最小化作为分离准则,提出了一种适用于非平稳语音卷积混合信号的时域盲分离算法。其目标函数同时考虑了语音信号的短时平稳性和长时非平稳性,在短时间段中计算平均互信息,在长时间段中引入权值因子,对短时间段中计算得到的平均互信息进行加权;其分离矩阵的更新采用快速收敛的自然梯度算法;其"去白化"的后处理步骤提高了分离语音的自然度。仿真实验和分析表明了算法的有效性。     

Entropy minimization for supervised digital communications channelequalization

This paper investigates the application of error-entropy minimization algorithms to digital communications channel equalization. The pdf of the error between the training sequence and the output of the equalizer is estimated using the Parzen windowing method with a Gaussian kernel, and then, the Renyi's quadratic entropy is minimized using a gradient descent algorithm. By estimating Renyi's entropy over a short sliding window, an online training algorithm is also introduced. Moreover, for a linear equalizer, an orthogonality condition for the minimum entropy solution that leads to an alternative fixed-point iterative minimization method is derived. The performance of linear and nonlinear equalizers trained with entropy and mean square error (MSE) is compared. As expected, the results of training a linear equalizer are very similar for both criteria since, even if the input noise is non-Gaussian, the output filtered noise tends to be Gaussian. On the other hand, for nonlinear channels and using a multilayer perceptron (MLP) as the equalizer, differences between both criteria appear. Specifically, it is shown that the additional information used by the entropy criterion yields a faster convergence in comparison with the MSE     

Blind source separation for convolutive mixtures based on the joint diagonalization of power spectral density matrices

This paper studies the problem of blind separation of convolutively mixed source signals on the basis of the joint diagonalization (JD) of power spectral density matrices (PSDMs) observed at the output of the separation system. Firstly, a general framework of JD-based blind source separation (BSS) is reviewed and summarized. Special emphasis is put on the separability conditions of sources and mixing system. Secondly, the JD-based BSS is generalized to the separation of convolutive mixtures. The definition of a time and frequency dependent characteristic matrix of sources allows us to state the conditions under which the separation of convolutive mixtures is possible. Lastly, a frequency-domain approach is proposed for convolutive mixture separation. The proposed approach exploits objective functions based on a set of PSDMs. These objective functions are defined in the frequency domain, but are jointly optimized with respect to the time-domain coefficients of the unmixing system. The local permutation ambiguity problems, which are inherent to most frequency-domain approaches, are effectively avoided with the proposed algorithm. Simulation results show that the proposed algorithm is valid for the separation of both simulated and real-word recorded convolutive mixtures.     

Stability analyses of information-theoretic blind separationalgorithms in the case where the sources are nonlinear processes

A basic approach to blind source separation is to define an index representing the statistical dependency among the output signals of the separator and minimize it with respect to the separator's parameters. The most natural index might be mutual information among the output signals of the separator. In the case of a convolutive mixture, however, since the signals must be treated as a time series, it becomes very complicated to concretely express the mutual information as a function of the parameters. To cope with this difficulty, in most of the conventional methods, the source signals are assumed to be independent identically distributed (i.i.d.) or linear. Based on this assumption, some simpler indices are defined, and their minimization is made by such an iterative calculation as the gradient method. In actual applications, however, the sources are often not linear processes. This paper discusses what will happen when those algorithms postulating the linearity of the sources are applied to the case of nonlinear sources. An analysis of local stability derives a couple of conditions guaranteeing that the separator stably tends toward a desired one with iteration. The obtained results reveal that those methods, which are based on the minimization of some indices related to the mutual information, do not work well when the sources signals are far from linear     

An MSE-Based Method to Avoid Permutation/Gain Indeterminacy in Frequency-Domain Blind Source Separation

This paper presents a novel technique for separating convolutive mixtures of statistically independent non-Gaussian signals without resorting to an a priori knowledge of the sources or the mixing system. This problem is solved in the frequency domain by transforming the convolutive mixture into several instantaneous mixtures which are independently separated using blind source separation (BSS) algorithms. First, the instantaneous mixture at one frequency is solved using the joint approximate diagonalization of eigenmatrices (JADE) technique, and the other mixtures are then separated using the mean squared error (MSE) criterion. As a special case of this method, we consider the separation of non-Gaussian temporally white signals transmitted in blocks with zero padding between them.     

Information theoretic learning with adaptive kernels

This paper presents an online algorithm for adapting the kernel width that is a free parameter in information theoretic cost functions using Renyi's entropy. This kernel computes the interactions between the error samples and essentially controls the nature of the performance surface over which the parameters of the system adapt. Since the error in an adaptive system is non-stationary during training, a fixed value of the kernel width may affect the adaptation dynamics and even compromise the location of the global optimum in parameter space. The proposed online algorithm for adapting the kernel width is derived from first principles and minimizes the Kullback-Leibler divergence between the estimated error density and the true density. We characterize the performance of this novel approach with simulations of linear and nonlinear systems training, using the minimum error entropy criterion with the proposed adaptive kernel algorithm. We conclude that adapting the kernel width improves the rate of convergence of the parameters, and decouples the convergence rate and misadjustment of the filter weights.     

On the convexity of higher order Jensen differences based on entropy functions (Corresp.)

In an earlier work, the authors introduced a divergence measure, called the first-order Jensen difference, or in shortcal j-divergence, which is based on entropy functions of degreealpha. This provided a generalization of the measure of mutual information based on Shannon's entropy (corresponding toalpha = 1). It was shown that the first-ordercal j-divergence is a convex function only when a is restricted to some range. We define higher order Jensen differences and show that they are convex functions only when the underlying entropy function is of degree two. A statistical application requiring the convexity of higher order Jensen differences is indicated.