Adaptive blind signal processing-neural network approaches

Learning algorithms and underlying basic mathematical ideas are presented for the problem of adaptive blind signal processing, especially instantaneous blind separation and multichannel blind deconvolution/equalization of independent source signals. We discuss developments of adaptive learning algorithms based on the natural gradient approach and their properties concerning convergence, stability, and efficiency. Several promising schemas are proposed and reviewed in the paper. Emphasis is given to neural networks or adaptive filtering models and associated online adaptive nonlinear learning algorithms. Computer simulations illustrate the performances of the developed algorithms. Some results presented in this paper are new and are being published for the first time  

Flexible Independent Component Analysis

This paper addresses an independent component analysis (ICA) learning algorithm with flexible nonlinearity, so named as flexible ICA, that is able to separate instantaneous mixtures of sub- and super-Gaussian source signals. In the framework of natural Riemannian gradient, we employ the parameterized generalized Gaussian density model for hypothesized source distributions. The nonlinear function in the flexible ICA algorithm is controlled by the Gaussian exponent according to the estimated kurtosis of demixing filter output. Computer simulation results and performance comparison with existing methods are presented.  

Second Order Nonstationary Source Separation

This paper addresses a method of blind source separation that jointly exploits the nonstationarity and temporal structure of sources. The method needs only multiple time-delayed correlation matrices of the observation data, each of which is evaluated at different time-windowed data frame, to estimate the demixing matrix. The method is insensitive to the temporally white noise since it is based on only time-delayed correlation matrices (with non-zero time-lags) and is applicable to the case of either nonstationary sources or temporally correlated sources. We also discuss the extension of some existing methods with the overview of second-order blind source separation methods. Extensive numerical experiments confirm the validity and high performance of the proposed method.  

Blind source separation-semiparametric statistical approach

The semiparametric statistical model is used to formulate the problem of blind source separation. The method of estimating functions is applied to this problem. It is shown that an estimator of the mixing matrix or its learning version can be described in terms of an estimating function. The statistical efficiencies of these algorithms are studied. The main results are as follows. (1) The space consisting of all the estimating functions is derived. (2) The space is decomposed into the orthogonal sum of the admissible part and a redundant ancillary part. For any estimating function, one can find a better or equally good estimator in the admissible part. (3) The Fisher efficient (that is, asymptotically best) estimating functions are derived. (4) The stability of learning algorithms is studied  

Noise reduction in brain evoked potentials based on third-order correlations

In this paper, we use third-order correlations (TOC) in developing a filtering technique for the recovery of brain evoked potentials (EPs). The main idea behind the presented technique is to pass the noisy signal through a finite impulse response filter whose impulse response is matched with the shape of the noise-free signal. It is shown that it is possible to estimate the filter impulse response on basis of a selected third-order correlation slice (TOCS) of the input noisy signal. This is justified by two facts. The first one is that the noise-free EPs can be modeled as a sum of damped sinusoidal signals and the selected TOCS preserve the signal structure. The second fact is that the TOCS is insensitive to both Gaussian noise and other symmetrically distributed non-Gaussian noise, (white or colored). Furthermore, the approach can be applied to either nonaveraged or averaged EP observation data. In the nonaveraged data case, the approach therefore preserves information about amplitude and latency changes. Both fixed and adaptive versions of the proposed filtering technique are described. Extensive simulation results are provided to show the validity and effectiveness of the proposed cumulant-based filtering technique in comparison with the conventional correlation-based counterpart.  

Nonnegative Matrix and Tensor Factorization

In these lecture notes, the authors have outlined several approaches to solve a NMF/NTF problem. The following main conclusions can be drawn: 1) Multiplicative algorithms are not necessary the best approaches for NMF, especially if data representations are not very redundant or sparse. 2) Much better performance can be achieved using the FP-ALS (especially for large-scale problems), IPC, and QN methods. 3) To achieve high performance it is quite important to use the multilayer structure with multistart initialization conditions. 4) To estimate physically meaningful nonnegative components it is often necessary to use some a priori knowledge and impose additional constraints or regularization terms (to control sparsity, boundness, continuity or smoothness of the estimated nonnegative components).  

Extraction of event-related signals from multichannel bioelectrical measurements

Independent component analysis (ICA) is a powerful tool for separating signals from their mixtures. In this field, many algorithms were proposed, but they poorly use a priori information in order to find the desired signal. Here, we propose a fixed point algorithm which uses a priori information to find the signal of interest out of a number of sensors. We particularly applied the algorithm to cancel cardiac artifacts from a magnetoencephalogram.  

Information geometry on hierarchy of probability distributions

An exponential family or mixture family of probability distributions has a natural hierarchical structure. This paper gives an “orthogonal” decomposition of such a system based on information geometry. A typical example is the decomposition of stochastic dependency among a number of random variables. In general, they have a complex structure of dependencies. Pairwise dependency is easily represented by correlation, but it is more difficult to measure effects of pure triplewise or higher order interactions (dependencies) among these variables. Stochastic dependency is decomposed quantitatively into an “orthogonal” sum of pairwise, triplewise, and further higher order dependencies. This gives a new invariant decomposition of joint entropy. This problem is important for extracting intrinsic interactions in firing patterns of an ensemble of neurons and for estimating its functional connections. The orthogonal decomposition is given in a wide class of hierarchical structures including both exponential and mixture families. As an example, we decompose the dependency in a higher order Markov chain into a sum of those in various lower order Markov chains  

Superefficiency in blind source separation

Blind source separation is the problem of extracting independent signals from their mixtures without knowing the mixing coefficients nor the probability distributions of source signals and may be applied to EEG and MEG imaging of the brain. It is already known that certain algorithms work well for the extraction of independent components. The present paper is concerned with superefficiency of these based on the statistical and dynamical analysis. In a statistical estimation using t examples, the covariance of any two extracted independent signals converges to 0 of the order of 1/t. On-line dynamics shows that the covariance is of the order of η when the learning rate η is fixed to a small constant. In contrast with the above general properties, a surprising superefficiency holds in blind source separation under certain conditions where superefficiency implies that covariance decreases in the order of 1/t² or of η² . The paper uses the natural gradient learning algorithm and method of estimating functions to obtain superefficient procedures for both batch estimation and on-line learning. A standardized estimating function is introduced to this end. Superefficiency does not imply that the error variances of the extracted signals decrease in the order of 1/t² or η² but implies that their covariances (and independencies) do  

Electron microscopic analysis of a fusion protein of postsynaptic density-95 and metallothionein in cultured hippocampal neurons

The subcellular localization of biomolecules at high resolution has traditionally been investigated by combining transmission electron microscopy (TEM) and chemical staining with heavy metals or immuno-based labeling with gold-conjugated antibodies. Here, we employ genetically encoded tags to examine the localization of proteins in transfected cultured cells by TEM. We purified a fusion protein of postsynaptic density-95 (PSD-95) coupled to three tandem repeats of metallothionein (MT) (PDS-95-3MT) from COS7 cells grown in the presence of Cd2+. PSD-95-3MT was detected as black particles by TEM. To visualize the subcellular localization of PSD-95-3MT, expression constructs encoding this fusion protein were transfected into primary hippocampal neurons cultured in medium containing Cd2+. The subcellular accumulation of PSD-95-3MT and Cd2+ provided excellent contrast in TEM micrographs. To address if genetically encoded tags affect the function of the target proteins, we found that the conjugation of 3MT to PSD-95 did not alter its association with known binding partners. These results demonstrate that 3MT coordinating Cd2+ is a valuable genetically encoded tag to study the localization of proteins by TEM.