Analysis of sparse representation and blind source separation

In this letter, we analyze a two-stage cluster-then-l(1)-optimization approach for sparse representation of a data matrix, which is also a promising approach for blind source separation (BSS) in which fewer sensors than sources are present. First, sparse representation (factorization) of a data matrix is discussed. For a given overcomplete basis matrix, the corresponding sparse solution (coefficient matrix) with minimum l(1) norm is unique with probability one, which can be obtained using a standard linear programming algorithm. The equivalence of the l(1)-norm solution and the l(0)-norm solution is also analyzed according to a probabilistic framework. If the obtained l(1)-norm solution is sufficiently sparse, then it is equal to the l(0)-norm solution with a high probability. Furthermore, the l(1)- norm solution is robust to noise, but the l(0)-norm solution is not, showing that the l(1)-norm is a good sparsity measure. These results can be used as a recoverability analysis of BSS, as discussed. The basis matrix in this article is estimated using a clustering algorithm followed by normalization, in which the matrix columns are the cluster centers of normalized data column vectors. Zibulevsky, Pearlmutter, Boll, and Kisilev (2000) used this kind of two-stage approach in underdetermined BSS. Our recoverability analysis shows that this approach can deal with the situation in which the sources are overlapped to some degree in the analyzed domain and with the case in which the source number is unknown. It is also robust to additive noise and estimation error in the mixing matrix. Finally, four simulation examples and an EEG data analysis example are presented to illustrate the algorithm's utility and demonstrate its performance.     

Multichannel Nonnegative Matrix Factorization in Convolutive Mixtures for Audio Source Separation

We consider inference in a general data-driven object-based model of multichannel audio data, assumed generated as a possibly underdetermined convolutive mixture of source signals. We work in the short-time Fourier transform (STFT) domain, where convolution is routinely approximated as linear instantaneous mixing in each frequency band. Each source STFT is given a model inspired from nonnegative matrix factorization (NMF) with the Itakura–Saito divergence, which underlies a statistical model of superimposed Gaussian components. We address estimation of the mixing and source parameters using two methods. The first one consists of maximizing the exact joint likelihood of the multichannel data using an expectation-maximization (EM) algorithm. The second method consists of maximizing the sum of individual likelihoods of all channels using a multiplicative update algorithm inspired from NMF methodology. Our decomposition algorithms are applied to stereo audio source separation in various settings, covering blind and supervised separation, music and speech sources, synthetic instantaneous and convolutive mixtures, as well as professionally produced music recordings. Our EM method produces competitive results with respect to state-of-the-art as illustrated on two tasks from the international Signal Separation Evaluation Campaign (SiSEC 2008).      

Mixing matrix estimation using discriminative clustering for blind source separation

Mixing matrix estimation in instantaneous blind source separation (BSS) can be performed by exploiting the sparsity and disjoint orthogonality of source signals. As a result, approaches for estimating the unknown mixing process typically employ clustering algorithms on the mixtures in a parametric domain, where the signals can be sparsely represented. In this paper, we propose two algorithms to perform discriminative clustering of the mixture signals for estimating the mixing matrix. For the case of overdetermined BSS, we develop an algorithm to perform linear discriminant analysis based on similarity measures and combine it with K-hyperline clustering. Furthermore, we propose to perform discriminative clustering in a high-dimensional feature space obtained by an implicit mapping, using the kernel trick, for the case of underdetermined source separation. Using simulations on synthetic data, we demonstrate the improvements in mixing matrix estimation performance obtained using the proposed algorithms in comparison to other clustering methods. Finally we perform mixing matrix estimation from speech mixtures, by clustering single source points in the time-frequency domain, and show that the proposed algorithms achieve higher signal to interference ratio when compared to other baseline algorithms.     

病态盲分离情况下的混合矩阵估计研究

在盲信号分离技术中,当混合矩阵是病态情况时,基于信号稀疏性的两步法可用来解决这一问题,而如何估计混合矩阵则是两步法的关键。提出了一种估计混合矩阵的新方法,即通过搜索重构观测信号采样点,每次只需搜索出少数某源信号取值占优的采样点,就可以通过这些采样点处的重构观测信号数据,估计出混合矩阵的某一列。依次类推,可以估计出整个混合矩阵。该方法估计混合矩阵时对源信号的稀疏度要求较低,其实现算法不需优化过程,计算简单,因此其实用性较高。仿真结果表明了该方法有效,有很好的性能。通过大量的仿真试验给出了方法的定量性能分析。     

Convolutive Blind Source Separation in the Frequency Domain Based on Sparse Representation

Convolutive blind source separation (CBSS) that exploits the sparsity of source signals in the frequency domain is addressed in this paper. We assume the sources follow complex Laplacian-like distribution for complex random variable, in which the real part and imaginary part of complex-valued source signals are not necessarily independent. Based on the maximum a posteriori (MAP) criterion, we propose a novel natural gradient method for complex sparse representation. Moreover, a new CBSS method is further developed based on complex sparse representation. The developed CBSS algorithm works in the frequency domain. Here, we assume that the source signals are sufficiently sparse in the frequency domain. If the sources are sufficiently sparse in the frequency domain and the filter length of mixing channels is relatively small and can be estimated, we can even achieve underdetermined CBSS. We illustrate the validity and performance of the proposed learning algorithm by several simulation examples.     

基于线性阵列的欠定盲源分离几何分析

采用线性阵列对欠定盲源分离问题进行建模,研究源信号的空间分布对欠定盲源分离的影响.利用二步法和稀疏分量分析解决欠定盲源分离问题,其中,混合矩阵的估计主要利用稀疏源信号的线性混合信号沿混合矩阵列向量方向线性聚类的特性.理论分析和仿真实验结果表明,当源信号在空间处于某些特定区域时,若采用线性聚类方法,混合矩阵是不可估计的,...     

Robust non-negative matrix factorization via joint sparse and graph regularization for transfer learning

In real-world applications, we often have to deal with some high-dimensional, sparse, noisy, and non-independent identically distributed data. In this paper, we aim to handle this kind of complex data in a transfer learning framework, and propose a robust non-negative matrix factorization via joint sparse and graph regularization model for transfer learning. First, we employ robust non-negative matrix factorization via sparse regularization model (RSNMF) to handle source domain data and then learn a meaningful matrix, which contains much common information between source domain and target domain data. Second, we treat this learned matrix as a bridge and transfer it to target domain. Target domain data are reconstructed by our robust non-negative matrix factorization via joint sparse and graph regularization model (RSGNMF). Third, we employ feature selection technique on new sparse represented target data. Fourth, we provide novel efficient iterative algorithms for RSNMF model and RSGNMF model and also give rigorous convergence and correctness analysis separately. Finally, experimental results on both text and image data sets demonstrate that our REGTL model outperforms existing start-of-art methods.     

基于ICA的相位编码信号欠定盲分离算法

针对二相编码信号时域或频城上不充分稀疏的情况,提出了欠定盲源分离中估计混合矩阵和恢复源信号的新方法.首先,利用二相编码信号成型模型的特异性,将欠定盲分离问题转化成卷积盲分离问题,然后通过抽头延时将其转化为线性瞬时混叠问题,通过独立分量分析(ICA)方法对延时后的观测信号进一步处理.为了准确地分离出源信号,利用峭度准则对...     

基于约束NMF的欠定盲信号分离算法*

提出一种约束非负矩阵分解方法用于解决欠定盲信号分离问题。非负矩阵分解直接用于求解欠定盲信号分离时,分解结果不唯一,无法正确分离源信号。本文在基本非负矩阵分解算法基础上,对分解得到的混合矩阵施加行列式约束,保证分解结果的唯一性;对分解得到的源信号同时施加稀疏性约束和最小相关约束,实现混合信号的唯一分解,提高源信号分离性能。仿真实验证明了本文算法的有效性。     

具有相似性传播的K-均值欠定盲分离算法

对于稀疏信源的欠定盲分离问题,混合矩阵的估计是至关重要的。为了提高估计性能,提出一种组合的聚类分析算法。首先,利用短时傅里叶变换把时域中的观测信号转变成频域中的稀疏信号,并通过数据的归一化把稀疏信号在频域的线性聚类转变成致密聚类。然后,利用相似性传播AP聚类方法搜索每个观测数据的邻域自动形成数据族的数量和相对应的关键数据。最后,以AP聚类的结果作为K-均值算法的初始值,对每类(族)数据的聚类中心进一步修正。仿真结果表明,组合聚类法能有效地提高混合矩阵的估计精度。把AP聚类和K-均值算法相结合的另一个优势是,能够克服经典K-均值算法需要事先知道信源数量和对数据的初始划分非常敏感的缺陷。     

基于信号稀疏特性和核函数的非线性盲信号分离算法

文章结合核函数,把基于信号稀疏特性的线性盲分离方法应用于非线性混叠情况而给出了一种非线性混叠信号盲分离算法。该算法首先将混叠信号映射到高维核特征空间,其次,在核特征空间中构造一组正交基,通过这组正交基将高维核特征空间的信号映射到这组正交基张成的参数空间中,从而把非线性混叠信号盲分离问题转化为参数空间的线性混叠信号盲分离问题。最后,在参数空间中,应用基于信号稀疏特性的线性盲分离方法对信号进行分离。该算法收敛精度较高,稳定性好。仿真结果表明该算法是有效的,具有良好的分离性能。     

基于功能连接信息熵的精神分裂症EEG分类研究

为了提高精神分裂症的有效诊断,利用网络功能连接信息熵的方法对51例精神分裂症患者和56例年龄匹配的正常人的脑电信号（Electroencephalogram,EEG）进行了分类。通过采用分频技术、相位同步分析方法、信息熵方法、支持向量机（Support Vector Machine,SVM）分类方法,大幅提高了分类准确率（98.13%）,实现了对精神分裂症的有效诊断。该分类方法主要涉及两阶段：利用分频技术和相位同步分析方法,获得各频段的脑电信号在各个时间点的功能连接矩阵;基于整个时间域上的功能连接计算各频段的信息熵,并将其分别作为功能脑网络的分类特征训练SVM分类器,进而对两组被试分类。分类结果表明,该方法大幅提高了精神分裂症检测的准确率。     

密度空间与密度峰值聚类的欠定混合矩阵估计

针对欠定盲源分离问题, 提出了增强信号稀疏性的方法,并把具有噪声的基于密度空间聚类与寻找密度峰值聚类相结合用于估计混合矩阵。首先,把时域观测信号变换成时频域的稀疏信号,通过单源点检测突出信号的线性聚类特性,并采用镜像映射将线性聚类转变成致密聚类以便于进行密度基的聚类分析;然后,利用密度空间聚类搜寻密集数据堆中高密度的点和与之相应的邻域,以自动形成聚类簇的数量和初步聚类中心;最后,把获得的聚类数量作为密度峰值聚类的输入参数,在数据簇的范围内搜索其密度峰值以实现对聚类中心位置的进一步修正。以上方法不仅可提高混合矩阵的估计精度,而且估计量具有较高的一致性。     

蚁群聚类的欠定盲源分离方法

利用欠定盲源分离情况下稀疏源信号具有直线聚类的特点,提出了一种估计混叠矩阵的新方法。通过对混叠信号进行标准化处理,使混叠信号形成球形簇,将线性聚类转变成致密聚类;利用蚁群聚类算法对其进行搜索得到聚类中心,从而获得对混叠矩阵的精确估计。该方法能实现源信号数目未知情况下的欠定盲源分离,且能推广到三路或更多路观测信号的情况。对语音信号的仿真结果证明,该方法能精确地分离和恢复原始信号。     

Parallel solution of sparse linear least squares problems on distributed-memory multiprocessors

This paper studies the parallel solution of large-scale sparse linear least squares problems on distributed-memory multiprocessors. The key components required for solving a sparse linear least squares problem are sparse QR factorization and sparse triangular solution. A block-oriented parallel algorithm for sparse QR factorization has already been described in the literature. In this paper, new block-oriented parallel algorithms for sparse triangular solution are proposed. The arithmetic and communication complexities of the new algorithms applied to regular grid problems are analyzed. The proposed parallel sparse triangular solution algorithms together with the block-oriented parallel sparse QR factorization algorithm result in a highly efficient approach to the parallel solution of sparse linear least squares problems. Performance results obtained on an IBM Scalable POWERparallel system SP2 are presented. The largest least squares problem solved has over two million rows and more than a quarter million columns. The execution speed for the numerical factorization of this problem achieves over 3.7 gigaflops per second on an IBM SP2 machine with 128 processors.     

Blind source separation with time series variational Bayes expectation maximization algorithm

This paper presents a variational Bayes expectation maximization algorithm for time series based on Attias? variational Bayesian theory. The proposed algorithm is applied in the blind source separation (BSS) problem to estimate both the source signals and the mixing matrix for the optimal model structure. The distribution of the mixing matrix is assumed to be a matrix Gaussian distribution due to the correlation of its elements and the inverse covariance of the sensor noise is assumed to be Wishart distributed for the correlation between sensor noises. The mixture of Gaussian model is used to approximate the distribution of each independent source. The rules to update the posterior hyperparameters and the posterior of the model structure are obtained. The optimal model structure is selected as the one with largest posterior. The source signals and mixing matrix are estimated by applying LMS and MAP estimators to the posterior distributions of the hidden variables and the model parameters respectively for the optimal structure. The proposed algorithm is tested with synthetic data. The results show that: (1) the logarithm posterior of the model structure increases with the accuracy of the posterior mixing matrix; (2) the accuracies of the prior mixing matrix, the estimated mixing matrix, and the estimated source signals increase with the logarithm posterior of the model structure. This algorithm is applied to Magnetoencephalograph data to localize the source of the equivalent current dipoles.     

Using underapproximations for sparse nonnegative matrix factorization

Nonnegative matrix factorization consists in (approximately) factorizing a nonnegative data matrix by the product of two low-rank nonnegative matrices. It has been successfully applied as a data analysis technique in numerous domains, e.g., text mining, image processing, microarray data analysis, collaborative filtering, etc.We introduce a novel approach to solve NMF problems, based on the use of an underapproximation technique, and show its effectiveness to obtain sparse solutions. This approach, based on Lagrangian relaxation, allows the resolution of NMF problems in a recursive fashion. We also prove that the underapproximation problem is NP-hard for any fixed factorization rank, using a reduction of the maximum edge biclique problem in bipartite graphs.We test two variants of our underapproximation approach on several standard image datasets and show that they provide sparse part-based representations with low reconstruction error. Our results are comparable and sometimes superior to those obtained by two standard sparse nonnegative matrix factorization techniques.     

Distributed generic approximate sparse inverses

The need for accuracy in the solution of linear systems derived from the discretization of partial differential equations leads to large sparse linear systems. The solution of sparse linear systems requires efficient scalable methods. Iterative solvers require efficient parallel preconditioning methods to solve effectively sparse linear systems. Herewith, a new parallel algorithm for the generic approximate sparse inverse matrix method for distributed memory systems is proposed. The computation of the distributed generic approximate sparse inverse matrix is based on a column-wise approach, which allows the separation to independent problems that can be handled in parallel without synchronization points or intermediate communications. This is achieved by reforming the generic approximate sparse inverse matrix algorithm and its process of computation with a new partial solution method for the computation of the nonzero elements of each column dictated by the approximate inverse sparsity pattern. Moreover, an algorithmic scheme is proposed for the efficient distribution of data amongst the available workstations, along with a load balancing scheme for problems with large standard deviation in the number of nonzero elements per column. Numerical results are presented for the proposed schemes for various model problems.     

Unified parametrization for the solutions to the polynomial diophantine matrix equation and the generalized Sylvester matrix equation

The polynomial Diophantine matrix equation and the generalized Sylvester matrix equation are important for controller design in frequency domain linear system theory and time domain linear system theory, respectively. By using the so-called generalized Sylvester mapping, right coprime factorization and Bezout identity associated with certain polynomial matrices, we present in this note a unified parametrization for the solutions to both of these two classes of matrix equations. Moreover, it is shown that solutions to the generalized Sylvester matrix equation can be obtained if solutions to the Diophantine matrix equation are available. The results disclose a relationship between the polynomial Diophantine matrix equation and generalized Sylvester matrix equation that are respectively studied and used in frequency domain linear system theory and time domain linear system theory.     

Highly scalable parallel algorithms for sparse matrix factorization

In this paper, we describe scalable parallel algorithms for symmetric sparse matrix factorization, analyze their performance and scalability, and present experimental results for up to 1,024 processors on a Gray T3D parallel computer. Through our analysis and experimental results, we demonstrate that our algorithms substantially improve the state of the art in parallel direct solution of sparse linear systems-both in terms of scalability and overall performance. It is a well known fact that dense matrix factorization scales well and can be implemented efficiently on parallel computers. In this paper, we present the first algorithms to factor a wide class of sparse matrices (including those arising from two- and three-dimensional finite element problems) that are asymptotically as scalable as dense matrix factorization algorithms on a variety of parallel architectures. Our algorithms incur less communication overhead and are more scalable than any previously known parallel formulation of sparse matrix factorization. Although, in this paper, we discuss Cholesky factorization of symmetric positive definite matrices, the algorithms can be adapted for solving sparse linear least squares problems and for Gaussian elimination of diagonally dominant matrices that are almost symmetric in structure. An implementation of one of our sparse Cholesky factorization algorithms delivers up to 20 GFlops on a Gray T3D for medium-size structural engineering and linear programming problems. To the best of our knowledge, this is the highest performance ever obtained for sparse Cholesky factorization on any supercomputer