Morphologically constrained ICA for extracting weak temporally correlated signals

Recently the constrained ICA (cICA) algorithm has been widely applied to many applications. But a crucial problem to the algorithm is how to design a reference signal in advance, which should be closely related to the desired source signal. If the desired source signal is very weak in mixed signals and there is no enough a priori information about it, the reference signal is difficult to design. With some detailed discussions on the cICA algorithm, the paper proposes a second-order statistics based approach to reliably find suitable reference signals for weak temporally correlated source signals. Simulations on synthetic data and real-world data have shown its validity and usefulness.     

Approach and applications of constrained ICA

This work presents the technique of constrained independent component analysis (cICA) and demonstrates two applications, less-complete ICA, and ICA with reference (ICA-R). The cICA is proposed as a general framework to incorporate additional requirements and prior information in the form of constraints into the ICA contrast function. The adaptive solutions using the Newton-like learning are proposed to solve the constrained optimization problem. The applications illustrate the versatility of the cICA by separating subspaces of independent components according to density types and extracting a set of desired sources when rough templates are available. The experiments using face images and functional MR images demonstrate the usage and efficacy of the cICA.     

A reference-based blind source extraction algorithm

By utilizing a priori information available as reference, constrained independent component analysis (cICA) or independent component analysis with reference (ICA-R) achieves some advantages over other methods. However, ICA-R is very time-consuming; moreover, it is very difficult to determine its threshold parameter, once the value is improperly chosen the algorithm will fail to converge. In order to overcome these drawbacks, a very simple blind source extraction method, whose optimization function is simply the closeness measure between the desired output and its corresponding reference in ICA-R, is proposed in this paper. Experiments with synthesized data and real-world electrocardiograph data confirm its validity and superiority.     

Extracting post-nonlinear signal with reference

Constrained independent component analysis (cICA) is an important technique which can extract the desired sources from the mixtures. The post-nonlinear (PNL) mixture model is more realistic and accurate than the linear instantaneous mixture model in many practical situations. In this paper, we address the problem of extracting the desired source as the first output from the PNL mixture. The prior knowledge about the desired source, such as its rough template (reference), is assumed to be available. Two approaches of extracting PNL signal with reference are discussed. Then a novel algorithm which alternately optimizes the contrast function and the closeness measure between the estimated output and the reference signal is proposed. The inverse of the unknown nonlinear function in the PNL mixture model is approximated by the multi-layer perception (MLP) network. The correctness and validity of the proposed algorithm are demonstrated by our experiment results.     

Content-based facial image retrieval using constrained independent component analysis

Content-based image retrieval (CBIR) is a method of searching, browsing, and querying images according to their content. In this paper, we focus on a specific domain of CBIR that involves the development of a content-based facial image retrieval system based on the constrained independent component analysis (cICA). Originating from independent component analysis (ICA), cICA is a source separation technique that uses priori constraints to extract desired independent components (ICs) from data. By providing query images as the constraints to the cICA, the ICs that share similar probabilistic features with the queries from the database can be extracted. Then, these extracted ICs are used to evaluate the rank of each image according to the query. In our approach, we demonstrate that, in addition to a single image-based query, a compound query with multiple query images can be used to search for images with compounding feature content. The experimental results of our CBIR system tested with different facial databases show that our system can improve retrieval performance by using a compound query. Furthermore, our system allows for online processing without the need to learn query images.     

An improved method for independent component analysis with reference

Through incorporating a priori information available in some applications for independent component analysis (ICA) as the reference into the negentropy contrast function for FastICA, ICA with reference (ICA-R) or constrained ICA (cICA) is obtained as a constrained optimization problem. ICA-R achieves some advantages over earlier methods, whereas its computation load is somewhat high and its performance is strongly dependent on the threshold parameter. By alternately optimizing the negentropy contrast function for FastICA and the closeness measure for ICA-R, an improved method for ICA-R is proposed in this paper which can avoid the inherent drawbacks of ICA-R. The validity of the proposed method is demonstrated by simulation experiments.     

鲁棒的交叉熵模糊聚类算法

针对模糊C-均值聚类(FCM)算法对噪声敏感、容易收敛到局部极小值的问题,提出一种基于交叉熵的模糊聚类算法。通过引入交叉熵重新定义了传统FCM算法的目标函数,利用交叉熵度量样本隶属度之间的差异性,并采用拉格朗日求解方法和朗伯W函数解决了目标函数的优化问题,此外,分析了样本划分矩阵的分布情况,依据分布特性对噪声样本进行识别。人工数据集合和标准数据集加噪的实验结果表明,该算法提高了传统FCM算法的抗干扰能力,具有更强的鲁棒性,噪声样本识别的准确率较高。     

一种基于增量加权平均的在线序贯极限学习机算法

针对在线序贯极限学习机(OS-ELM)对增量数据学习效率低、准确性差的问题, 提出一种基于增量加权平均的在线序贯极限学习机(WOS-ELM)算法.将算法的原始数据训练模型残差与增量数据训练模型残差进行加权作为代价函数,推导出用于均衡原始数据与增量数据的训练模型,利用原始数据来弱化增量数据的波动,使在线极限学习机具有较好的稳定性,从而提高算法的学习效率和准确性. 仿真实验结果表明, 所提出的WOS-ELM算法对增量数据具有较好的预测精度和泛化能力.     

基于整数多项式环的全同态加密算法

为确保云计算环境下用户数据的安全性,利用同态加密算法对数据和加密函数的隐私保护功能,设计一种基于整数多项式环的全同态加密算法。该算法包括同态算法和重加密算法,前者针对明文数据进行加密,后者针对密文数据进行二次加密。分析结果表明,该算法的计算复杂度为O(n5),低于理想格全同态加密算法。     

变分贝叶斯概率数据关联算法

针对杂波环境下的目标跟踪问题,提出了一种基于变分贝叶斯的概率数据关联算法(Variational Bayesian based probabilistic data association algorithm, VB-PDA).该算法首先将关联事件视为一个随机变量并利用多项分布对其进行建模,随后基于数据集、目标状态、关联事件的联合概率密度函数求取关联事件的后验概率密度函数,最后将关联事件的后验概率密度函数引入变分贝叶斯框架中以获取状态近似后验概率密度函数.相比于概率数据关联算法, VB-PDA算法在提高算法实时性的同时在权重Kullback-Leibler (KL)平均准则下获取了近似程度更高的状态后验概率密度函数.相关仿真实验对提出算法的有效性进行了验证.     

基于密度函数加权的模糊C均值聚类算法研究

模糊聚类算法具有较强的实用性,但传统模糊C均值算法（FCM）具有对样本集进行等划分趋势的缺陷,没有考虑不同样本的实际分布对聚类效果的影响,当数据集中各样本密集程度相差较大时,聚类结果不是很理想。因此,提出一种基于密度函数加权的模糊C均值聚类算法（DFCM算法）,该算法利用数据对象的密度函数作为每个数据点权值。实验结果表明,与传统的模糊C均值算法相比,DFCM算法具有较好的聚类效果。     

基于粒子群优化的深度神经网络分类算法

针对神经网络分类算法中节点函数不可导,分类精度不够高等问题,提出了一种基于粒子群优化(PSO)算法的深度神经网络分类算法.使用深度学习中的自动编码机,结合PSO算法优化权值,利用自动编码机对输入样本数据进行编解码,为提高网络分类精度,以编码机本身的误差函数和Softmax分类器的代价函数加权求和共同作为PSO算法的评价函数,使编码后的数据更加适应分类器.实验结果证明:与其他传统的神经网络相比,在邮件分类问题上,此分类算法有更高的分类精度.     

基于核函数的谱嵌入聚类算法

谱嵌入聚类(SEC)算法要求样本满足流形假设,样本标签总是可以嵌入到一个线性空间中去,这为线性可分数据的谱嵌入聚类问题提供了新的思路,但该算法使用的线性映射函数不适用于处理高维非线性数据。针对这一问题,通过核化线性映射函数,建立了基于核函数的谱嵌入聚类(KSEC)模型,该模型既能解决线性映射函数不能处理非线性数据的问题,又实现了对高维数据的核降维。在真实数据集上的实验分析结果表明,使用所提算法后聚类正确率平均提高了13.11%,最高可提高31.62%,特别在高维数据上平均提高了16.53%,而且在算法关于参数的敏感度实验中发现算法的稳定性更好。所以改进后的算法对高维非线性数据具有很好的聚类效果,获得了比传统谱嵌入聚类算法更高的聚类准确率和更好的聚类性能。所提方法可以用于诸如遥感影像这类复杂图像的处理领域。     

An iterative algorithm learning the maximal margin classifier

A simple learning algorithm for maximal margin classifiers (also support vector machines with quadratic cost function) is proposed. We build our iterative algorithm on top of the Schlesinger-Kozinec algorithm (S-K-algorithm) from 1981 which finds a maximal margin hyperplane with a given precision for separable data. We suggest a generalization of the S-K-algorithm (i) to the non-linear case using kernel functions and (ii) for non-separable data. The requirement in memory storage is linear to the data. This property allows the proposed algorithm to be used for large training problems.The resulting algorithm is simple to implement and as the experiments showed competitive to the state-of-the-art algorithms. The implementation of the algorithm in Matlab is available. We tested the algorithm on the problem aiming at recognition poor quality numerals.     

Kernel approach to possibilistic C‐means clustering

Kernel approaches can improve the performance of conventional clustering or classification algorithms for complex distributed data. This is achieved by using a kernel function, which is defined as the inner product of two values obtained by a transformation function. In doing so, this allows algorithms to operate in a higher dimensional space (i.e., more degrees of freedom for data to be meaningfully partitioned) without having to compute the transformation. As a result, the fuzzy kernel C‐means (FKCM) algorithm, which uses a distance measure between patterns and cluster prototypes based on a kernel function, can obtain more desirable clustering results than fuzzy C‐means (FCM) for not only spherical data but also nonspherical data. However, it can still be sensitive to noise as in the FCM algorithm. In this paper, to improve the drawback of FKCM, we propose a kernel possibilistic C‐means (KPCM) algorithm that applies the kernel approach to the possibilistic C‐means (PCM) algorithm. The method includes a variance updating method for Gaussian kernels for each clustering iteration. Several experimental results show that the proposed algorithm can outperform other algorithms for general data with additive noise. © 2009 Wiley Periodicals, Inc.     

灵敏性分析下的因果网络参数的扰动学习研究

联合观察数据和扰动数据学习因果网络是一种基于扰动的机器学习方法,通过扰动学习可以利用少量样本发现网络中的因果关系,扰动对于因果关系的影响主要体现在网络参数方面。提出了一种基于灵敏性分析的因果网络参数的扰动学习算法(intervention learning of parameter sensitivity analysis,ILPSA)。对于给定的先验网络,ILPSA算法利用联合树推理算法生成灵敏性函数,通过对灵敏性函数的参数重要性分析提出扰动结点的一种主动选取方法;对扰动结点的主动干扰产生扰动数据,然后联合观察数据和扰动数据,利用最大似然估计(maximum likelihood estimation,MLE)方法学习因果网络的参数,并利用KL距离对学习结果进行评价。算法比较和实验结果表明,ILPSA算法的学习结果明显好于随机选择扰动结点和无扰动情况下的方法,特别在样本较小的情况下优势更明显。     

嵌套删失数据期望最大化的高斯混合聚类算法

针对聚类问题中的非随机性缺失数据,本文基于高斯混合聚类模型,分析了删失型数据期望最大化算法的有效性,并揭示了删失数据似然函数对模型算法的作用机制.从赤池弘次信息准则、信息散度等指标,比较了所提出方法与标准的期望最大化算法的优劣性.通过删失数据划分及指示变量,推导了聚类模型参数后验概率及似然函数,调整了参数截尾正态函数的...     

基于鱼群优化算法和Cholesky分解的RELM的基因表达数据分类

提出一种基于鱼群优化算法和Cholesky分解的改进的正则极限学习机算法(FSC-RELM)来对基因表达数据进行分类。FSC-RELM算法中,首先用鱼群优化算法对RELM输入层权值进行优化,其中目标函数定义为误差函数的倒数;再对RELM输出层权值矩阵进行分解,采用Cholesky分解法进行优化,以提高算法速度,减少训练时间。为了评价算法性能,对若干标准基因数据集进行了实验,结果表明,FSC-RELM算法在较短的时间内可以获得较高的分类精度,性能优异。     

基于模糊支持向量机的多分类算法研究

针对支持向量机理论中的多分类问题以及SVM对噪声数据的敏感性问题,提出了一种基于二叉树的模糊支持向量机多分类算法。该算法是在基于二叉树的支持向量机多分类算法的基础上引入模糊隶属度函数,根据每个样本数据对分类结果的不同影响,通过基于KNN的模糊隶属度的度量方法计算出相应的值,由此得到不同的惩罚值,这样在构造分类超平面时,就可以忽略对分类结果不重要的数据。通过实验证明,该算法有较好的抗干扰能力和分类效果。     

有监督核化邻域投影分析算法

通过将鉴别邻域嵌入分析算法扩展到非线性场景,提出了一种有监督核化邻域投影分析算法。该算法在目标函数中引入类别标签和线性投影矩阵,并利用核函数处理非线性数据。通过两种不同策略优化目标函数,可将该算法进一步细分为有监督核化邻域投影分析算法一及有监督核化邻域投影分析算法二。其中,在有监督核化邻域投影分析算法一中应用拉普拉斯搜索方向达到了较快的收敛速度并降低了计算复杂度。实验结果表明,所提算法对于复杂的数据流形具有较高的识别率,且与鉴别邻域嵌入分析等相关算法相比在有效性和鲁棒性方面的表现更为出色。