Joint feature re-extraction and classification using an iterative semi-supervised support vector machine algorithm

The focus of this paper is on joint feature re-extraction and classification in cases when the training data set is small. An iterative semi-supervised support vector machine (SVM) algorithm is proposed, where each iteration consists both feature re-extraction and classification, and the feature re-extraction is based on the classification results from the previous iteration. Feature extraction is first discussed in the framework of Rayleigh coefficient maximization. The effectiveness of common spatial pattern (CSP) feature, which is commonly used in Electroencephalogram (EEG) data analysis and EEG-based brain computer interfaces (BCIs), can be explained by Rayleigh coefficient maximization. Two other features are also defined using the Rayleigh coefficient. These features are effective for discriminating two classes with different means or different variances. If we extract features based on Rayleigh coefficient maximization, a large training data set with labels is required in general; otherwise, the extracted features are not reliable. Thus we present an iterative semi-supervised SVM algorithm embedded with feature re-extraction. This iterative algorithm can be used to extract these three features reliably and perform classification simultaneously in cases where the training data set is small. Each iteration is composed of two main steps: (i) the training data set is updated/augmented using unlabeled test data with their predicted labels; features are re-extracted based on the augmented training data set. (ii) The re-extracted features are classified by a standard SVM. Regarding parameter setting and model selection of our algorithm, we also propose a semi-supervised learning-based method using the Rayleigh coefficient, in which both training data and test data are used. This method is suitable when cross-validation model selection may not work for small training data set. Finally, the results of data analysis are presented to demonstrate the validity of our approach. Editor: Olivier Chapelle.     

基于CMAC的非参数化近似策略迭代增强学习

为解决在线近似策略迭代增强学习计算复杂度高、收敛速度慢的问题，引入CMAC结构作为值函数逼近器，提出一种基于CMAC的非参数化近似策略迭代增强学习（NPAPI-CMAC）算法。算法通过构建样本采集过程确定CMAC泛化参数，利用初始划分和拓展划分确定CMAC状态划分方式，利用量化编码结构构建样本数集合定义增强学习率，实现了增强学习结构和参数的完全自动构建。此外，该算法利用delta规则和最近邻思想在学习过程中自适应调整增强学习参数，利用贪心策略对动作投票器得到的结果进行选择。一级倒立摆平衡控制的仿真实验结果验证了算法的有效性、鲁棒性和快速收敛能力。     

A robust and outlier-adaptive method for non-rigid point registration

In this paper, a flexible probabilistic method is introduced for non-rigid point registration, which is motivated by the pioneering research named Coherent Point Drift (CPD). Being different from CPD, our algorithm is robust and outlier-adaptive, which does not need prior information about data such as the appropriate outlier ratio when the point sets are perturbed by outliers. We consider the registration as the alignment of the data (one point set) to a set of Gaussian Mixture Model centroids (the other point set), and initially formulate it as maximizing the likelihood problem, then the problem is solved under Expectation–Maximization (EM) framework. The outlier ratio is also formulated in EM framework and will be updated during the EM iteration. Moreover, we use the volume of the point set region to determine the uniform distribution for modeling the outliers. The resulting registration algorithm exhibits inherent statistical robustness and has an explicit interpretation. The experiments demonstrate that our algorithm outperforms the state-of-the-art method.     

最优区域共空间模式的运动想象脑电信号分类方法

共空间模式(Common Spatial Pattern,CSP)是脑机接口(Brain-Computer Interface,BCI)中一种有效的特征提取方法,然而传统CSP算法并未考虑在提取前剔除可能会影响其性能的不相关的嘈杂通道信号。所以针对不同对象的通道选择问题,提出了一种最优区域共空间模式(ORCSP)特征提取方法。首先通过欧式距离得到每个通道的附近区域,再根据方差比选择可分性最高的区域,然后采用5折交叉验证对区域内通道数目进行寻优,进而得到区分度最高的区域特征,最后使用支持向量机(SVM)进行分类。所提方法在BCI竞赛数据上进行了实验测试,并与同类型的正则化CSP和局部区域CSP算法进行了对比,在BCI Competition Ⅲ Dataset Ⅳ a数据集上达到了89.78%的平均准确率。实验结果验证了所提出方法的有效性。     

多特征融合的运动想象脑电特征提取方法

针对单一特征识别率低、自适应性差等问题,提出一种基于希尔伯特-黄变换（HHT）和共同空间模式（CSP）的特征提取方法HCHT。首先,对原始脑电信号（EEG）进行经验模态分解（EMD）得到固有模态函数（IMF）,并将IMF分量合并成新的信号矩阵;然后,对IMF进行希尔伯特谱分析,得到信号的时-频域特征;接着,对构造的信号矩阵进行进一步的CSP分解,将时-频域特征扩展成时-频-空域特征;最后,通过支持向量机（SVM）对特征集进行分类。在BCI Competition II数据集的实验表明,与HHT时-频域和CSP空域特征的方法相比,所提方法的识别准确率分别提高了7.5、10.3和9.2个百分点,且标准差更小。在智能轮椅平台进行在线实验的结果表明,HCHT能有效提高识别准确率和稳定性。     

基于粒子滤波的非线性系统静态参数估计方法*

针对基于滤波方法的最大似然参数估计步长序列过于单一,算法收敛缓慢并很容易收敛于局部最优解的问题,提出了基于似然权值的在线EM参数估计算法(LWOEM)。通过粒子滤波方法实时估计系统的状态值变化,结合最大似然方法计算静态参数的点估计,然后通过计算更新参数的似然值来动态更新步长序列.与在线EM参数估计算法(OEM)的实验结果比较,表明该算法具有更好的适应性和收敛效果。     

基于人工蜂群优化高斯过程的运动想象脑电信号分类

针对传统的高斯过程采用共轭梯度法确定超参数时对初值有较强依赖性且易陷入局部最优的问题,提出了一种基于人工蜂群优化的高斯过程分类方法,用于脑电信号的模式识别.首先,构建高斯过程模型,选择合适的核函数且确定待优化的参数.然后,选取识别错误率的倒数为适应度函数,使用人工蜂群算法搜索寻找出限定范围内可以取得最优准确率的超参数.最后,采用参数优化后的高斯过程分类器对样本分类.分别采用2008年竞赛数据集BCI Competition Ⅳ Data Set 1和2005年数据集BCI Competition Ⅲ Data Set Ⅳa对所提方法进行验证,并与支持向量机(SVM)、人工蜂群优化的支持向量机(ABC-SVM)、高斯过程分类(GPC)方法进行比较,实验结果表明了所提方法的有效性.     

一种快速、贪心的高斯混合模型EM算法研究

针对传统EM算法存在初始模型成分数目需要预先指定以及收敛速度随样本数目的增长而急剧减慢等问题,提出了一种快速、贪心的高斯混合模型EM算法。该算法采用贪心的策略以及对隐含参数设置适当阈值的方法,使算法能够快速收敛,从而在很少的迭代次数内获取高斯混合模型的模型成分数。该算法通过与传统EM算法、无监督EM算法和鲁棒EM算法的聚类结果进行比较,实验结果证明该算法具有很强的鲁棒性,并且能够提高算法的效率以及模型成分数的准确性。     

Wavelet fuzzy LVQ based speaker verification system

This work evaluates the performance of speaker verification system based on Wavelet based Fuzzy Learning Vector Quantization (WLVQ) algorithm. The parameters of Gaussian mixture model (GMM) are designed using this proposed algorithm. Mel Frequency Cepstral Coefficients (MFCC) are extracted from the speech data and vector quantized through Wavelet based FLVQ algorithm. This algorithm develops a multi resolution codebook by updating both winning and nonwinning prototypes through an unsupervised learning process. This codebook is used as mean vector of GMM. The other two parameters, weight and covariance are determined from the clusters formed by the WLVQ algorithm. The multi resolution property of wavelet transform and ability of FLVQ in regulating the competition between prototypes during learning are combined in this algorithm to develop an efficient codebook for GMM. Because of iterative nature of Expectation Maximization (EM) algorithm, the applicability of alternative training algorithms is worth investigation. In this work, the performance of speaker verification system using GMM trained by LVQ, FLVQ and WLVQ algorithms are evaluated and compared with EM algorithm. FLVQ and WLVQ based training algorithms for modeling speakers using GMM yields better performance than EM based GMM.     

退火期望最大化算法A-EM

使用EM算法训练随机多层前馈网具有低开销、易于实现和全局收敛的特点,在EM算法的基础上提出了一种训练随机多层前馈网络的新方法AEM.AEM算法利用热力学系统的最大熵原理计算网络中隐变量的条件概率,借鉴退火过程,引入温度参数,减小了初始参数值对最终结果的影响.该算法既保持了原EM算法的优点,又有利于训练结果收敛到全局极小.从数学角度证明了该算法的收敛性,同时,实验也证明了该算法的正确性和有效性.     

Mutual information-based optimization of sparse spatio-spectral filters in brain–computer interface

Recently, neuro-rehabilitation based on brain–computer interface (BCI) has been considered one of the important applications for BCI. A key challenge in this system is the accurate and reliable detection of motor imagery. In motor imagery-based BCIs, the common spatial patterns (CSP) algorithm is widely used to extract discriminative patterns from electroencephalography signals. However, the CSP algorithm is sensitive to noise and artifacts, and its performance depends on the operational frequency band. To address these issues, this paper proposes a novel optimized sparse spatio-spectral filtering (OSSSF) algorithm. The proposed OSSSF algorithm combines a filter bank framework with sparse CSP filters to automatically select subject-specific discriminative frequency bands as well as to robustify against noise and artifacts. The proposed algorithm directly selects the optimal regularization parameters using a novel mutual information-based approach, instead of the cross-validation approach that is computationally intractable in a filter bank framework. The performance of the proposed OSSSF algorithm is evaluated on a dataset from 11 stroke patients performing neuro-rehabilitation, as well as on the publicly available BCI competition III dataset IVa. The results show that the proposed OSSSF algorithm outperforms the existing algorithms based on CSP, stationary CSP, sparse CSP and filter bank CSP in terms of the classification accuracy, and substantially reduce the computational time of selecting the regularization parameters compared with the cross-validation approach.     

基于困惑度数据挑选的半监督声学建模*

针对资源稀少情况下小语种的声学建模问题,提出根据解码后文本的困惑度挑选无监督数据并重新训练声学模型的策略.使用少量精标数据训练得到一个初始种子模型后,解码大量无监督数据,计算解码后的文本与精标数据文本的困惑度,从中挑选与精标数据相近的数据,再将这些数据与原有精标数据共同用于声学模型训练.为了提高解码的无监督数据的正确性,在基于深层神经网络的模型参数训练中,当最后一次模型参数更新时只使用精标数据修正网络参数.在NIST 2015年关键词识别比赛中Swahili语的VLLP识别任务上,相比其它方法,文中方法的识别率有一定提升.     

基于特征和基于样本的EM方法

本文主要讨论在非监督学习中，用EM方法求解有限混合分布(己知类别数)未知参数的问题。由于通过方程式直接求得参数比较困难，一般采用迭代法，但是也存在计算量过大的问题。大多数文献中的计算都是逐个样本进行的(基于样本的EM方法)，我们证明：把特征相同的样本放在一起计算(基于特征的EM方法)和基于样本的EM方法完全等价。这样，EM方法的计算量可以大幅度减少，再加上其他措施(如去除频率为0的特征、适当量化)，其计算速度完全可以达到实际运用的要求。     

基于支持向量机的脑电信号中左右手判别

在脑-机接口的研究中分类识别技术占有重要地位。介绍了一种方法,用于对单次信号的分类。这种方法主要思想是将共空域子空间分解和支持向量机相结合,以便提取信号特征。该方法被用于“BCI Competition 2003”第IV数据包上,分类正确率达89%。     

基于公共空间频率模型的脑电数据分类

利用公共空间频率模型算法实现较少训练数据的脑电识别。首先给出公共空间频率模型算法的数学公式和求解过程,然后从数学分析角度说明表达实质含义,以及如何实现空间和频率上的同时滤波;再对于提取出来的特征,如何构造分类器,进行分类;最后用BCI Competition III的Iva数据包为数据源,证明了公共空间频率模型算法在BCI的实用性,能够达到更好的识别效果。     

On-line EM Variants for Multivariate Normal Mixture Model in Background Learning and Moving Foreground Detection

The unsupervised learning of multivariate mixture models from on-line data streams has attracted the attention of researchers for its usefulness in real-time intelligent learning systems. The EM algorithm is an ideal choice for iteratively obtaining maximum likelihood estimation of parameters in presumable finite mixtures, comparing to some popular numerical methods. However, the original EM is a batch algorithm that works only on fixed datasets. To endow the EM algorithm with the capability to process streaming data, two on-line variants are studied, including Titterington’s method and a sufficient statistics-based method. We first prove that the two on-line EM variants are theoretically feasible for training the multivariate normal mixture model by showing that the model belongs to the exponential family. Afterward, the two on-line learning schemes for multivariate normal mixtures are applied to the problems of background learning and moving foreground detection. Experiments show that the two on-line EM variants can efficiently update the parameters of the mixture model and are capable of generating reliable backgrounds for moving foreground detection.     

基于在线分裂合并EM算法的高斯混合模型分类方法*

为了解决传统高斯混合模型中期望值EM处理必须具备足够数量的样本才能开始训练的问题,提出了一种新的高斯混合模型在线增量训练算法。本算法在Ueda等人提出的Split-and-Merge EM方法基础上对分裂合并准则的计算进行了改进,能够有效避免陷入局部极值并减少奇异值出现的情况;通过引入时间序列参数提出了增量EM训练方法,能够实现增量式的期望最大化训练,从而能够逐样本在线更新GMM模型参数。对合成数据和实际语音识别应用的实验结果表明,本算法具有较好的运算效率和分类准确性。     

A modified Smith predictive scheme based back-propagation neural network approach for FOPDT processes control

In this paper, a novel control scheme to deal with process uncertainties in the form of disturbance loads and modelling errors, as well as time-varying process parameters is proposed by applying the back-propagation neural network (BPNN) approach. A BPNN predictive controller that replaces the entire Smith predictor structure is initially trained offline. Lyapunov direct method is used to prove that the convergence of this BPNN is guaranteed by selecting a suitable learning rate during the learning process. However, the Smith predictor based BPNN control is an off-line training based algorithm, which is a time consuming method and requires a known process plant input from the controller. A desired control input to the process is difficult to obtain for the training of the network. As a result a group of proper training data (target control inputs and outputs) can hardly be provided. In order to overcome this problem, a BPNN with an on-line training algorithm is introduced for the control of a First Order plus Dead Time (FOPDT) process. The stability analysis is carried out using the Lyapunov criterion to demonstrate the network convergence ability. Simulation results show that this proposed online trained neural Smith predictor based controller provides excellent robustness to process modelling errors and disturbance loads, and high adaptability to time varying processes parameters.     

An stable online clustering fuzzy neural network for nonlinear system identification

In this paper, we propose a online clustering fuzzy neural network. The proposed neural fuzzy network uses the online clustering to train the structure, the gradient to train the parameters of the hidden layer, and the Kalman filter algorithm to train the parameters of the output layer. In our algorithm, learning structure and parameter learning are updated at the same time, we do not make difference in structure learning and parameter learning. The center of each rule is updated to obtain the center is near to the incoming data in each iteration. In this way, it does not need to generate a new rule in each iteration, i.e., it neither generates many rules nor need to prune the rules. We prove the stability of the algorithm.     

奇异值分解下在线鲁棒正则化随机网络

在线鲁棒随机权神经网络(OR-RVFLN)具有较好的逼近性、较快的收敛速度、较高的鲁棒性能以及较小的存储空间.但是, OR-RVFLN算法计算过程中会产生矩阵的不适定问题,使得隐含层输出矩阵的精度较低.针对这个问题,本文提出了奇异值分解下在线鲁棒正则化随机网络(SVD-OR-RRVFLN).该算法在OR-RVFLN算法的基础上,将正则化项引入到权值的估计中,并且对隐含层输出矩阵进行奇异值分解;同时采用核密度估计(KDE)法,对整个SVD-OR-RRVFLN网络的权值矩阵进行更新,并分析了所提算法的必要性和收敛性.最后,将所提的方法应用于Benchmark数据集和磨矿粒度的指标预测中,实验结果证实了该算法不仅可以有效地提高模型的预测精度和鲁棒性能,而且具有更快的训练速度.