Speaker recognition using pyramid match kernel based support vector machines

Gaussian mixture model (GMM) based approaches have been commonly used for speaker recognition tasks. Methods for estimation of parameters of GMMs include the expectation-maximization method which is a non-discriminative learning based method. Discriminative classifier based approaches to speaker recognition include support vector machine (SVM) based classifiers using dynamic kernels such as generalized linear discriminant sequence kernel, probabilistic sequence kernel, GMM supervector kernel, GMM-UBM mean interval kernel (GUMI) and intermediate matching kernel. Recently, the pyramid match kernel (PMK) using grids in the feature space as histogram bins and vocabulary-guided PMK (VGPMK) using clusters in the feature space as histogram bins have been proposed for recognition of objects in an image represented as a set of local feature vectors. In PMK, a set of feature vectors is mapped onto a multi-resolution histogram pyramid. The kernel is computed between a pair of examples by comparing the pyramids using a weighted histogram intersection function at each level of pyramid. We propose to use the PMK-based SVM classifier for speaker identification and verification from the speech signal of an utterance represented as a set of local feature vectors. The main issue in building the PMK-based SVM classifier is construction of a pyramid of histograms. We first propose to form hard clusters, using k-means clustering method, with increasing number of clusters at different levels of pyramid to design the codebook-based PMK (CBPMK). Then we propose the GMM-based PMK (GMMPMK) that uses soft clustering. We compare the performance of the GMM-based approaches, and the PMK and other dynamic kernel SVM-based approaches to speaker identification and verification. The 2002 and 2003 NIST speaker recognition corpora are used in evaluation of different approaches to speaker identification and verification. Results of our studies show that the dynamic kernel SVM-based approaches give a significantly better performance than the state-of-the-art GMM-based approaches. For speaker recognition task, the GMMPMK-based SVM gives a performance that is better than that of SVMs using many other dynamic kernels and comparable to that of SVMs using state-of-the-art dynamic kernel, GUMI kernel. The storage requirements of the GMMPMK-based SVMs are less than that of SVMs using any other dynamic kernel.     

基于KL散度的支持向量机方法及应用研究

针对ICA提取的说话人语音特征,导出以库尔贝克—莱布勒（KL）散度作为距离测度的KL核函数用来设计支持向量机,实现了一个高分辨率的ICA/SVM说话人确认系统.说话人确认的仿真实验结果表明,使用ICA特征基函数系数比直接使用语音数据训练SVM得到的分类间隔大,支持向量少,而且使用KL核函数的ICA/SVM系统确认的等差率也低于其它传统SVM方法,证明了基于KL散度的支持向量机方法在实现分类和判决上具有高效性能.     

Support vector learning for fuzzy rule-based classification systems

To design a fuzzy rule-based classification system (fuzzy classifier) with good generalization ability in a high dimensional feature space has been an active research topic for a long time. As a powerful machine learning approach for pattern recognition problems, the support vector machine (SVM) is known to have good generalization ability. More importantly, an SVM can work very well on a high- (or even infinite) dimensional feature space. This paper investigates the connection between fuzzy classifiers and kernel machines, establishes a link between fuzzy rules and kernels, and proposes a learning algorithm for fuzzy classifiers. We first show that a fuzzy classifier implicitly defines a translation invariant kernel under the assumption that all membership functions associated with the same input variable are generated from location transformation of a reference function. Fuzzy inference on the IF-part of a fuzzy rule can be viewed as evaluating the kernel function. The kernel function is then proven to be a Mercer kernel if the reference functions meet a certain spectral requirement. The corresponding fuzzy classifier is named positive definite fuzzy classifier (PDFC). A PDFC can be built from the given training samples based on a support vector learning approach with the IF-part fuzzy rules given by the support vectors. Since the learning process minimizes an upper bound on the expected risk (expected prediction error) instead of the empirical risk (training error), the resulting PDFC usually has good generalization. Moreover, because of the sparsity properties of the SVMs, the number of fuzzy rules is irrelevant to the dimension of input space. In this sense, we avoid the "curse of dimensionality." Finally, PDFCs with different reference functions are constructed using the support vector learning approach. The performance of the PDFCs is illustrated by extensive experimental results. Comparisons with other methods are also provided.     

Ginisupport vector machines for segmental minimum Bayes risk decoding of continuous speech

We describe the use of support vector machines (SVMs) for continuous speech recognition by incorporating them in segmental minimum Bayes risk decoding. Lattice cutting is used to convert the Automatic Speech Recognition search space into sequences of smaller recognition problems. SVMs are then trained as discriminative models over each of these problems and used in a rescoring framework. We pose the estimation of a posterior distribution over hypotheses in these regions of acoustic confusion as a logistic regression problem. We also show that GiniSVMs can be used as an approximation technique to estimate the parameters of the logistic regression problem. On a small vocabulary recognition task we show that the use of GiniSVMs can improve the performance of a well trained hidden Markov model system trained under the Maximum Mutual Information criterion. We also find that it is possible to derive reliable confidence scores over the GiniSVM hypotheses and that these can be used to good effect in hypothesis combination. We discuss the problems that we expect to encounter in extending this approach to large vocabulary continuous speech recognition and describe initial investigation of constrained estimation techniques to derive feature spaces for SVMs.     

Training SVMs on a bound vectors set based on Fisher projection

Standard support vector machines (SVMs) training algorithms have O(l ³) computational and O(l ²) space complexities, where l is the training set size. It is thus computationally infeasible on very large data sets. To alleviate the computational burden in SVM training, we propose an algorithm to train SVMs on a bound vectors set that is extracted based on Fisher projection. For linear separate problems, we use linear Fisher discriminant to compute the projection line, while for non-linear separate problems, we use kernel Fisher discriminant to compute the projection line. For each case, we select a certain ratio samples whose projections are adjacent to those of the other class as bound vectors. Theoretical analysis shows that the proposed algorithm is with low computational and space complexities. Extensive experiments on several classification benchmarks demonstrate the effectiveness of our approach.     

An enhanced Support Vector Machine classification framework by using Euclidean distance function for text document categorization

This paper presents the implementation of a new text document classification framework that uses the Support Vector Machine (SVM) approach in the training phase and the Euclidean distance function in the classification phase, coined as Euclidean-SVM. The SVM constructs a classifier by generating a decision surface, namely the optimal separating hyper-plane, to partition different categories of data points in the vector space. The concept of the optimal separating hyper-plane can be generalized for the non-linearly separable cases by introducing kernel functions to map the data points from the input space into a high dimensional feature space so that they could be separated by a linear hyper-plane. This characteristic causes the implementation of different kernel functions to have a high impact on the classification accuracy of the SVM. Other than the kernel functions, the value of soft margin parameter, C is another critical component in determining the performance of the SVM classifier. Hence, one of the critical problems of the conventional SVM classification framework is the necessity of determining the appropriate kernel function and the appropriate value of parameter C for different datasets of varying characteristics, in order to guarantee high accuracy of the classifier. In this paper, we introduce a distance measurement technique, using the Euclidean distance function to replace the optimal separating hyper-plane as the classification decision making function in the SVM. In our approach, the support vectors for each category are identified from the training data points during training phase using the SVM. In the classification phase, when a new data point is mapped into the original vector space, the average distances between the new data point and the support vectors from different categories are measured using the Euclidean distance function. The classification decision is made based on the category of support vectors which has the lowest average distance with the new data point, and this makes the classification decision irrespective of the efficacy of hyper-plane formed by applying the particular kernel function and soft margin parameter. We tested our proposed framework using several text datasets. The experimental results show that this approach makes the accuracy of the Euclidean-SVM text classifier to have a low impact on the implementation of kernel functions and soft margin parameter C.     

基于线性对数似然核函数的说话人识别

为了提高文本无关的说话人识别系统的性能,提出了基于线性对数似然核函数的说话人识别系统。线性对数似然核函数利用高斯混合模型对频谱特征序列进行压缩;将频谱特征序列之间的相似程度转化为高斯混合模型参数之间的距离;根据距离表达式,利用极化恒等式求得频谱特征序列向高维矢量空间的映射方法;最后,在高维矢量空间,采用支持向量机(SVM)为目标说话人建立模型。在美国国家标准技术署公布的说话人识别数据库上的实验结果表明,所提核函数具有优异的识别性能。     

一种概率序列核在说话人识别中的应用

以说话人识别中的背景模型为基础,根据模型中的各个高斯分量,构造出说话人特征空间,将长度不一样的语句映射成为空间中大小相同的向量,且经过相关矩阵进行规整后,采用线性支持向量机进行说话人识别。借鉴几种常见的特征规整方式,结合语句映射后的向量,提出四种不同的规整方法：均值/方差规整、权重规整、WLOG规整和球形规整,并与概率序列核进行比较研究。根据语音特征向量序列中相邻的特征向量的前后转移关系,结合提出的概率序列核,构造出转移概率序列核。实验在NIST2001库上进行,结果表明概率序列核模型识别性能接近经典的UBM-MAP模型,将这两类模型得分进行融合,可非常明显地提高识别性能,进一步融合转移概率序列核后,性能还可提高19.1%。     

Predicting business failure using support vector machines with straightforward wrapper: A re-sampling study

Business failure prediction (BFP) is an effective tool to help financial institutions and relevant people to make the right decision in investments, especially in the current competitive environment. This topic belongs to a classification-type task, one of whose aims is to generate more accurate hit ratios. Support vector machine (SVM) is a statistical learning technique, whose advantage is its high generalization performance. The objective of this context is threefold. Firstly, SVM is used to predict business failure by utilizing a straightforward wrapper approach to help the model produce more accurate prediction. The wrapper approach is fulfilled by employing a forward feature selection method, composed of feature ranking and feature selection. Meanwhile, this work attempts to investigate the feasibility of using linear SVMs to select features for all SVMs in the wrapper since non-linear SVMs yield to over-fit the data. Finally, a robust re-sampling approach is used to evaluate model performances for the task of BFP in China. In the empirical research, performances of linear SVM, polynomial SVM, Gaussian SVM, and sigmoid SVM with the best filter of stepwise MDA, and wrappers respectively using linear SVM and non-linear SVMs as evaluating functions are to be compared. The results indicate that the non-linear SVM with radial basis function kernel and features selected by linear SVM compare significantly superiorly to all the other SVMs. Meanwhile, all SVMs with features selected by linear SVM produce at least as good performances as SVMs with other optimal features.     

高斯序列核支持向量机用于说话人识别

说话人识别问题具有重要的理论价值和深远的实用意义,在研究支持向量机核方法理论的基础上,将其与传统高斯混合模型（GMM）相结合构建成基于高斯序列核的支持向量机（SVM）。SVM的灵活性和强大分类能力主要在于可以根据要处理的问题来相应的选取核函数。在识别的过程中引入特征空间归正技术NAP（Nuisance Attribute Projection）对同一说话人在不同信道和环境所带来的特征差异进行弥补。用美国国家标准与技术研究所（NIST）2004年评测数据集进行实验,结果表明该方法可以大幅度提高识别率。     

A novel hybrid KPCA and SVM with GA model for intrusion detection

A novel support vector machine (SVM) model combining kernel principal component analysis (KPCA) with genetic algorithm (GA) is proposed for intrusion detection. In the proposed model, a multi-layer SVM classifier is adopted to estimate whether the action is an attack, KPCA is used as a preprocessor of SVM to reduce the dimension of feature vectors and shorten training time. In order to reduce the noise caused by feature differences and improve the performance of SVM, an improved kernel function (N-RBF) is proposed by embedding the mean value and the mean square difference values of feature attributes in RBF kernel function. GA is employed to optimize the punishment factor C, kernel parameters σ and the tube size ɛ of SVM. By comparison with other detection algorithms, the experimental results show that the proposed model performs higher predictive accuracy, faster convergence speed and better generalization.     

NAP序列核函数在话者识别中的应用

针对话者识别系统中特征向量不定长和交叉信道干扰等问题,提出一种基于超向量的扰动属性投影(NAP)核函数。该函数是一种新型的序列核函数,使支持向量机能在整体语音序列上分类,移除核函数空间中与话者识别无关的信道子空间信息。仿真实验结果表明,该函数可有效提高支持向量机的分类性能和话者识别系统的识别准确率。     

Automatic detection of erythemato-squamous diseases using PSO–SVM based on association rules

In this paper, we develop a diagnosis model based on particle swarm optimization (PSO), support vector machines (SVMs) and association rules (ARs) to diagnose erythemato-squamous diseases. The proposed model consists of two stages: first, AR is used to select the optimal feature subset from the original feature set; then a PSO based approach for parameter determination of SVM is developed to find the best parameters of kernel function (based on the fact that kernel parameter setting in the SVM training procedure significantly influences the classification accuracy, and PSO is a promising tool for global searching). Experimental results show that the proposed AR_PSO–SVM model achieves 98.91% classification accuracy using 24 features of the erythemato-squamous diseases dataset taken from UCI (University of California at Irvine) machine learning database. Therefore, we can conclude that our proposed method is very promising compared to the previously reported results.     

核对齐多核模糊支持向量机

支持向量机（SVMs）是当前被广泛使用的机器学习技术,其通过最优分割超平面来提高分类器的泛化能力,在实际应用中表现优异。然而SVM也存在易受噪声影响,以及核函数选择等难题。针对以上问题,本文将基于核对齐的多核学习方法引入到模糊支持向量机（fuzzy support vector machine, FSVM）中,提出了模糊多核支持向量机模型（multiple kernel fuzzy support vector machine,MFSVM）。MFSVM通过模糊粗糙集方法计算每一样例隶属度;其次,利用核对齐的多核方法计算每一单核权重,并将组合核引入到模糊支持向量机中。该方法不仅提高了支持向量机的抗噪声能力,也有效避免了核选择难题。在UCI数据库上进行实验,结果表明本文所提方法具有较高的分类精度,验证了该方法的可行性与有效性。     

A support vector machine with a hybrid kernel and minimal Vapnik-Chervonenkis dimension

We present a mechanism to train support vector machines (SVMs) with a hybrid kernel and minimal Vapnik-Chervonenkis (VC) dimension. After describing the VC dimension of sets of separating hyperplanes in a high-dimensional feature space produced by a mapping related to kernels from the input space, we proposed an optimization criterion to design SVMs by minimizing the upper bound of the VC dimension. This method realizes a structural risk minimization and utilizes a flexible kernel function such that a superior generalization over test data can be obtained. In order to obtain a flexible kernel function, we develop a hybrid kernel function and a sufficient condition to be an admissible Mercer kernel based on common Mercer kernels (polynomial, radial basis function, two-layer neural network, etc.). The nonnegative combination coefficients and parameters of the hybrid kernel are determined subject to the minimal upper bound of the VC dimension of the learning machine. The use of the hybrid kernel results in a better performance than those with a single common kernel. Experimental results are discussed to illustrate the proposed method and show that the SVM with the hybrid kernel outperforms that with a single common kernel in terms of generalization power.     

Forecasting foreign exchange rates using kernel methods

First, the all-important no free lunch theorems are introduced. Next, kernel methods, support vector machines (SVMs), preprocessing, model selection, feature selection, SVM software and the Fisher kernel are introduced and discussed. A hidden Markov model is trained on foreign exchange data to derive a Fisher kernel for an SVM, the DC algorithm and the Bayes point machine (BPM) are also used to learn the kernel on foreign exchange data. Further, the DC algorithm was used to learn the parameters of the hidden Markov model in the Fisher kernel, creating a hybrid algorithm. The mean net returns were positive for BPM; and BPM, the Fisher kernel, the DC algorithm and the hybrid algorithm were all improvements over a standard SVM in terms of both gross returns and net returns, but none achieved net returns as high as the genetic programming approach employed by Neely, Weller, and Dittmar (1997) and published in Neely, Weller, and Ulrich (2009). Two implementations of SVMs for Windows with semi-automated parameter selection are built.     

Identity management based on PCA and SVM

A new approach for face recognition, based on kernel principal component analysis (KPCA) and support vector machines (SVMs), is presented to improve the recognition performance of the method based on principal component analysis (PCA). This method can simultaneously be applied to solve both the over-fitting problem and the small sample problem. The KPCA method is performed on every facial image of the training set to get the core facial features of the training samples. To ensure that the loss of the image information will be as less as possible, the facial data of high-dimensional feature space is projected into low-dimensional space, and then the SVM face recognition model is established to identify the low-dimensional space facial data. Our experimental results demonstrate that the approach proposed in this paper is efficient, and the recognition accuracy of the proposed method reaches 95.4 %.     

Sparse least squares support vector training in the reduced empirical feature space

In this paper we discuss sparse least squares support vector machines (sparse LS SVMs) trained in the empirical feature space, which is spanned by the mapped training data. First, we show that the kernel associated with the empirical feature space gives the same value with that of the kernel associated with the feature space if one of the arguments of the kernels is mapped into the empirical feature space by the mapping function associated with the feature space. Using this fact, we show that training and testing of kernel-based methods can be done in the empirical feature space and that training of LS SVMs in the empirical feature space results in solving a set of linear equations. We then derive the sparse LS SVMs restricting the linearly independent training data in the empirical feature space by the Cholesky factorization. Support vectors correspond to the selected training data and they do not change even if the value of the margin parameter is changed. Thus for linear kernels, the number of support vectors is the number of input variables at most. By computer experiments we show that we can reduce the number of support vectors without deteriorating the generalization ability.

基于流形正则化极限学习机的语种识别系统

支持向量机 (Support vector machine, SVM) 在语种识别中已经起到了重要的作用.近些年来,极限学习机 (Extreme learning machine, ELM) 在很多领域取得了成功的应用.相比于 SVM, ELM 最大的优点在于极易实现、训练速度快,而且通常可以取得与 SVM 相近甚至优于 SVM 的识别性能. 鉴于 ELM 这些优异的特点,本文将 ELM 引入到语种识别中,并针对 ELM 由于随机初始化模型参 数所带来的潜在问题,提出了流形正则化极限学习机 (Manifold regularized extreme learning machine, MRELM) 算法.实验结果表明,在高斯超矢量(Gaussian supervector, GSV)特征空间上,相对于 SVM 基线系统,该算法对30秒语音的识别性能有明显的提升. 同时该算法也可以成功地应用到 i-vector 特征空间中,取得与当前主流的打分算法相近的识别性能.     

基于支撑向量机的说话人确认系统

支撑向量机(SVM)是一种新的统计学习方法,和以往的学习方法不同的是SVM的学习原则是使结构风险(Structural Risk)最小,而经典的学习方法遵循经验风险(Empirical Risk)最小原则,这使得SVM具有较好的总体性能.文章提出一种基于支撑向量机的文本无关的说话人确认系统,实验表明同基于向量量化(VQ)和高斯混合模式(GMM)的经典方法相比,基于SVM的方法具有更高的区分力和更好的总体性能.