多模型数据集的免疫鲁棒回归分析

针对传统多模型数据集回归分析方法计算时间长、模型识别准确率低的问题,提出了一种新的启发式鲁棒回归分析方法。该方法模拟免疫系统聚类学习的原理,采用B细胞网络作为数据集的分类和存储工具,通过判断数据对模型的符合度进行分类,提高了数据分类的准确性,将模型集抽取过程分解成“聚类”“回归”“再聚类”的反复尝试过程,利用并行启发式搜索逼近模型集的解。仿真结果表明,所提方法回归分析时间明显少于传统算法,模型识别准确率明显高于传统算法。根据8模型数据集分析结果,传统算法中,效果最好的是基于RANSAC的逐次提取算法,其平均模型识别准确率为90.37%,需53.3947s;计算时间小于0.5s的传统算法,其准确率不足1%;所提算法仅需0.5094s,其准确率达到了98.25%。     

随机森林在滚动轴承故障诊断中的应用

针对不同轴承数据特征选择困难和单个分类器方法在滚动轴承故障诊断中精度较低的问题,提出了一种基于分类回归树（CART）的随机森林滚动轴承故障诊断算法。随机森林是包含了多种分类器的集成学习方法。通过随机森林的“集成”思想来提高滚动轴承故障诊断的精度。从滚动轴承的振动信号中提取时域统计指标,将其作为特征向量,利用随机森林（Random Forest）对滚动轴承故障进行诊断。利用SQI-MFS实验平台的轴承数据,与传统分类器（SVM、kNN和ANN）以及单个分类回归树的诊断结果相比,随机森林算法具有比较高的诊断精度。     

一种多结构数据的同步拟合方法

计算机视觉领域,多结构模型参数的提取是一个常见任务。传统的提取算法一般先对输入数据集进行分类,然后通过对相关数据类的拟合获得对模型集参数的估计。然而,由于模型集未知,对如何实现数据集的准确划分一直是一个难点。针对这个问题,借鉴免疫系统识别抗原产生抗体的工作机理,提出了一种启发式的鲁棒回归分析方法。该方法将数据集的分类过程设计成一个逐步精确化的逼近过程：先通过随机抽样模型对数据集进行粗略划分,然后利用单模型鲁棒回归方法对各数据类中的优势数据进行尝试建模,以获得更好的模型估计。接着以此模型为基础重新对数据集进行划分,以提高分类的准确性。通过这种“分类”、“提纯”、“再分类”、“再提纯”的反复尝试,逐步逼近准确的数据类划分,同时得到模型集的准确解。仿真结果表明,该方法计算时间少,数据分类准确率高,具有较强的多结构模型参数提取能力。     

稀疏结构化最小二乘双支持向量回归机

最小二乘双支持向量回归机（LSTSVR）通过引入最小二乘损失将双支持向量回归机（TSVR）中的二次规划问题简化为两个线性方程组的求解,从而大大减少了训练时间。然而,LSTSVR最小化基于最小二乘损失的经验风险易导致以下不足：（1）“过学习”问题;（2）模型的解缺乏稀疏性,难以训练大规模数据。针对（1）,提出结构化最小二乘双支持向量回归机（S-LSTSVR）以提升模型的泛化能力;针对（2）,进一步利用不完全Choesky分解对核矩阵进行低秩近似,给出求解S-LSTSVR的稀疏算法SS-LSTSVR,使模型能有效地训练大规模数据。人工数据和UCI数据集中的实验证明SS-LSTSVR不但可以避免“过学习”,而且能够高效地解决大规模训练问题。     

基于逻辑回归的多任务域快速分类学习算法

多任务学习通过寻找并共享不同任务域之间的共性特征来完成学习,利用知识迁移加速不同任务域的学习为每个任务域构建一个分类器。提出了一种基于罗杰斯特回归模型的多任务学习方法MTC-LR（Multi-task Coupled Logistic Regression）。“罗杰斯特回归模型”已经被成功应用于单任务分类器上,该模型被众多实验证明是有效的,正是这种方法给人们带来了启示。从理论上证明了通过构造多任务分类器的“开销函数”和“差异性度量函数”,MTC-LR算法可以提高多任务分类器的各自分类精度。相比传统的基于SVM的多任务学习方法,MTC-LR并不依赖于核方法而是通过共轭梯度下降法寻找各个分类器的最优参数。同时MTC-LR与采用“罗杰斯特回归模型”的快速算法CDdual更容易结合,可扩展至大样本的多任务分类学习。正是基于上述发现,为了充分高效利用大样本的多任务域数据,满足大样本的快速运算,在MTC-LR算法的基础上,结合最新的CDdual（The Dual Coordinate Descent Method）算法,提出了MTC-LR的快速算法MTC-LR-CDdual,并对该算法进行了相关的理论分析。将该算法在人工数据集和真实数据集上进行了验证,实验结果表明该算法有着较高的识别率、快速的识别速度和较好的鲁棒性。     

一种新的回归型约简多分辨率相关向量机

提出一种新的稀疏贝叶斯回归算法.基于相关向量机,首先通过尺度核和小波核构造完备基以提高预测精度;然后利用保局投影对输入矩阵的列进行主成分提取以减少训练时间,从而形成算法的初步模型.为进一步减小较大规模训练数据集的回归时间压力,算法对训练数据集的分层采样建立了初步模型,进而产生实际较小规模的训练数据集.实验结果表明,算法在预测精度和鲁棒性上优于传统支持向量机和相关向量机,且其训练时间较相关向量机少.     

基于地形信息的Landsat与Radarsat-2遥感数据协同分类研究

针对不同成像机理的光学与雷达遥感数据协同应用于地表信息提取瓶颈问题,提出了一种基于地形信息的光学与雷达数据协同分类方法。首先利用InSAR测量技术从Radarsat-2数据中提取DEM地形信息,然后构建基于地形信息的Landsat光学数据和Radarsat-2雷达数据的不同特征集输入模型,最后通过随机样本选取构建随机森林（Random Forest,RF）、支持向量机（Support Vector Machine, SVM）和决策树（Decision Tree,DT）分类算法模型提取地表信息。结果表明：①针对不同特征协同策略,在随机选取10%训练样本时,Radarsat-2干涉提取DEM与Landsat数据集提取精度优于ASTER GDEM与光学影像协同策略;②针对不同地表信息提取算法模型,通过50次随机选取训练样本构建模型评价分类精度,验证RF算法的鲁棒性和提取精度都要优于DT算法和SVM算法。研究充分利用光学和雷达遥感的优势信息,为光学和雷达遥感协同地表信息提取提供新的思路。     

核向量机与支持向量机相结合的二阶段快速学习方法

支持向量机(SVM)作为一种有效的模式分类方法,当数据集规模较大时,学习时间长、泛化能力下降;而核向量机(CVM)分类算法的时间复杂度与样本规模无关,但随着支持向量的增加,CVM的学习时间会快速增长。针对以上问题,提出一种CVM与SVM相结合的二阶段快速学习算法(CCS),首先使用CVM初步训练样本,基于最小包围球(MEB)筛选出潜在核向量,构建新的最有可能影响问题解的训练样本,以此降低样本规模,并使用标记方法快速提取新样本;然后对得到的新训练样本使用SVM进行训练。通过在6个数据集上与SVM和CVM进行比较,实验结果表明,CCS在保持分类精度的同时训练时间平均减少了30%以上,是一种有效的大规模分类学习算法。     

自注意力信用评估模型

在信用评估问题中，用户信息中既包含类别数据，也包含数值数据。传统的基于人工智能的信用评估模型通常对类别数据进行one-hot变换后，再与数值数据进行拼接作为判别器的输入。与之不同，借鉴了自然语言处理中的词嵌入技术来提取类别数据的词向量；将输入的词向量集合类比为“句子”，并基于自注意力机制从“句子”中提取出用户特征；最后采用多层感知机来预测用户违约的概率。新模型可以使用反向传播算法实现端到端的训练。在三个不同的数据集上将新模型和六种基准算法进行了比较，结果表明该模型能够比基准算法取得更好的性能。     

基于云模型的增量SVM入侵检测方法

针对支持向量机增量算法中边界样本的提取机制效率不高的问题,提出基于云模型的增量SVM入侵检测方法。该算法利用云模型稳定性和不确定性的特点,将异类样本间的特征距离映射成隶属度函数,对初始集中边界向量进行提取。分析新增样本对支持向量集的影响,淘汰无用样本。理论分析和仿真实验表明,该算法在保证分类精度的同时有效地提高了检测速度。     

Rule Extraction from Support Vector Machines: A Sequential Covering Approach

In this paper, we propose a novel algorithm for rule extraction from support vector machines (SVMs), termed SQRex-SVM. The proposed method extracts rules directly from the support vectors (SVs) of a trained SVM using a modified sequential covering algorithm. Rules are generated based on an ordered search of the most discriminative features, as measured by interclass separation. Rule performance is then evaluated using measured rates of true and false positives and the area under the receiver operating characteristic (ROC) curve (AUC). Results are presented on a number of commonly used data sets that show the rules produced by SQRex-SVM exhibit both improved generalization performance and smaller more comprehensible rule sets compared to both other SVM rule extraction techniques and direct rule learning techniques.     

Access the IEEE Computer Society Digital Library

In this paper, we propose a novel algorithm for rule extraction from support vector machines (SVMs), termed SQRex-SVM. The proposed method extracts rules directly from the support vectors (SVs) of a trained SVM using a modified sequential covering algorithm. Rules are generated based on an ordered search of the most discriminative features, as measured by interclass separation. Rule performance is then evaluated using measured rates of true and false positives and the area under the receiver operating characteristic (ROC) curve (AUC). Results are presented on a number of commonly used data sets that show the rules produced by SQRex-SVM exhibit both improved generalization performance and smaller more comprehensible rule sets compared to both other SVM rule extraction techniques and direct rule learning techniques     

Rule-Based Learning Systems for Support Vector Machines

In this article, we propose some methods for deriving symbolic interpretation of data in the form of rule based learning systems by using Support Vector Machines (SVM). First, Radial Basis Function Neural Networks (RBFNN) learning techniques are explored, as is usual in the literature, since the local nature of this paradigm makes it a suitable platform for performing rule extraction. By using support vectors from a learned SVM it is possible in our approach to use any standard Radial Basis Function (RBF) learning technique for the rule extraction, whilst avoiding the overlapping between classes problem. We will show that merging node centers and support vectors explanation rules can be obtained in the form of ellipsoids and hyper-rectangles. Next, in a dual form, following the framework developed for RBFNN, we construct an algorithm for SVM. Taking SVM as the main paradigm, geometry in the input space is defined from a combination of support vectors and prototype vectors obtained from any clustering algorithm. Finally, randomness associated with clustering algorithms or RBF learning is avoided by using only a learned SVM to define the geometry of the studied region. The results obtained from a certain number of experiments on benchmarks in different domains are also given, leading to a conclusion on the viability of our proposal.     

Decompositional Rule Extraction from Support Vector Machines by Active Learning

Support vector machines (SVMs) are currently state-of-the-art for the classification task and, generally speaking, exhibit good predictive performance due to their ability to model nonlinearities. However, their strength is also their main weakness, as the generated nonlinear models are typically regarded as incomprehensible black-box models. In this paper, we propose a new active learning-based approach (ALBA) to extract comprehensible rules from opaque SVM models. Through rule extraction, some insight is provided into the logics of the SVM model. ALBA extracts rules from the trained SVM model by explicitly making use of key concepts of the SVM: the support vectors, and the observation that these are typically close to the decision boundary. Active learning implies the focus on apparent problem areas, which for rule induction techniques are the regions close to the SVM decision boundary where most of the noise is found. By generating extra data close to these support vectors that are provided with a class label by the trained SVM model, rule induction techniques are better able to discover suitable discrimination rules. This performance increase, both in terms of predictive accuracy as comprehensibility, is confirmed in our experiments where we apply ALBA on several publicly available data sets.     

A hierarchical multiclass support vector machine incorporated with holistic triple learning units

This paper proposes a new hierarchical learning structure, namely the holistic triple learning (HTL), for extending the binary support vector machine (SVM) to multi-classification problems. For an N-class problem, a HTL constructs a decision tree up to a depth of éN/3ù+1\lceil N/3\rceil+1. A leaf node of the decision tree is allowed to be placed with a holistic triple learning unit whose generalisation abilities are assessed and approved. Meanwhile, the remaining nodes in the decision tree each accommodate a standard binary SVM classifier. The holistic triple classifier is a regression model trained on three classes, whose training algorithm is originated from a recently proposed implementation technique, namely the least-squares support vector machine (LS-SVM). A major novelty with the holistic triple classifier is the reduced number of support vectors in the solution. For the resultant HTL-SVM, an upper bound of the generalisation error can be obtained. The time complexity of training the HTL-SVM is analysed, and is shown to be comparable to that of training the one-versus-one (1-vs.-1) SVM, particularly on small-scale datasets. Empirical studies show that the proposed HTL-SVM achieves competitive classification accuracy with a reduced number of support vectors compared to the popular 1-vs-1 alternative.     

二次损失函数支持向量机性能的研究

通过比较二次损失函数支持向量机和标准支持向量机在模式识别问题上的表现，分析了二次损失函数支持向量机的性能．实验表明这两种支持向量机对平衡数据有相似的分类能力，但二次损失函数支持向量机的优化参数更小，支持向量更多；对不平衡数据，二次损失函数支持向量机的分类准确率随不平衡度的增加而急剧下降．研究同时表明基于RM界的梯度方法对某些数据无效．文中定性分析了导致上述各种现象的原因．最后提出了一种利用黄金分割原理缩减二次损失函数支持向量机支持向量的方法，该方法冗余的支持向量数不超过一个．     

边界K邻近大样本支持向量机分类

针对大样本支持向量机内存开销大、训练速度慢的缺点,提出了一种改进的支持向量机算法。算法先利用KNN方法找出可能支持向量,然后利用SVM在可能支持向量集上训练得到分类器。实验表明改进算法训练速度提高明显。     

A correlation-based ant miner for classification rule discovery

In recent years, a few sequential covering algorithms for classification rule discovery based on the ant colony optimization meta-heuristic (ACO) have been proposed. This paper proposes a new ACO-based classification algorithm called AntMiner-C. Its main feature is a heuristic function based on the correlation among the attributes. Other highlights include the manner in which class labels are assigned to the rules prior to their discovery, a strategy for dynamically stopping the addition of terms in a rule’s antecedent part, and a strategy for pruning redundant rules from the rule set. We study the performance of our proposed approach for twelve commonly used data sets and compare it with the original AntMiner algorithm, decision tree builder C4.5, Ripper, logistic regression technique, and a SVM. Experimental results show that the accuracy rate obtained by AntMiner-C is better than that of the compared algorithms. However, the average number of rules and average terms per rule are higher.     

基于半监督学习和支持向量机的煤与瓦斯突出预测研究

针对支持向量机要求输入向量为已标记样本,而实际应用中已标记样本很难获取的问题,提出将半监督学习和支持向量机结合的煤与瓦斯突出预测方法;介绍了采用SVM预测煤与瓦斯突出的流程及其输入向量的选择;对半监督学习中的协同训练算法进行了改进:在同一属性集上训练2个不同分类器SVM和KNN,将2个分类器标记一致的样本加入训练集,从而充分利用未标记样本不断补充信息,更新训练集标记样本,达到强化训练集的目的。测试结果表明,改进后的算法比单独的支持向量机预测方法准确率更高。     

改进的结合密度聚类的SVM快速分类方法

针对SVM在对大规模数据分类时求解规模过大的问题,提出了一种缩减数据集以提高训练速度的方法。该算法的第一步利用基于密度的方法大致定位能代表某个局域的质点,然后用SVM训练缩减后的数据得到一组支持向量,第二步的训练数据由支持向量以及其所代表的样本点构成。仿真实验证明该算法在保证分类准确率的情况下能有效地提高分类速度。