A two-level committee machine: a representation and a learning procedure for general piecewise linear discriminant functions

In order to improve the pattern classification power of the committee machine, a two-level committee machine, which is a committee machine with several lower committees, is proposed and the learning algorithm for it is described. The discriminant function realized by the two-level committee machine can be considered as the general piecewise linear discriminant function which includes Chang's definition.⁽¹⁵⁾ The proposed algorithm is a kind of error-correction procedure, and the learning procedure of the usual committee machine and the perceptron are clearly explained as special cases of the proposed algorithm.     

A general method for training the committee machine

A method for training the committee machine with an arbitrary logic is described. First, an expression of the discriminant function realized by the committee machine is introduced. By making use of the expression, an error-correction procedure for training the committee machine is proposed. The procedure of the perceptron is clearly explained as the special case of the proposed procedure. Experimental results show that the procedure is effective.     

Multiple threshold perceptron

As a natural generalization of the linear perceptron with a single threshold element, we define a multiple threshold perceptron which learns a particular type of piecewise linear discriminant set of functions, namely those that are describable by a set of parallel hyperplanes. We also suggest a learning method in the form of an error-correction procedure, and a kind of gradient method that seeks the minimum of a rational criterion function. Experimental results show that the proposed procedure is effective.     

一种自动计算参数的多密度网格聚类算法

针对多密度数据集聚类的时间复杂度过高和聚类结果对参数设置的依赖性过强的问题,提出了一种自动计算参数的多密度网格聚类算法MGCP ,该方法用网格单元的密度和单元间质心距离来构造判别函数,用判别函数的统计信息自动确定参数。实验结果表明,MGCP算法能够有效处理任意形状和不同密度的类,以较小的时间代价获得较高的聚类精度。     

实际噪声下基于时序卷积网络的手机来源识别

针对实际环境噪声下的手机来源识别问题,提出一种基于线性判别分析和时序卷积网络的手机来源识别方法.首先,通过分析不同手机语音特征在实际环境噪声下的分类性能,基于带能量描述符、常数Q变换域和线性判别分析得到一种新的手机语音混合特征.然后,以此混合特征为输入,基于时序卷积网络进行训练和分类.最后,在10个品牌、47种手机型号、32900条语音样本的实际环境噪声语音库上的测试结果显示,所提方法的平均识别准确率达到99.82％.此外,与经典的基于带能量描述符和支持向量机的方法,以及基于常数Q变换域和卷积神经网络的方法相比,平均识别准确率分别提高了0.44和0.54个百分点,平均召回率分别提高了0.45和0.55个百分点,平均精确率分别提高了0.41和0.57个百分点,平均F1分数分别提高了0.49和0.55个百分点.实验结果表明,所提方法具有更优的综合识别性能.     

BP算法的改进及用模拟电路实现的神经网络分类器

基于用模拟电路实现神经网络分类器的目的，对多层静态前馈神经网络的ＢＰ算法做了改进，采用线性限幅函数代替Ｓｉｇｍｏｉｄ函数作为神经元的激活函数，给出了改进的ＢＰ算法。对该算法性能的实验研究表明：这种改进算法不但方便了用线性模拟集成运算放大电路实现神经网络，而且具有学习速度快，映射能力强等优点。根据本文算法设计的神经网络分类器，无论是计算机仿真，还是模拟电路实现，都得到了比较高的识别率。     

A committee machine with empirical formulas for permeability prediction

This study integrates log-derived empirical formulas and the concept of the committee machine to develop an improved model for predicting permeability. A set of three empirical formulas, such as the Wyllie–Rose, Coates–Dumanoir, and porosity models to correlate reservoir well-logging information with measured core permeability, are used as expert members in a committee machine. A committee machine, a new type of neural network, has a parallel architecture that fuses knowledge by combining the individual outputs of its experts to arrive at an overall output. In this study, an ensemble-based committee machine with empirical formulas (CMEF) is used. This machine combines three individual formulas, each of which performs the same evaluation task. The overall output of each ensemble member is then computed according to the coefficients (weights) of the ensemble averaging method that reflects the contribution of each formula. The optimal combination of weights for prediction is also investigated using a genetic algorithm.We illustrate the method using a case study. Eighty-two data sets composed of well log data and core data were clustered into 41 training sets to construct the model and 41 testing sets to validate the model's predictive ability. A comparison of prediction results from the CMEF model and from three individual empirical formulas showed that the proposed CMEF model for permeability prediction provided the best generalization and performance for validation. This indicated that the CMEF model was more accurate than any one of the individual empirical formulas performing alone.     

Incorporating linear discriminant analysis in neural tree for multidimensional splitting

In this paper, a new hybrid classifier is proposed by combining neural network and direct fractional-linear discriminant analysis (DF-LDA). The proposed hybrid classifier, neural tree with linear discriminant analysis called NTLD, adopts a tree structure containing either a simple perceptron or a linear discriminant at each node. The weakly performing perceptron nodes are replaced with DF-LDA in an automatic way. Taking the advantage of this node substitution, the tree building process converges faster and avoids the over-fitting of complex training sets in training process resulting a shallower tree together with better classification performance. The proposed NTLD algorithm is tested on various synthetic and real datasets. The experimental results show that the proposed NTLD leads to very satisfactory results in terms of tree depth reduction as well as classification accuracy.     

常用Fisher判别函数的判别矩阵研究

在线性判别分析(Linear discriminant analysis, LDA)中, 比迹函数、比值函数和迹比函数是三种常用的Fisher判别函数, 每一个判别函数都可得到一个正交判别(Orthogonal discriminant, OD)矩阵和一个不相关判别(Uncorrelated discriminant, UD)矩阵. 本文的主要目的是对这6种判别矩阵的获取方法及其性质进行系统分析, 拟期更清楚地认识它们的联系与区别. 当类内协方差阵非奇异时, 比迹、比值函数的判别矩阵和迹比函数的OD矩阵的获取方法及性质已有研究, 本文对迹比函数的UD矩阵的获取方法及性质进行了补充研究, 得到了迹比函数的UD矩阵与比迹、比值函数的UD矩阵是同一矩阵以及迹比函数的UD矩阵的判别函数值不超过它的OD矩阵的结论. 当类内协方差阵奇异时, 6种判别矩阵的获取方法遇到了困难, 为克服这一困难, 本文首先用极限的思想重新定义了这三种判别函数, 然后采用求极限的方法得到了6种判别矩阵的获取方法. 从所得的获取方法可以看出, 当所需的判别向量均在类内协方差阵的零空间中时, 6个判别矩阵是同一矩阵.     

融合判别式深度特征学习的图像识别算法

目的 卷积神经网络在图像识别算法中得到了广泛应用。针对传统卷积神经网络学习到的特征缺少更有效的鉴别能力而导致图像识别性能不佳等问题,提出一种融合线性判别式思想的损失函数LDloss（linear discriminant loss）并用于图像识别中的深度特征提取,以提高特征的鉴别能力,进而改善图像识别性能。方法 首先利用卷积神经网络搭建特征提取所需的深度网络,然后在考虑样本分类误差最小化的基础上,对于图像多分类问题,引入LDA（linear discriminant analysis）思想构建新的损失函数参与卷积神经网络的训练,来最小化类内特征距离和最大化类间特征距离,以提高特征的鉴别能力,从而进一步提高图像识别性能,分析表明,本文算法可以获得更有助于样本分类的特征。其中,学习过程中采用均值分批迭代更新的策略实现样本均值平稳更新。结果 该算法在MNIST数据集和CK+数据库上分别取得了99.53%和94.73%的平均识别率,与现有算法相比较有一定的提升。同时,与传统的损失函数Softmax loss和Hinge loss对比,采用LDloss的深度网络在MNIST数据集上分别提升了0.2%和0.3%,在CK+数据库上分别提升了9.21%和24.28%。结论 本文提出一种新的融合判别式深度特征学习算法,该算法能有效地提高深度网络的可鉴别能力,从而提高图像识别精度,并且在测试阶段,与Softmax loss相比也不需要额外的计算量。     

基于改进BN模型的网络切片安全部署方法

为了满足5G垂直用户对于网络切片部署时细粒度安全隔离需求,同时兼顾用户的隔离需求和提高资源利用率,提出了一种基于改进BN模型的网络切片安全部署方法。首先提出了一种双层BN模型的网络切片部署架构,基于SBA（service based architecture）设计了虚拟机容器的双层虚拟化架构,将网络切片根据其所属用户的隔离需求分配利益冲突类标签,基于改进的BN模型部署规则确定网络切片的隔离部署策略;然后将该部署方法建立为整数线性规划模型,并将部署成本作为目标函数,通过最小化目标函数实现低成本部署网络切片;最后使用遗传算法对该问题仿真求解。实验结果表明,该安全部署方法在满足网络切片安全隔离需求的前提下降低了部署成本。     

阈交方法在无线网络业务分析中的应用

与有线计算机网络相比,由于无线网络的有限带宽和可变信道,使得无线网络提供业务 服务的QoS保证比有线网络来说更加困难。在提供QoS保证方面一个重要的因素是网络业务预报, 通过对无线网络业务超阈值数据采用方差分析法构造阈值v的偏差函数,为阈值选取提供一定的依 据,通过对无线网络业务到达数据对阈值穿越强度的计算验证了该方法在阈值选取方面的有效性。 可以根据该阈值更好地实现对无线网络业务的预报。     

The C-loss function for pattern classification

This paper presents a new loss function for neural network classification, inspired by the recently proposed similarity measure called Correntropy. We show that this function essentially behaves like the conventional square loss for samples that are well within the decision boundary and have small errors, and L₀ or counting norm for samples that are outliers or are difficult to classify. Depending on the value of the kernel size parameter, the proposed loss function moves smoothly from convex to non-convex and becomes a close approximation to the misclassification loss (ideal 0–1 loss). We show that the discriminant function obtained by optimizing the proposed loss function in the neighborhood of the ideal 0–1 loss function to train a neural network is immune to overfitting, more robust to outliers, and has consistent and better generalization performance as compared to other commonly used loss functions, even after prolonged training. The results also show that it is a close competitor to the SVM. Since the proposed method is compatible with simple gradient based online learning, it is a practical way of improving the performance of neural network classifiers.     

最大散度差和大间距线性投影与支持向量机

首先对Fisher鉴别准则作了必要的修正,并基于新的鉴别准则设计了最大散度差分 类器;然后探讨了当参数C趋向无穷大时,最大散度差分类器的极限情况,得到了大间距线 性投影分类器;最后通过分析说明,大间距线性投影分类器实际上是在模式样本线性可分的条 件下,线性支持向量机的一种特殊情况.在ORL和NUST603人脸库上的测试结果表明,最 大散度差分类器和大间距线性投影分类器可以与线性支持向量机、不相关线性鉴别分析相媲 美,优于Foley-Sammon鉴别分析方法.     

微铣削刀具磨损状态监测方法研究

为提高金属微铣削过程中刀具磨损状态在线监测系统的预测效率与精度,提出一种基于线性判别分析与改进型BP神经网络模型识别刀具磨损的方法;该方法通过传感器与数据采集系统采集微铣削过程振动信号,提取其时域和频域特征并通过线性判别方法进行降维约简;将降维后的特征输入经灰狼优化改进的BP神经网络模型,从而实现微铣刀磨损状态特征的分类;结果表明,提出的微铣刀在线监测方法能够准确识别微铣刀的各种磨损状态;此外,和其它分类算法相比,提出的基于灰狼优化算法的BP神经网络模型在分类精度和计算效率方面具有综合优势;这对实际生产过程中微铣刀的磨损状态监测具有非常重要的实际意义.     

一种LDA与SVM混合的多类分类方法

针对决策有向无环图支持向量机（ＤＤＡＧＳＶＭ）需训练大量支持向量机（ＳＶＭ）和误差积累的问题,提出一种线性判别分析（ＬＤＡ）与ＳＶＭ 混合的多类分类算法．首先根据高维样本在低维空间中投影的特点,给出一种优化ＬＤＡ 分类阈值;然后以优化ＬＤＡ 对每个二类问题的分类误差作为类间线性可分度,对线性可分度较低的问题采用非线性ＳＶＭ 加以解决,并以分类误差作为对应二类问题的可分度;最后将可分度作为混合ＤＤＡＧ 分类器的决策依据．实验表明,与ＤＤＡＧＳＶＭ 相比,所提出算法在确保泛化精度的条件下具有更高的训练和分类速度．     

极速非线性判别分析网络

由于线性判别分析仅是线性方法,难以有效应对非线性问题,而对其非线性化是解决这一问题的关键途径。非线性化判别方法主要包括神经网络和核化方法。神经网络判别分析方法虽然继承了神经网络所具有的自适应、分布存储、并行处理和非线性映射等优点,但也遗传了其训练速度慢且易陷入局部最小值缺点;而核线性判别分析方法虽能获得全局最优解析解,但因受制于隐节点数目（等于样本个数）,当数据规模大时,计算成本变大。本文受随机映射启发,对神经网络判别分析方法进行极速化改造,实现了一种极速非线性判别分析方法,兼具神经网络的自适应性和全局最优解的快速性。最后在UCI真实数据集上的实验表明,极速非线性判别分析方法具有更优的分类性能。     

基于新阈值函数的小波阈值去噪算法

小波基、分解层数、阈值和阈值函数是小波阈值去噪的关键性因素.针对小波基和分解层数的确定,提出了一个算法来实现;对于传统硬、软阈值函数的局限性和阈值函数在临界阈值处不存在平滑过渡区的现象,提出了一个参数化的新阈值函数,该阈值函数具有更高阶,通过灵活调节参数使之介于硬、软阈值函数之间,且兼具硬、软阈值函数的优点,并在临界阈值内添加平滑过渡区,可在阈值处理时保留一部分有用的高频信号,较好地抑制了细节系数的过扼杀和信号振荡现象.仿真结果表明:新阈值函数提高了去噪信号的信噪比,减小了均方误差,取得了较好的去噪效果.     

On the generalization error of fixed combinations of classifiers

We consider the generalization error of concept learning when using a fixed Boolean function of the outputs of a number of different classifiers. Here, we take into account the ‘margins’ of each of the constituent classifiers. A special case is that in which the constituent classifiers are linear threshold functions (or perceptrons) and the fixed Boolean function is the majority function. This corresponds to a ‘committee of perceptrons,’ an artificial neural network (or circuit) consisting of a single layer of perceptrons (or linear threshold units) in which the output of the network is defined to be the majority output of the perceptrons. Recent work of Auer et al. studied the computational properties of such networks (where they were called ‘parallel perceptrons’), proposed an incremental learning algorithm for them, and demonstrated empirically that the learning rule is effective. As a corollary of the results presented here, generalization error bounds are derived for this special case that provide further motivation for the use of this learning rule.     

基于可靠性的5G-低轨星座网络切片映射算法

随着国家天地一体化信息网络重大项目的 推进,5G-低轨星座网络切片的可靠映射成为业内的研究热点.在基于软件定义网络(software-defined network,SDN)和网络功能虚拟化(network function virtualization,NFV)的5G-低轨星座集成网络架构下,将5G-低轨星座网络切片的可靠映射问题建模为一个混合整数线性规划问题.在此基础上,研究了切片请求的资源编排,进而提出了基于广度优先搜索的可靠映射算法.该算法综合考虑切片请求的可靠性阈值及虚拟网络功能(virtual network function,VNF)的资源需求,在虚拟网络映射中根据节点的可靠重要度对节点进行排序.仿真结果表明,该算法在满足可靠性阈值约束的条件下,能够最大化收益开销比,提高虚拟网络映射成功率,在切片可靠性及接受率等方面优于对比算法.