Enhancing and Relaxing Competitive Units for Feature Discovery

In this paper, we propose a new information-theoretic method called enhancement and relaxation to discover main features in input patterns. We have so far shown that competitive learning is a process of mutual information maximization between input patterns and connection weights. However, because mutual information is an average over all input patterns and competitive units, it is not adequate for discovering detailed information on the roles of elements in a network. To extract information on the roles of elements in a networks, we enhance or relax competitive units through the elements. Mutual information should be changed by these processes. The change in information is called enhanced information. The enhanced information can be used to discover features in input patterns, because the information includes detailed information on elements in a network. We applied the method to the symmetry data, the well-known Iris problem and the OECD countries classification. In all cases, we succeeded in extracting the main features in input patterns.     

Mining software repositories for comprehensible software fault prediction models

Software managers are routinely confronted with software projects that contain errors or inconsistencies and exceed budget and time limits. By mining software repositories with comprehensible data mining techniques, predictive models can be induced that offer software managers the insights they need to tackle these quality and budgeting problems in an efficient way. This paper deals with the role that the Ant Colony Optimization (ACO)-based classification technique AntMiner+ can play as a comprehensible data mining technique to predict erroneous software modules. In an empirical comparison on three real-world public datasets, the rule-based models produced by AntMiner+ are shown to achieve a predictive accuracy that is competitive to that of the models induced by several other included classification techniques, such as C4.5, logistic regression and support vector machines. In addition, we will argue that the intuitiveness and comprehensibility of the AntMiner+ models can be considered superior to the latter models.     

基于物联网和SOM算法的信息监控系统设计

在物联网环境下进行信息监控系统设计,实现对网络信息的监控和自适应采集,保障网络安全。针对采用传统的神经网络控制方法进行信息监控的数据挖掘准确性不好的问题,提出一种基于物联网和自组织映射SOM算法的信息监控系统设计方法,首先进行信息监控系统的总体设计和功能模块化分析,然后设计改进的SOM算法,应用在信息监控的数据挖掘和分类识别中,在程序加载模块中进行算法加载,最后在物联网环境下构建嵌入式Linux内核进行信息监控系统的软件设计和开发。系统仿真实验结果表明,采用该信息监控系统进行大型物联网的数据信息监控,对数据的准确挖掘和识别性能较好。     

基于最大联合条件互信息的特征选择

在高维数据如图像数据、基因数据、文本数据等的分析过程中,当样本存在冗余特征时会大大增加问题分析复杂难度,因此在数据分析前从中剔除冗余特征尤为重要。基于互信息（MI）的特征选择方法能够有效地降低数据维数,提高分析结果精度,但是,现有方法在特征选择过程中评判特征是否冗余的标准单一,无法合理排除冗余特征,最终影响分析结果。为此,提出一种基于最大联合条件互信息的特征选择方法（MCJMI）。MCJMI选择特征时考虑整体联合互信息与条件互信息两个因素,两个因素融合增强特征选择约束。在平均预测精度方面,MCJMI与信息增益（IG）、最小冗余度最大相关性（mRMR）特征选择相比提升了6个百分点;与联合互信息（JMI）、最大化联合互信息（JMIM）相比提升了2个百分点;与LW向前搜索方法（SFS-LW）相比提升了1个百分点。在稳定性方面,MCJMI稳定性达到了0.92,优于JMI、JMIM、SFS-LW方法。实验结果表明MCJMI能够有效地提高特征选择的准确率与稳定性。     

Separating visual information into position and direction by SOM

A model is proposed to self-organize a map for the visual recognition of position and direction by a robot moving autonomously in a room. The robot is assumed to have visual sensors. The model is based on Kohonens self-organizing map (SOM), which was proposed as a model of self-organization of the cortex. An ordinary SOM consists of a two-dimensional array of neuron-like feature detector units. In our model, however, units are arranged in a three-dimensional array, and a periodic boundary condition is assumed in one dimension. Also, some new learning rules are added. Our model is shown by a computer simulation to form a map which can extract from the visual input two factors of information separately, i.e., the position and direction of the robot. This is an example of so-called two-factor problems. In our algorithm, the difference in the topology of the information is used to separate two factors of information.This work was presented in part at the 8th International Symposium on Artificial Life and Robotics, Oita, Japan, January 24–26, 2003     

Improving information-theoretic competitive learning by accentuated information maximization

In this paper, we propose a new computational method for information-theoretic competitive learning. We have so far developed information-theoretic methods for competitive learning in which competitive processes can be simulated by maximizing mutual information between input patterns and competitive units. Though the methods have shown good performance, networks have had difficulty in increasing information content, and learning is very slow to attain reasonably high information. To overcome the shortcoming, we introduce the rth power of competitive unit activations used to accentuate actual competitive unit activations. Because of this accentuation, we call the new computational method “accentuated information maximization”. In this method, intermediate values are pushed toward extreme activation values, and we have a high possibility to maximize information content. We applied our method to a vowel–consonant classification problem in which connection weights obtained by our methods were similar to those obtained by standard competitive learning. The second experiment was to discover some features in a dipole problem. In this problem, we showed that as the parameter r increased, less clear representations could be obtained. For the third experiment of economic data analysis, much clearer representations were obtained by our method, compared with those obtained by the standard competitive learning method.     

Nonlinear registration using variational principle for mutual information

We extend the multimodal image registration method described in Alexander and Summers [Fast registration algorithm using a variational principle for mutual information, Proc. SPIE Int. Soc. Opt. Eng. 5032 (2003) 1053–1063] to nonlinear registration. A variational principle maximizing mutual information leads to an Euler–Lagrange (EL) equation for the displacement field, represented here in a basis of cubic B-spline functions. A cost function is constructed from the sum of squares of the residuals of the EL equation at a subset of pixels where the magnitude of the spatial gradient of intensity exceeds a user-chosen threshold. The unknown coefficients in the displacement field representation are evaluated using a Levenberg–Marquardt minimization procedure. The proposed method was successfully applied to several image pairs of the same and different modalities, and an artificially constructed series of images containing nonlinear distortions and noise.     

SOM神经网络在独立学院招生决策中的应用

自组织映射(SOM)是一种竞争型无指导学习的神经网络方法。SOM神经网络已广泛地应用于模式聚类、模式识别、拓扑不变性映射等方面。本文利用SOM对中国31个省份进行聚类分析,建立独立学院招生决策模型。首先,选取各省份的报到率、第一志愿率和人均GDP等作为SOM神经网络的输入模式;然后,用SOM进行聚类;最后,对聚类结果进行分析得出各类的生源地特征和等级。实验结果表明,利用SOM对生源地进行聚类分析是可行的、有效的,可以避开人的主观因素,更迅速客观地得到聚类结果。它为独立学院编制招生计划和招生宣传方案提供了一种新的参考依据,在独立学院招生领域具有较好的应用前景。     

基于SOM的成都水处理进水水质变化模式分析

城市水处理过程中,进水水质经常在发生变化。为分析汶川地震后成都水质变化,使用自组织神经网络对进水数据进行数据挖掘和分析,揭示了进水水质变化的模式分类情况,并对各模式及其异常进行了理论分析和解释,从而为污水处理设施运营和决策提供支持。     

基于SOM的离群数据挖掘集成框架研究*

在分析了当前基于距离的离群数据挖掘算法的基础上,提出了一种基于SOM的离群数据挖掘集成框架,其具有可扩展性、可预测性、交互性、适应性、简明性等特征.实验结果表明,基于SOM的离群数据挖掘是有效的.     

基于SOM神经网络的服务质量预测

服务质量预测在服务计算领域中是一个热点研究问题.在历史QoS数据稀疏的情况下,设计一个满足用户个性化需求的服务质量预测方法成为一项挑战.为解决这一挑战问题,本文提出一种基于SOM神经网络的服务质量预测方法SOMQP.首先,基于历史QoS数据,应用SOM神经网络算法分别对用户和服务进行聚类,得到用户关系矩阵和服务关系矩阵;进而,综合考虑用户信誉和服务关联性,采用一种新的Top-k选择机制获得相似用户和相似服务;最后,采用基于用户的和基于项目的混合策略对缺失QoS值进行预测.在真实的数据集WS-Dream上进行大量实验,结果表明,与经典的CF算法和K-means算法相比,本文方法较大程度上提高了QoS预测精度.     

结合Mercer核与SOM的动态免疫网络聚类算法

提出一种用于聚类分析的进化免疫网络算法,借鉴自组织映射原理改进网络拓扑进化机制,利用改进的免疫机制控制抗体数量,提高抗原聚类效果.当输入样本分布呈高度非线性时,使用核方法提高聚类质量,为了避免在特征空间中聚类时失去对原输入空间聚类中心及结果的直观刻画,使用核代入为原输入空间导出一类不同于欧氏距离的新的距离度量,训练过程仍在原空间中进行.实验结果表明了算法的可行性和有效性.     

基于SOM的散乱点云法矢计算

点云法矢计算对点云分布密度较敏感,而且在尖锐边界处计算误差较大。为此,提出一种基于自组织神经网络(SOM)的散乱点云法矢计算方法。为利用散乱点云拓扑和几何信息计算法矢,以球面SOM学习点云拓扑结构,得到被测曲面的三角网格近似图,使用三角网格构成的连通图组织点云数据结构,通过k-近邻点拟合微切平面,从而计算点云法矢,并调整点云法矢指向。实验结果表明,该方法具有较高的计算精度,法矢误差在0.08以内,标准差为0.009。     

Feature selection with dynamic mutual information

Feature selection plays an important role in data mining and pattern recognition, especially for large scale data. During past years, various metrics have been proposed to measure the relevance between different features. Since mutual information is nonlinear and can effectively represent the dependencies of features, it is one of widely used measurements in feature selection. Just owing to these, many promising feature selection algorithms based on mutual information with different parameters have been developed. In this paper, at first a general criterion function about mutual information in feature selector is introduced, which can bring most information measurements in previous algorithms together. In traditional selectors, mutual information is estimated on the whole sampling space. This, however, cannot exactly represent the relevance among features. To cope with this problem, the second purpose of this paper is to propose a new feature selection algorithm based on dynamic mutual information, which is only estimated on unlabeled instances. To verify the effectiveness of our method, several experiments are carried out on sixteen UCI datasets using four typical classifiers. The experimental results indicate that our algorithm achieved better results than other methods in most cases.     

An information theoretic sparse kernel algorithm for online learning

Kernel-based algorithms have been proven successful in many nonlinear modeling applications. However, the computational complexity of classical kernel-based methods grows superlinearly with the increasing number of training data, which is too expensive for online applications. In order to solve this problem, the paper presents an information theoretic method to train a sparse version of kernel learning algorithm. A concept named instantaneous mutual information is investigated to measure the system reliability of the estimated output. This measure is used as a criterion to determine the novelty of the training sample and informative ones are selected to form a compact dictionary to represent the whole data. Furthermore, we propose a robust learning scheme for the training of the kernel learning algorithm with an adaptive learning rate. This ensures the convergence of the learning algorithm and makes it converge to the steady state faster. We illustrate the performance of our proposed algorithm and compare it with some recent kernel algorithms by several experiments.     

Fault detection and identification with a new feature selection based on mutual information

This paper presents a fault diagnosis procedure based on discriminant analysis and mutual information. In order to obtain good classification performances, a selection of important features is done with a new developed algorithm based on the mutual information between variables. The application of the new fault diagnosis procedure on a benchmark problem, the Tennessee Eastman Process, shows better results than other well known published methods.     

A linear discriminant analysis method based on mutual information maximization

We present a new linear discriminant analysis method based on information theory, where the mutual information between linearly transformed input data and the class labels is maximized. First, we introduce a kernel-based estimate of mutual information with a variable kernel size. Furthermore, we devise a learning algorithm that maximizes the mutual information w.r.t. the linear transformation. Two experiments are conducted: the first one uses a toy problem to visualize and compare the transformation vectors in the original input space; the second one evaluates the performance of the method for classification by employing cross-validation tests on four datasets from the UCI repository. Various classifiers are investigated. Our results show that this method can significantly boost class separability over conventional methods, especially for nonlinear classification.     

Evolutionary feature subsets selection based on interaction information for high dimensional imbalanced data classification

With the advent of technology in various scientific fields, high dimensional data are becoming abundant. A general approach to tackle the resulting challenges is to reduce data dimensionality through feature selection. Traditional feature selection approaches concentrate on selecting relevant features and ignoring irrelevant or redundant ones. However, most of these approaches neglect feature interactions. On the other hand, some datasets have imbalanced classes, which may result in biases towards the majority class. The main goal of this paper is to propose a novel feature selection method based on the interaction information (II) to provide higher level interaction analysis and improve the search procedure in the feature space. In this regard, an evolutionary feature subset selection algorithm based on interaction information is proposed, which consists of three stages. At the first stage, candidate features and candidate feature pairs are identified using traditional feature weighting approaches such as symmetric uncertainty (SU) and bivariate interaction information. In the second phase, candidate feature subsets are formed and evaluated using multivariate interaction information. Finally, the best candidate feature subsets are selected using dominant/dominated relationships. The proposed algorithm is compared with some other feature selection algorithms including mRMR, WJMI, IWFS, IGFS, DCSF, IWFS, K_OFSD, WFLNS, Information Gain and ReliefF in terms of the number of selected features, classification accuracy, F-measure and algorithm stability using three different classifiers, namely KNN, NB, and CART. The results justify the improvement of classification accuracy and the robustness of the proposed method in comparison with the other approaches.     

基于LVQ算法的SOM神经网络在入侵检测系统中的应用

目前,入侵检测技术（IDS）已成为网络安全领域研究的焦点,神经网络被应用到这项技术的研究上．文章在建立一、类基于SOM神经网络的分类器的基础上,应用了LVQ算法对SOM进行二次监督学习训练,极大提高了分类器的检测性能。仿真试验结果证明了该检测模型的有效性。     

传感器网络中结合SOM与免疫克隆选择的分簇路由策略

提出一种结合自组织映射(SOM)与免疫克隆选择算法的分簇路由策略SICR(SOM and Immune Clonal Selection Based Clustering Routing Scheme for Wireless Sensor Networks).在分簇聚类时,充分考虑了网络节点密度、剩余能量以及与汇聚点间距离等因素,采用一种基于自组织映射原理的簇头竞争算法,构建了能量消耗均衡的分簇结构.该结构可以有效的应用于节点能量异构的网络场景.同时,为了减少簇重构次数,降低重构开销,在维护阶段引入了自适应调整机制,簇首可根据簇内各成员的剩余能量估算簇的稳定性,并据此调整簇规模.路由的组织则分为簇内通信和簇间通信两部分:簇内通信基于建立的簇内拓扑路径集进行;簇间通信则通过基于克隆选择算法建立的最小汇集树进行.模拟实验表明,与现有的几种算法相比,SICR能更好均衡节点的能量消耗和延长网络寿命.