Non-linear variable selection for artificial neural networks using partial mutual information

Artificial neural networks (ANNs) have been widely used to model environmental processes. The ability of ANN models to accurately represent the complex, non-linear behaviour of relatively poorly understood processes makes them highly suited to this task. However, the selection of an appropriate set of input variables during ANN development is important for obtaining high-quality models. This can be a difficult task when considering that many input variable selection (IVS) techniques fail to perform adequately due to an underlying assumption of linearity, or due to redundancy within the available data.This paper focuses on a recently proposed IVS algorithm, based on estimation of partial mutual information (PMI), which can overcome both of these issues and is considered highly suited to the development of ANN models. In particular, this paper addresses the computational efficiency and accuracy of the algorithm via the formulation and evaluation of alternative techniques for determining the significance of PMI values estimated during selection. Furthermore, this paper presents a rigorous assessment of the PMI-based algorithm and clearly demonstrates the superior performance of this non-linear IVS technique in comparison to linear correlation-based techniques.     

基于互信息的贝叶斯网络结构学习算法

结构学习是贝叶斯网络的重要分支之一,而由数据学习贝叶斯网络是NP-完全问题,提出了一个由数据学习贝叶斯网络的改进算法。该算法基于互信息知识构造初始无向图,并通过条件独立测试对无向边添加方向;同时提出了一个针对4节点环和5节点环的局部优化方法来构造初始框架,最后利用贪婪搜索算法得到最优网络结构。数值实验结果表明,改进的算法无论是在BIC评分值,还是在结构的误差上都有一定的改善,并且在迭代次数、运行时间上均有明显降低,能较快地确定出与数据匹配程度最高的网络结构。     

Shot boundary detection based on mutual information and canny edge detector

Shot boundary detection servers as the preliminary step to video retrieval. Most of error detections in the present algorithms are caused by object and camera. Many researchers make use of mutual information to detect shot boundary, and the effect is good, however, object motion reduce performance of algorithm in the methods. In this paper, the author present a novel method of shot boundary detection based on the knowledge of mutual information and canny edge detector. We extract video frame edge using canny edge detector, then distinguish object motion and shot transform effectively by analyzing frame edge differences, reducing error detections caused by object motion, improving recall and precision. Experiments prove this method is robust to object motion, performance is better.     

Statistical voice activity detection based on sparse representation over learned dictionary

In this paper, we present a novel approach to voice activity detection (VAD) based on the sparse representation of an input noisy speech over a learned dictionary. First, we investigate the relationship between the signal detection and the sparse representation based on the Bayesian framework. Second, we derive the decision rule and an adaptive threshold based on a likelihood ratio test, by modeling the non-zero elements in the sparse representation as a Gaussian distribution. The experimental results show that the proposed approach outperforms the current statistical model-based methods, such as Gaussian, Laplacian, and Gamma, under white, babble, and vehicle noise conditions.     

基于互信息可信度的贝叶斯网络入侵检测研究

传统贝叶斯入侵检测算法没有考虑不同属性和属性权值对入侵检测结果的影响,因此分类准确率不够高.针对传统贝叶斯入侵检测算法存在的不足,提出基于互信息可信度的贝叶斯网络入侵检测算法.在综合考虑网络入侵检测数据特点和传统贝叶斯分类算法优点的基础上,用互信息相对可信度进行特征选择,删除一些冗余属性,把互信息相对可信度作为权值引进贝叶斯分类算法中,得到优化的贝叶斯网络入侵检测算法(MI-NB).实验结果表明,MI-NB算法能大大降低分类数据的维数,比传统贝叶斯入侵检测算法及改进算法有更高的分类准确率.     

Application of partial mutual information variable selection to ANN forecasting of water quality in water distribution systems

Recent trends in the management of water supply have increased the need for modelling techniques that can provide reliable, efficient, and accurate representation of the complex, non-linear dynamics of water quality within water distribution systems. Statistical models based on artificial neural networks (ANNs) have been found to be highly suited to this application, and offer distinct advantages over more conventional modelling techniques. However, many practitioners utilise somewhat heuristic or ad hoc methods for input variable selection (IVS) during ANN development.This paper describes the application of a newly proposed non-linear IVS algorithm to the development of ANN models to forecast water quality within two water distribution systems. The intention is to reduce the need for arbitrary judgement and extensive trial-and-error during model development. The algorithm utilises the concept of partial mutual information (PMI) to select inputs based on the analysis of relationship strength between inputs and outputs, and between redundant inputs. In comparison with an existing approach, the ANN models developed using the IVS algorithm are found to provide optimal prediction with significantly greater parsimony. Furthermore, the results obtained from the IVS procedure are useful for developing additional insight into the important relationships that exist between water distribution system variables.     

An active learning based TCM-KNN algorithm for supervised network intrusion detection

As network attacks have increased in number and severity over the past few years, intrusion detection is increasingly becoming a critical component of secure information systems and supervised network intrusion detection has been an active and difficult research topic in the field of intrusion detection for many years. However, it hasn't been widely applied in practice due to some inherent issues. The most important reason is the difficulties in obtaining adequate attack data for the supervised classifiers to model the attack patterns, and the data acquisition task is always time-consuming and greatly relies on the domain experts. In this paper, we propose a novel supervised network intrusion detection method based on TCM-KNN (Transductive Confidence Machines for K-Nearest Neighbors) machine learning algorithm and active learning based training data selection method. It can effectively detect anomalies with high detection rate, low false positives under the circumstance of using much fewer selected data as well as selected features for training in comparison with the traditional supervised intrusion detection methods. A series of experimental results on the well-known KDD Cup 1999 data set demonstrate that the proposed method is more robust and effective than the state-of-the-art intrusion detection methods, as well as can be further optimized as discussed in this paper for real applications.