DESNN Algorithm for Communication Network Intrusion Detection

Intrusion detection is a crucial technology in the communication network security field. In this paper, a dynamic evolutionary sparse neural network (DESNN) is proposed for intrusion detection, named as DESNN algorithm. Firstly, an ensemble neural network model is constructed, which is processed by a dynamic pruning rule and further divided into advantage subnetworks and disadvantage subnetworks. The dynamic pruning rule can effectively reduce the subnetworks weight parameters, thereby increasing the speed of the subnetworks intrusion detection. Then considering the subnetworks performance loss caused by the dynamic pruning rule, a novel evolutionary mechanism is proposed to optimize the training process of the disadvantage subnetworks. The weight of the disadvantage subnetworks approach the weight of the advantage subnetworks by the evolutionary mechanism, such that the performance of the ensemble neural network can be improved. Finally, an optimal subnetwork is selected from the ensemble neural network, which is used to detect multiple types of intrusion. Experiments show that the proposed DESNN algorithm improves intrusion detection speed without causing significant performance loss compare with other fully-connected neural network models.

     

Technique for multi-focus image fusion based on fuzzy-adaptive pulse-coupled neural network

Multi-focus image fusion technique can solve the problem that not all the targets in an image are clear in case of imaging in the same scene. In this paper, a novel multi-focus image fusion technique is presented, which is developed by using the nonsubsampled contourlet transform (NSCT) and a proposed fuzzy logic based adaptive pulse-coupled neural network (PCNN) model. In our method, sum-modified Laplacian (SML) is calculated as the motivation for PCNN neurons in NSCT domain. Since the linking strength plays an important role in PCNN, we propose an adaptively fuzzy way to determine it by computing each coefficient’s importance relative to the surrounding coefficients. Combined with human visual perception characteristics, the fuzzy membership value is employed to automatically achieve the degree of importance of each coefficient, which is utilized as the linking strength in PCNN model. Experimental results on simulated and real multi-focus images show that the proposed technique has a superior performance to series of exist fusion methods.     

Model-based neural decoding of reaching movements: a maximum likelihood approach

A new paradigm for decoding reaching movements from the signals of an ensemble of individual neurons is presented. This new method not only provides a novel theoretical basis for the task, but also results in a significant decrease in the error of reconstructed hand trajectories. By using a model of movement as a foundation for the decoding system, we show that the number of neurons required for reconstruction of the trajectories of point-to-point reaching movements in two dimensions can be halved. Additionally, using the presented framework, other forms of neural information, specifically neural "plan" activity, can be integrated into the trajectory decoding process. The decoding paradigm presented is tested in simulation using a database of experimentally gathered center-out reaches and corresponding neural data generated from synthetic models.     

Ensembles of Neural Networks for Fault Diagnosis in Analog Circuits

A new neural network-based analog fault diagnosis strategy is introduced. Ensemble of neural networks is constructed and trained for efficient and accurate fault classification of the circuit under test (CUT). In the testing phase, the outputs of the individual ensemble members are combined to isolate the actual CUT fault. Prominent techniques for producing the ensemble are utilized, analyzed and compared. The created ensemble exhibit high classification accuracy even if the CUT has overlapping fault classes which cannot be isolated using a unitary neural network. Each neural classifier of the ensemble focuses on a particular region in the CUT measurement space. As a result, significantly better generalization performance is achieved by the ensemble as compared to any of its individual neural nets. Moreover, the selection of the proper architecture of the neural classifiers is simplified. Experimental results demonstrate the superior performance of the developed approach.     

Classification with ensembles and case study on functional magnetic resonance imaging

The ensemble is a technique that strategically combines basic models to achieve better accuracy rates. Diversity, combination methods, and selection topology are the main factors determining ensemble performance. Consequently, it is a challenging task to design an efficient ensemble scheme. Even though numerous paradigms have been proposed to classify ensemble schemes, there is still much room for improvement. This paper proposes a general framework for creating ensembles in the context of classification. Specifically, the ensemble framework consists of four stages: objectives, data preparing, model training, and model testing. It is comprehensive to design diverse ensembles. The proposed ensemble approach can be used for a wide variety of machine learning tasks. We validate our approach on real-world datasets. The experimental results show the efficiency of the proposed approach.     

A study on several machine-learning methods for classification of malignant and benign clustered microcalcifications

In this paper, we investigate several state-of-the-art machine-learning methods for automated classification of clustered microcalcifications (MCs). The classifier is part of a computer-aided diagnosis (CADx) scheme that is aimed to assisting radiologists in making more accurate diagnoses of breast cancer on mammograms. The methods we considered were: support vector machine (SVM), kernel Fisher discriminant (KFD), relevance vector machine (RVM), and committee machines (ensemble averaging and AdaBoost), of which most have been developed recently in statistical learning theory. We formulated differentiation of malignant from benign MCs as a supervised learning problem, and applied these learning methods to develop the classification algorithm. As input, these methods used image features automatically extracted from clustered MCs. We tested these methods using a database of 697 clinical mammograms from 386 cases, which included a wide spectrum of difficult-to-classify cases. We analyzed the distribution of the cases in this database using the multidimensional scaling technique, which reveals that in the feature space the malignant cases are not trivially separable from the benign ones. We used receiver operating characteristic (ROC) analysis to evaluate and to compare classification performance by the different methods. In addition, we also investigated how to combine information from multiple-view mammograms of the same case so that the best decision can be made by a classifier. In our experiments, the kernel-based methods (i.e., SVM, KFD, and RVM) yielded the best performance (Az = 0.85, SVM), significantly outperforming a well-established, clinically-proven CADx approach that is based on neural network (Az = 0.80).     

利用奇异谱分析的深度神经网络语音增强方法

针对现有深度神经网络语音增强方法对带噪语音的去噪能力有限、语音质量提升不高的问题，提出了一种基于奇异谱分析的深度神经网络语音增强方法。通过引入奇异谱分析算法对带噪语音进行预处理，以初步分离得到语音信号与噪声。接着将语音信号与噪声用于深度神经网络模型得训练，以得到性能更优的网络模型，从而使得本文方法具有更好的性能。最后在重建干净语音的环节中，同时使用神经网络估计得到的对数功率谱和带噪语音的对数功率谱，并加入了权重系数，使得本文提出的方法可以适应不同信噪比的情形，有效的去除背景噪声，降低语音信号的失真。本文通过仿真实验验证了该方法的有效性和鲁棒性。      

Testing of programmable analog neural network chips

Artificial neural network chips can achieve high-speed performance in solving complex computational problems for signal and information processing applications. These chips contain regular circuit units such as synapse matrices that interconnect linear arrays of input and output neurons. The neurons and synapses may be implemented in an analog or digital design style. Although the neural processing has some degree of fault tolerance, a significant percentage of processing defects can result in catastrophic failure of the neural network processors. Systematic testing of these arrays of circuitry is of great importance in order to assure the quality and reliability of VLSI neural network processor chips. The proposed testing method consists of parametric test and behavioral test. Two programmable analog neural chips have been designed and fabricated. The systematic approach used to test the chips is described, and measurement results on parametric test are presented.This research was partially supported by DARPA under Contract MDA 972-90-C-0037 and by National Science Foundation under Grant MIP-8904172.     

Privacy Based Data Publishing Model for Cloud Computing Environment

The evaluation of corporate social responsibility (CSR) performance may enhance companies’ willingness to undertake social responsibilities, so it is very important to improve the quality of CSR performance evaluation. Based on the three factors of economic performance, social performance and environmental performance, this paper proposed an improved analytic hierarchy process-back propagation (AHP-BP) neural network algorithm, and introduced the improved AHP-BP neural network algorithm into CSR performance evaluation model. In the stage of improved AHP, the model included the importance of the knowledge and experience of the experts by expert scoring, and reduced the subjective influence of expert judgment on the results by introducing a personality test scale. In the stage of BP neural network, trained models have been used for CSR performance evaluation. The results showed that the prediction result of improved AHP-BP neural network model was better than that of BP neural network model. Therefore, the improved AHP-BP neural network algorithm can be used as a good predictor for CSR performance evaluation.

     

基于混合多样性生成与修剪的集成单类分类算法

针对传统集成学习方法直接应用于单类分类器效果不理想的问题,该文首先证明了集成学习方法能够提升单类分类器的性能,同时证明了若基分类器集不经选择会导致集成后性能下降;接着指出了经典集成方法直接应用于单类分类器集成时存在基分类器多样性严重不足的问题,并提出了一种能够提高多样性的基单类分类器混合生成策略;最后从集成损失构成的角度拆分集成单类分类器的损失函数,针对性地构造了集成单类分类器修剪策略并提出一种基于混合多样性生成和修剪的单类分类器集成算法,简称为PHD-EOC。在UCI标准数据集和恶意程序行为检测数据集上的实验结果表明,PHD-EOC算法兼顾多样性与单类分类性能,在各种单类分类器评价指标上均较经典集成学习方法有更好的表现,并降低了决策阶段的时间复杂度。     

基于点积自注意力卷积神经网络的歌声检测

传统的歌声检测过程往往包含了复杂的特征工程,而基于深度神经网络统一框架的算法则可以利用其强大的学习能力学习到特征,从而忽略特征工程。但是,这些学习到的特征通常得不到重要性区分,在网络中所占权重相同。针对这一问题,提出在卷积神经网络中嵌入点积自注意力模块的算法,该算法通过学习得到各个特征的注意力分布,调整注意力权重,使得卷积神经元在“观察”这些特征时能区分轻重,从而提升网络的整体性能。在实验部分,通过在两个公开数据集下测试,并和基准模型进行对比,证明了该算法对提升歌声检测水平切实有效。      

混合量子衍生神经网络模型及算法

为提高人工神经网络的逼近能力,该文从研究隐层神经元的映射机制入手,提出基于量子比特在Bloch球面的绕轴旋转构造神经网络模型的新思想。首先将样本线性变换为量子比特的相位,并使量子比特在Bloch球面上分别绕着3个坐标轴旋转,旋转角度即为网络参数。然后通过投影测量可以得到量子比特的球面坐标,将这些坐标值提交到隐层激励函数,可得隐层神经元的输出。输出层采用普通神经元。基于L-M(Levenberg-Marquardt)算法设计了该模型的学习算法。实验结果表明,该文提出的模型在逼近能力、泛化能力、鲁棒性能方面,均优于采用L-M算法的普通神经网络。     

Gaussian process modeling of EEG for the detection of neonatal seizures

Gaussian process (GP) probabilistic models have attractive advantages over parametric and neural network modeling approaches. They have a small number of tuneable parameters, can be trained on relatively small training sets, and provide a measure of prediction certainty. In this paper, these properties are exploited to develop two methods of highlighting the presence of neonatal seizures from electroencephalograph (EEG) signals. In the first method, the certainty of the GP model prediction is used to indicate the presence of seizures. In the second approach, the hyperparameters of the GP model are used. Tests are carried out with a feature set of ten EEG measures developed from various signal processing techniques. Features are evaluated using a neural network classifier on 51 h of real neonatal EEG. The GP measures, in particular, the prediction certainty approach, produce a high level of performance compared to other modeling methods and methods currently in clinical use for EEG analysis, indicating that they are an important and useful tool for the real-time detection of neonatal seizures.     

Ensemble convolutional neural networks for automatic fusion recognition of multi‐platform radar emitters

Presently, the extraction of hand‐crafted features is still the dominant method in radar emitter recognition. To solve the complicated problems of selection and updation of empirical features, we present a novel automatic feature extraction structure based on deep learning. In particular, a convolutional neural network (CNN) is adopted to extract high‐level abstract representations from the time‐frequency images of emitter signals. Thus, the redundant process of designing discriminative features can be avoided. Furthermore, to address the performance degradation of a single platform, we propose the construction of an ensemble learning‐based architecture for multi‐platform fusion recognition. Experimental results indicate that the proposed algorithms are feasible and effective, and they outperform other typical feature extraction and fusion recognition methods in terms of accuracy. Moreover, the proposed structure could be extended to other prevalent ensemble learning alternatives.     

通用神经网络硬件中神经元基本数学模型的讨论

在介绍了作者实现通用神经网络硬件中应用的通用计算公式的基础上,提出了一种能同时模拟包括RBF与传统BP网络神经元在内的各种神经元通用的新的数学计算模型,并把基于这种通用数学计算模型的神经网络CASSANDRA-Ⅱ型神经计算机结构设计中并予以硬件实现.文中还讨论了它所模拟神经元网络的灵活性.     

Critic-driven ensemble classification

We develop new rules for combining the estimates obtained from each classifier in an ensemble, in order to address problems involving multiple (>2) classes. A variety of techniques have been previously suggested, including averaging probability estimates from each classifier, as well as hard (0-1) voting schemes. In this work, we introduce the notion of a critic associated with each classifier, whose objective is to predict the classifier's errors. Since the critic only tackles a two class problem, its predictions are generally more reliable than those of the classifier and, thus, can be used as the basis for improved combination rules. Several such rules are suggested here. While previous techniques are only effective when the individual classifier error rate is p<0.5, the new approach is successful, as proved under an independence assumption, even when this condition is violated-in particular, so long as p+q<1, with q the critic's error rate. More generally, critic-driven combining is found to achieve significant performance gains over alternative methods on a number of benchmark data sets. We also propose a new analytical tool for modeling ensemble performance, based on dependence between experts. This approach is substantially more accurate than the analysis based on independence that is often used to justify ensemble methods     

S2‐Net: Machine reading comprehension with SRU‐based self‐matching networks

Machine reading comprehension is the task of understanding a given context and finding the correct response in that context. A simple recurrent unit (SRU) is a model that solves the vanishing gradient problem in a recurrent neural network (RNN) using a neural gate, such as a gated recurrent unit (GRU) and long short‐term memory (LSTM); moreover, it removes the previous hidden state from the input gate to improve the speed compared to GRU and LSTM. A self‐matching network, used in R‐Net, can have a similar effect to coreference resolution because the self‐matching network can obtain context information of a similar meaning by calculating the attention weight for its own RNN sequence. In this paper, we construct a dataset for Korean machine reading comprehension and propose an S²‐Net model that adds a self‐matching layer to an encoder RNN using multilayer SRU. The experimental results show that the proposed S²‐Net model has performance of single 68.82% EM and 81.25% F1, and ensemble 70.81% EM, 82.48% F1 in the Korean machine reading comprehension test dataset, and has single 71.30% EM and 80.37% F1 and ensemble 73.29% EM and 81.54% F1 performance in the SQuAD dev dataset.     

Medical diagnosis with C4.5 rule preceded by artificial neural network ensemble

Comprehensibility is very important when machine learning techniques are used in computer-aided medical diagnosis. Since an artificial neural network ensemble is composed of multiple artificial neural networks, its comprehensibility is worse than that of a single artificial neural network. In this paper, C4.5 Rule-PANE, which combines an artificial neural network ensemble with rule induction by regarding the former as a preprocess of the latter, is proposed. At first, an artificial neural network ensemble is trained. Then, a new training data set is generated by feeding the feature vectors of original training instances to the trained ensemble and replacing the expected class labels of original training instances with the class labels output from the ensemble. Additional training data may also be appended by randomly generating feature vectors and combining them with their corresponding class labels output from the ensemble. Finally, a specific rule induction approach, i.e., C4.5 Rule, is used to learn rules from the new training data set. Case studies on diabetes, hepatitis , and breast cancer show that C4.5 Rule-PANE could generate rules with strong generalization ability, which benefits from an artificial neural network ensemble, and strong comprehensibility, which benefits from rule induction.     

Estimation of electronic parameters of neurons using an inverseFourier transform technique

The objective of this paper is to estimate the passive electrotonic parameters of hippocampal granule cells. Accurate estimation of these parameters is important in understanding the information processing of neurons. A shunt cable model, where the somatic and dendritic time constants can be different, is used to describe the potential changes in the soma and along the dendritic tree. For this model, parameter values are estimated by nonlinear least-squares fitting of the model output to the voltage response of the stimulated cell to current pulses. The solutions are obtained in a two-step process: First, the sensitivity functions are derived from the Laplace transform solution of the theoretical model. Second, the time domain solutions are obtained numerically by an inverse FFT. A sensitivity analysis indicates that accurate estimates require the use of a short current pulse injected at the soma and the sampling of the voltage response close to the end of that pulse. This parameter estimation procedure has been tested on hippocampal granule cells. It yields accurate estimation of neural parameters and will be a useful tool for measuring passive properties of neurons.     

基于集成学习的FY-4A/GIIRS红外通道亮温偏差订正研究

资料变分同化方法基于观测误差无偏的假设,故偏差订正是卫星资料质量控制的重要环节之一。开展了基于集成学习的风云四号A 星(Feng-Yun 4A, FY-4A)干涉式大气垂直探测仪(Geostationary Interferometric Infrared Sounder, GIIRS)中波红外通道亮温偏差订正研究。将随机森林、极端梯度提升(eXtreme Gradient Boosting, XGBoost)、Decision Tree和Extra Tree作为集成学习的基础模型。在优化基础模型的超参数后,采用广义误差极小化方法集成基础模型回归结果。基于台风“利奇马”期间的加密晴空视场点资料,对比了集成学习、基础模型和离线法的GIIRS通道亮温偏差订正效果。试验结果表明,本文所采用的订正方法均取得了好的结果。在所有方法中,集成学习的订正效果最佳。在气团预报因子中,地理(经度和纬度)信息对基础模型贡献率较大。本文方法可推广至其他资料的偏差或误差订正。