An investigation and comparison of machine learning approaches for intrusion detection in IoMT network

The Journal of Supercomputing - Internet of Medical Things (IoMT) is network of interconnected medical devices (smart watches, pace makers, prosthetics, glucometer, etc.), software applications,...     

DeepAM: a heterogeneous deep learning framework for intelligent malware detection

With computers and the Internet being essential in everyday life, malware poses serious and evolving threats to their security, making the detection of malware of utmost concern. Accordingly, there have been many researches on intelligent malware detection by applying data mining and machine learning techniques. Though great results have been achieved with these methods, most of them are built on shallow learning architectures. Due to its superior ability in feature learning through multilayer deep architecture, deep learning is starting to be leveraged in industrial and academic research for different applications. In this paper, based on the Windows application programming interface calls extracted from the portable executable files, we study how a deep learning architecture can be designed for intelligent malware detection. We propose a heterogeneous deep learning framework composed of an AutoEncoder stacked up with multilayer restricted Boltzmann machines and a layer of associative memory to detect newly unknown malware. The proposed deep learning model performs as a greedy layer-wise training operation for unsupervised feature learning, followed by supervised parameter fine-tuning. Different from the existing works which only made use of the files with class labels (either malicious or benign) during the training phase, we utilize both labeled and unlabeled file samples to pre-train multiple layers in the heterogeneous deep learning framework from bottom to up for feature learning. A comprehensive experimental study on a real and large file collection from Comodo Cloud Security Center is performed to compare various malware detection approaches. Promising experimental results demonstrate that our proposed deep learning framework can further improve the overall performance in malware detection compared with traditional shallow learning methods, deep learning methods with homogeneous framework, and other existing anti-malware scanners. The proposed heterogeneous deep learning framework can also be readily applied to other malware detection tasks.     

Extreme learning machine for structured output spaces

Recently, extreme learning machine (ELM) has attracted increasing attention due to its successful applications in classification, regression, and ranking. Normally, the desired output of the learning system using these machine learning techniques is a simple scalar output. However, there are many applications in machine learning which require more complex output rather than a simple scalar one. Therefore, structured output is used for such applications where the system is trained to predict structured output instead of simple one. Previously, support vector machine (SVM) has been introduced for structured output learning in various applications. However, from machine learning point of view, ELM is known to offer better generalization performance compared to other learning techniques. In this study, we extend ELM to more generalized framework to handle complex outputs where simple outputs are considered as special cases of it. Besides the good generalization property of ELM, the resulting model will possesses rich internal structure that reflects task-specific relations and constraints. The experimental results show that structured ELM achieves similar (for binary problems) or better (for multi-class problems) generalization performance when compared to ELM. Moreover, as verified by the simulation results, structured ELM has comparable or better precision performance with structured SVM when tested for more complex output such as object localization problem on PASCAL VOC2006. Also, the investigation on parameter selections is presented and discussed for all problems.

     

Extreme learning machine for interval neural networks

Interval data offer a valuable way of representing the available information in complex problems where uncertainty, inaccuracy, or variability must be taken into account. Considered in this paper is the learning of interval neural networks, of which the input and output are vectors with interval components, and the weights are real numbers. The back-propagation (BP) learning algorithm is very slow for interval neural networks, just as for usual real-valued neural networks. Extreme learning machine (ELM) has faster learning speed than the BP algorithm. In this paper, ELM is applied for learning of interval neural networks, resulting in an interval extreme learning machine (IELM). There are two steps in the ELM for usual feedforward neural networks. The first step is to randomly generate the weights connecting the input and the hidden layers, and the second step is to use the Moore–Penrose generalized inversely to determine the weights connecting the hidden and output layers. The first step can be directly applied for interval neural networks. But the second step cannot, due to the involvement of nonlinear constraint conditions for IELM. Instead, we use the same idea as that of the BP algorithm to form a nonlinear optimization problem to determine the weights connecting the hidden and output layers of IELM. Numerical experiments show that IELM is much faster than the usual BP algorithm. And the generalization performance of IELM is much better than that of BP, while the training error of IELM is a little bit worse than that of BP, implying that there might be an over-fitting for BP.     

Extreme learning machine for regression and multiclass classification

Due to the simplicity of their implementations, least square support vector machine (LS-SVM) and proximal support vector machine (PSVM) have been widely used in binary classification applications. The conventional LS-SVM and PSVM cannot be used in regression and multiclass classification applications directly, although variants of LS-SVM and PSVM have been proposed to handle such cases. This paper shows that both LS-SVM and PSVM can be simplified further and a unified learning framework of LS-SVM, PSVM, and other regularization algorithms referred to extreme learning machine (ELM) can be built. ELM works for the "generalized" single-hidden-layer feedforward networks (SLFNs), but the hidden layer (or called feature mapping) in ELM need not be tuned. Such SLFNs include but are not limited to SVM, polynomial network, and the conventional feedforward neural networks. This paper shows the following: 1) ELM provides a unified learning platform with a widespread type of feature mappings and can be applied in regression and multiclass classification applications directly; 2) from the optimization method point of view, ELM has milder optimization constraints compared to LS-SVM and PSVM; 3) in theory, compared to ELM, LS-SVM and PSVM achieve suboptimal solutions and require higher computational complexity; and 4) in theory, ELM can approximate any target continuous function and classify any disjoint regions. As verified by the simulation results, ELM tends to have better scalability and achieve similar (for regression and binary class cases) or much better (for multiclass cases) generalization performance at much faster learning speed (up to thousands times) than traditional SVM and LS-SVM.     

Extreme learning machine and its applications

Recently, a novel learning algorithm for single-hidden-layer feedforward neural networks (SLFNs) named extreme learning machine (ELM) was proposed by Huang et al. The essence of ELM is that the learning parameters of hidden nodes, including input weights and biases, are randomly assigned and need not be tuned while the output weights can be analytically determined by the simple generalized inverse operation. The only parameter needed to be defined is the number of hidden nodes. Compared with other traditional learning algorithms for SLFNs, ELM provides extremely faster learning speed, better generalization performance and with least human intervention. This paper firstly introduces a brief review of ELM, describing the principle and algorithm of ELM. Then, we put emphasis on the improved methods or the typical variants of ELM, especially on incremental ELM, pruning ELM, error-minimized ELM, two-stage ELM, online sequential ELM, evolutionary ELM, voting-based ELM, ordinal ELM, fully complex ELM, and symmetric ELM. Next, the paper summarized the applications of ELM on classification, regression, function approximation, pattern recognition, forecasting and diagnosis, and so on. In the last, the paper discussed several open issues of ELM, which may be worthy of exploring in the future.     

A multi-stage learning framework for intelligent system

As information technologies advance and user-friendly interfaces develop, the interaction between humans and computers, information devices, and new consumer electronics is increasingly gaining attention. One example that most people can relate to is Apple’s innovation in human–computer interaction which has been used on many products such as iPod and iPhone. Siri, the intelligent personal assistant, is a typical application of machine-learning human–computer interaction.Algorithms in machine learning have been employed in many disciplines, including gesture recognition, speaker recognition, and product recommendation systems. While the existing learning algorithms compute and learn from a large quantity of data, this study proposes an improved learning to rank algorithm named MultiStageBoost. In addition to ranking data through multiple stages, the MultiStageBoost algorithm significantly improves the existing algorithms in two ways. Firstly, it classifies and filters data to small quantities and applies the Boosting algorithm to achieve faster ranking performance. Secondly, it enhances the original binary classification by using the reciprocal of fuzzily weighted membership as the ranking distance.The importance of data is revealed in their ranked positions. Usually data ranked in the front are given more attention than those ranked in the middle. For example, after ranking 10,000 pieces of data, the top 10, or at most 100, are the most important and relevant. Whether the data after the top ones are ranked precisely does not really matter. Due to this reason, this study has made improvement on the conventional methods of the pair-wise ranking approach. Not only are data classified and ranked binarily, they are also given different weights depending on whether they are concordant or discordant. Incorporating the concept of weighting into the ranking distance allows us to increase the precision of ranking. Results from experiments demonstrate that our proposed algorithm outperforms the conventional methods in three evaluation measures: P@n, MAP, and NDCG. MultiStageBoost was then applied to speech recognition. However, we do not aim to improve the technology of speech recognition, but simply hope to provide evidences that MultiStageBoost can be used in the classification and ranking in speech recognition. Experiments show that the recognition optimization procedures established by this study are able to increase the recognition rate to over 95% in the personal computing device and industrial personal computer. It is expected that in the future this voice management system will accurately and effectively identify speakers answering the voice response questionnaire and will successfully carry out the functions in the choice of answers, paying the way for the formation of a virtual customer service person.     

Extreme learning machine model for water network management

Neural Computing and Applications - A novel failure rate prediction model is developed by the extreme learning machine (ELM) to provide key information needed for optimum ongoing...     

Extreme learning machine terrain-based navigation for unmanned aerial vehicles

Unmanned aerial vehicles (UAVs) rely on global positioning system (GPS) information to ascertain its position for navigation during mission execution. In the absence of GPS information, the capability of a UAV to carry out its intended mission is hindered. In this paper, we learn alternative means for UAVs to derive real-time positional reference information so as to ensure the continuity of the mission. We present extreme learning machine as a mechanism for learning the stored digital elevation information so as to aid UAVs to navigate through terrain without the need for GPS. The proposed algorithm accommodates the need of the on-line implementation by supporting multi-resolution terrain access, thus capable of generating an immediate path with high accuracy within the allowable time scale. Numerical tests have demonstrated the potential benefits of the approach.     

基于聚类欠采样的极端学习机

针对极端学习机算法对不平衡数据分类问题的处理效果不够理想,提出了一种基于聚类欠采样的极端学习机算法。新算法首先对训练集的负类样本进行聚类生成不同的簇,然后在各簇中按规定的采样率对其进行欠采样,取出的样本组成新的负类数据集,从而使训练集正负类数据个数达到相对平衡,最后训练分类器对测试集进行测试。实验结果表明,新算法有效地降低了数据的不平衡对分类准确率的影响,具有更好的分类性能。     

Extreme learning machine with errors in variables

Extreme learning machine (ELM) is widely used in training single-hidden layer feedforward neural networks (SLFNs) because of its good generalization and fast speed. However, most improved ELMs usually discuss the approximation problem for sample data with output noises, not for sample data with noises both in input and output values, i.e., error-in-variable (EIV) model. In this paper, a novel algorithm, called (regularized) TLS-ELM, is proposed to approximate the EIV model based on ELM and total least squares (TLS) method. The proposed TLS-ELM uses the idea of ELM to choose the hidden weights, and applies TLS method to determine the output weights. Furthermore, the perturbation quantities of hidden output matrix and observed values are given simultaneously. Comparison experiments of our proposed TLS-ELM with least square method, TLS method and ELM show that our proposed TLS-ELM has better accuracy and less training time.     

改进人工蜂群优化的极限学习机

针对极限学习机(ELM)在训练过程中需要大量隐含层节点的问题,提出了差分进化与克隆算法改进人工蜂群优化的极限学习机(DECABC-ELM),在人工蜂群算法的基础上,引入了差分进化算法的差分变异算子和免疫克隆算法的克隆扩增算子,改进了人工蜂群收敛速度慢等缺点,使用改进的人工蜂群算法计算ELM的隐含层节点参数.将算法应用于回归和分类数据集,并与其他算法进行比较,获得了良好的效果.     

Extreme learning machine with autoencoding receptive fields for image classification

Neural Computing and Applications - Based on the theory of local receptive field based extreme learning machine (ELM-LRF) and ELM auto encoder (ELM-AE), a new network...     

Semi-supervised machine learning approach for DDoS detection

Even though advanced Machine Learning (ML) techniques have been adopted for DDoS detection, the attack remains a major threat of the Internet. Most of the existing ML-based DDoS detection approaches are under two categories: supervised and unsupervised. Supervised ML approaches for DDoS detection rely on availability of labeled network traffic datasets. Whereas, unsupervised ML approaches detect attacks by analyzing the incoming network traffic. Both approaches are challenged by large amount of network traffic data, low detection accuracy and high false positive rates. In this paper we present an online sequential semi-supervised ML approach for DDoS detection based on network Entropy estimation, Co-clustering, Information Gain Ratio and Exra-Trees algorithm. The unsupervised part of the approach allows to reduce the irrelevant normal traffic data for DDoS detection which allows to reduce false positive rates and increase accuracy. Whereas, the supervised part allows to reduce the false positive rates of the unsupervised part and to accurately classify the DDoS traffic. Various experiments were performed to evaluate the proposed approach using three public datasets namely NSL-KDD, UNB ISCX 12 and UNSW-NB15. An accuracy of 98.23%, 99.88% and 93.71% is achieved for respectively NSL-KDD, UNB ISCX 12 and UNSW-NB15 datasets, with respectively the false positive rates 0.33%, 0.35% and 0.46%.     

谷歌TensorFlow机器学习框架及应用

TensorFlow是谷歌的第二代开源的人工智能学习系统,是用来实现神经网络的内置框架学习软件库。目前,TensorFlow机器学习已经成为了一个研究热点。由基本的机器学习算法入手,简析机器学习算法与TensorFlow框架,并通过在Linux系统下搭建环境,仿真手写字符识别的TensorFlow模型,实现手写字符的识别,从而实现TensorFlow机器学习框架的学习与应用。     

Extreme learning machine assessment for estimating sediment transport in open channels

The minimum velocity required to prevent sediment deposition in open channels is examined in this study. The parameters affecting transport are first determined and then categorized into different dimensionless groups, including “movement,” “transport,” “sediment,” “transport mode,” and “flow resistance.” Six different models are presented to identify the effect of each of these parameters. The feed-forward neural network (FFNN) is used to predict the densimetric Froude number (Fr) and the extreme learning machine (ELM) algorithm is utilized to train it. The results of this algorithm are compared with back propagation (BP), genetic programming (GP) and existing sediment transport equations. The results indicate that FFNN-ELM produced better results than FNN-BP, GP and existing sediment transport methods in both training (RMSE = 0.26 and MARE = 0.052) and testing (RMSE = 0.121 and MARE = 0.023). Moreover, the performance of FFNN-ELM is examined for different pipe diameters.     

A framework for machine learning based on dynamic physical fields

Despite recent successes and advancements in artificial intelligence and machine learning, this domain remains under continuous challenge and guidance from phenomena and processes observed in natural world. Humans remain unsurpassed in their efficiency of dealing and learning from uncertain information coming in a variety of forms, whereas more and more robust learning and optimisation algorithms have their analytical engine built on the basis of some nature-inspired phenomena. Excellence of neural networks and kernel-based learning methods, an emergence of particle-, swarms-, and social behaviour-based optimisation methods are just few of many facts indicating a trend towards greater exploitation of nature inspired models and systems. This work intends to demonstrate how a simple concept of a physical field can be adopted to build a complete framework for supervised and unsupervised learning methodology. An inspiration for artificial learning has been found in the mechanics of physical fields found on both micro and macro scales. Exploiting the analogies between data and charged particles subjected to gravity, electrostatic and gas particle fields, a family of new algorithms has been developed and applied to classification, clustering and data condensation while properties of the field were further used in a unique visualisation of classification and classifier fusion models. The paper covers extensive pictorial examples and visual interpretations of the presented techniques along with some comparative testing over well-known real and artificial datasets.

MLife: a lite framework for machine learning lifecycle initialization

Machine Learning - Machine learning (ML) lifecycle is a cyclic process to build an efficient ML system. Though a lot of commercial and community (non-commercial) frameworks have been proposed to...