Application of ELM–Hammerstein model to the identification of solid oxide fuel cells

In this paper, a new method is proposed to identify solid oxide fuel cell using extreme learning machine–Hammerstein model (ELM–Hammerstein). The ELM–Hammerstein model consists of a static ELM neural network followed by a linear dynamic subsystem. First, the structure of ELM–Hammerstein model is determined by Lipschitz quotient criterion from input–output data. Then, a generalized ELM algorithm is proposed to estimate the parameters of ELM–Hammerstein model, including the parameters of linear dynamic part and the output weights of ELM. The proposed method can obtain accurate identification results and its computation is more efficient. Simulation results demonstrate its effectiveness.

     

基于MapReduce的大数据主动学习

针对传统的主动学习算法只能处理中小型数据集的问题,提出一种基于MapReduce的大数据主动学习算法。首先,在有类别标签的初始训练集上,用极限学习机（ELM）算法训练一个分类器,并将其输出用软最大化函数变换为一个后验概率分布。然后,将无类别标签的大数据集划分为l个子集,并部署到l个云计算节点上。在每一个节点,用训练出的分类器并行地计算各个子集中样例的信息熵,并选择信息熵大的前q个样例进行类别标注,将标注类别的l×q个样例添加到有类别标签的训练集中。重复以上步骤直到满足预定义的停止条件。在Artificial、Skin、Statlog和Poker 4个数据集上与基于ELM的主动学习算法进行了比较,结果显示,所提算法在4个数据集上均能完成主动样例选择,而基于ELM的主动学习算法只在规模最小的数据集上能完成主动样例选择。实验结果表明,所提算法优于基于极限学习机的主动学习算法。     

Extreme‐learning‐machine‐based land cover classification

The extreme learning machine (ELM), a single hidden layer neural network based supervised classifier is used for remote sensing classifications. In comparison to the backpropagation neural network, which requires the setting of several user‐defined parameters and may produce local minima, the ELM requires setting of one parameter, and produces a unique solution for a set of randomly assigned weights. Two datasets, one multispectral and another hyperspectral, were used for classification. Accuracies of 89.0% and 91.1% are achieved with this classifier using multispectral and hyperspectral data, respectively. Results suggest that the ELM provides a classification accuracy comparable to a backpropagation neural network with both datasets. The computational cost using the ELM classifier (1.25 s with Enhanced Thematic Mapper (ETM+) and 0.675 s with Digital Airborne Imaging Spectrometer (DAIS) data) is very small in comparison to the backpropagation neural network.     

MR-ELM: a MapReduce-based framework for large-scale ELM training in big data era

In the big data era, extreme learning machine (ELM) can be a good solution for the learning of large sample data as it has high generalization performance and fast training speed. However, the emerging big and distributed data blocks may still challenge the method as they may cause large-scale training which is hard to be finished by a common commodity machine in a limited time. In this paper, we propose a MapReduce-based distributed framework named MR-ELM to enable large-scale ELM training. Under the framework, ELM submodels are trained parallelly with the distributed data blocks on the cluster and then combined as a complete single-hidden layer feedforward neural network. Both classification and regression capabilities of MR-ELM have been theoretically proven, and its generalization performance is shown to be as high as that of the original ELM and some common ELM ensemble methods through many typical benchmarks. Compared with the original ELM and the other parallel ELM algorithms, MR-ELM is a general and scalable ELM training framework for both classification and regression and is suitable for big data learning under the cloud environment where the data are usually distributed instead of being located in one machine.     

基于深度学习的抓取目标姿态检测与定位

机器人对抓取目标进行高准确的姿态检测与定位依然是一个开放性的难题.本文提出了一种基于卷积神经网络对抓取目标快速姿态检测与精确定位的方法.该方法采用Faster R-CNN Inception-V2网络模型,在网络中将抓取目标的姿态角度采用分类标签方式输出,位置坐标采用回归方法,对Cornell公开数据集重新标注并训练端到端模型.模型在实例检测和对象检测测试集上分别取得96.18%和96.32%的准确率,对于每一幅图像的处理时间小于0.06 s.实验结果表明模型能够实时地对图像中单个或多个抓取目标进行快速地姿态检测与定位,准确度高并具有很强的鲁棒性和稳定性.     

集成重复训练极限学习机的数据分类

极限学习机是一种随机化算法,它随机生成单隐含层神经网络输入层连接权和隐含层偏置,用分析的方法确定输出层连接权。给定网络结构,用极限学习机重复训练网络,会得到不同的学习模型。本文提出了一种集成模型对数据进行分类的方法。首先用极限学习机算法重复训练若干个单隐含层前馈神经网络,然后用多数投票法集成训练好的神经网络,最后用集成模型对数据进行分类,并在10个数据集上和极限学习机及集成极限学习机进行了实验比较。实验结果表明,本文提出的方法优于极限学习机和集成极限学习机。     

主动学习的白细胞图像自动分割

提出利用极端学习机算法(ELM)在线构建像素分类模型分割白细胞图像。训练阶段根据白细胞核深染色的特点,先利用一个Mean-shift过程在RGB空间定位白细胞核区;再经核区形态学膨胀,得到一个熵与面积之比最大的区域作为正样本候选区域,而此区域外像素则作为负样本候选区域;通过正负样本像素抽样组成训练集,能在线训练得到一个两分类ELM模型。多次抽样得到的训练集可以产生多个ELM模型。测试阶段利用上述ELM模型集成分类全体像素,可实现白细胞自动分割。与传统图像分割算法相比,本文方法基本无参数调整,可自适应光照和染色条件导致的图像颜色变化,分割效果好。相关实验结果表明算法的有效性。     

Upper integral network with extreme learning mechanism

The upper integral is a type of non-linear integral with respect to non-additive measures, which represents the maximum potential of efficiency for a group of features with interaction. The value of upper integrals can be evaluated through solving a linear programming problem. Considering the upper integral as a classifier, this paper first investigates its implementation and performance. Fusing multiple upper integral classifiers together by using a single layer neural network, this paper considers a upper integral network as a classification system. The learning mechanism of ELM is used to train this single layer neural network. A comparison of performance between a single upper integral classifier and the upper integral network is given on a number of benchmark databases.     

基于半监督协同训练的百科知识库实体对齐

针对传统实体对齐方法中的有监督学习算法依赖大量标注数据,以及特征表示不适用于百科知识库等问题,提出一种基于半监督协同训练的实体对齐方法。将实体对齐建模为一个带约束的二分类问题,充分利用实体名、属性、描述文本及其中的时间、数值等关键信息,组合生成多维特征;将特征划分为2个相对独立的视图,通过2个视图上分类器的协同训练,迭代地从未标注数据中学习同义实体的分布情况。在2个中文百科上的实验结果表明,使用半监督协同训练方法进行实体对齐的F1值达到84.3%,较其他方法效果最优,证明了其有效性和在百科知识库上的实用价值。     

Selection of input features across subjects for classifying crewmember workload using artificial neural networks

The issue of crewmember workload is important in complex system operation because operator overload leads to decreased mission effectiveness. Psychophysiological research on mental workload uses measures such as electroencephalogram (EEG), cardiac, eye-blink, and respiration measures to identify mental workload levels. This paper reports a research effort whose primary objective was to determine if one parsimonious set of salient psychophysiological features can be identified to accurately classify mental workload levels across multiple test subjects performing a multiple task battery. To accomplish this objective, a stepwise multivariate discriminant analysis heuristic and artificial neural network feature selection with a signal-to-noise ratio (SNR) are used. In general, EEG power in the 31-40-Hz frequency range and ocular input features appeared highly salient. The second objective was to assess the feasibility of a single model to classify mental workload across different subjects. A classification accuracy of 87% was obtained for seven independent validation subjects using neural network models trained with data from other subjects. This result provides initial evidence for the potential use of generalized classification models in multitask workload assessment.     

A fast incremental extreme learning machine algorithm for data streams classification

Data streams classification is an important approach to get useful knowledge from massive and dynamic data. Because of concept drift, traditional data mining techniques cannot be directly applied in data streams environment. Extreme learning machine (ELM) is a single hidden layer feedforward neural network (SLFN), comparing with the traditional neural network (e.g. BP network), ELM has a faster speed, so it is very suitable for real-time data processing. In order to deal with the challenge in data streams classification, a new approach based on extreme learning machine is proposed in this paper. The approach utilizes ELMs as base classifiers and adaptively decides the number of the neurons in hidden layer, in addition, activation functions are also randomly selected from a series of functions to improve the performance of the approach. Finally, the algorithm trains a series of classifiers and the decision results for unlabeled data are made by weighted voting strategy. When the concept in data streams keeps stable, every classifier is incrementally updated by using new data; if concept drift is detected, the classifiers with weak performance will be cleared away. In the experiment, we used 7 artificial data sets and 9 real data sets from UCI repository to evaluate the performance of the proposed approach. The testing results showed, comparing with the conventional classification methods for data streams such as ELM, BP, AUE2 and Learn++.MF, on most data sets, the new approach could not only be simplest in the structure, but also get a higher and more stable accuracy with lower time consuming.     

融合多重实例关系的无监督跨模态哈希检索

大多数跨模态哈希检索方法仅使用余弦相似度进行特征匹配,计算方式过于单一,没有考虑到实例的关系对于性能的影响.为此,提出一种基于多重实例关系图推理的方法,通过构造相似度矩阵,建立全局和局部的实例关系图,充分挖掘实例之间的细粒度关系.在多重实例关系图的基础上进行相似度推理,首先分别进行图像模态和文本模态关系图内部的推理,然后将模态内的关系映射到实例图中进行推理,最后执行实例图内部的推理.此外,为了适应图像和文本两种模态的特点,使用分步训练策略训练神经网络.在MIRFlickr和NUS-WIDE数据集上实验表明,提出的方法在mAP指标上具有很明显的优势,在Top-k-Precision曲线上也获得良好的效果.这也说明所提方法对实例关系进行深入挖掘,从而显著地提升检索性能.     

Adjusting the outputs of a classifier to new a priori probabilities: a simple procedure.

It sometimes happens (for instance in case control studies) that a classifier is trained on a data set that does not reflect the true a priori probabilities of the target classes on real-world data. This may have a negative effect on the classification accuracy obtained on the real-world data set, especially when the classifier's decisions are based on the a posteriori probabilities of class membership. Indeed, in this case, the trained classifier provides estimates of the a posteriori probabilities that are not valid for this real-world data set (they rely on the a priori probabilities of the training set). Applying the classifier as is (without correcting its outputs with respect to these new conditions) on this new data set may thus be suboptimal. In this note, we present a simple iterative procedure for adjusting the outputs of the trained classifier with respect to these new a priori probabilities without having to refit the model, even when these probabilities are not known in advance. As a by-product, estimates of the new a priori probabilities are also obtained. This iterative algorithm is a straightforward instance of the expectation-maximization (EM) algorithm and is shown to maximize the likelihood of the new data. Thereafter, we discuss a statistical test that can be applied to decide if the a priori class probabilities have changed from the training set to the real-world data. The procedure is illustrated on different classification problems involving a multilayer neural network, and comparisons with a standard procedure for a priori probability estimation are provided. Our original method, based on the EM algorithm, is shown to be superior to the standard one for a priori probability estimation. Experimental results also indicate that the classifier with adjusted outputs always performs better than the original one in terms of classification accuracy, when the a priori probability conditions differ from the training set to the real-world data. The gain in classification accuracy can be significant.     

Performance enhancement of extreme learning machine for multi-category sparse data classification problems

This paper presents a performance enhancement scheme for the recently developed extreme learning machine (ELM) for multi-category sparse data classification problems. ELM is a single hidden layer neural network with good generalization capabilities and extremely fast learning capacity. In ELM, the input weights are randomly chosen and the output weights are analytically calculated. The generalization performance of the ELM algorithm for sparse data classification problem depends critically on three free parameters. They are, the number of hidden neurons, the input weights and the bias values which need to be optimally chosen. Selection of these parameters for the best performance of ELM involves a complex optimization problem.In this paper, we present a new, real-coded genetic algorithm approach called ‘RCGA-ELM’ to select the optimal number of hidden neurons, input weights and bias values which results in better performance. Two new genetic operators called ‘network based operator’ and ‘weight based operator’ are proposed to find a compact network with higher generalization performance. We also present an alternate and less computationally intensive approach called ‘sparse-ELM’. Sparse-ELM searches for the best parameters of ELM using K-fold validation. A multi-class human cancer classification problem using micro-array gene expression data (which is sparse), is used for evaluating the performance of the two schemes. Results indicate that the proposed RCGA-ELM and sparse-ELM significantly improve ELM performance for sparse multi-category classification problems.     

基于Plane-Gaussian神经网络的网络流状态监测

针对复杂网络环境下网络流监测（分类）问题,为实现多个类别直接分类以及提高学习方法的训练速度,提出了一种随机的人工神经网络学习方法。该方法借鉴平面高斯（PG）神经网络模型,引入随机投影思想,通过计算矩阵伪逆的方法解析获得网络连接矩阵,理论上可证明该网络具有全局逼近能力。在人工数据和标准网络流监测数据上进行了实验仿真,与同样采用随机方法的极限学习机（ELM）和PG网络相比,分析与实验结果表明：1）由于继承了PG网络的几何特性,对平面型分布数据更为有效;2）采用了随机方法,训练速度与ELM相当,但比PG网络快得多;3）三种方法中,该方法更有利于解决网络流监测问题。     

Effective data mining using neural networks

Classification is one of the data mining problems receiving great attention recently in the database community. The paper presents an approach to discover symbolic classification rules using neural networks. Neural networks have not been thought suited for data mining because how the classifications were made is not explicitly stated as symbolic rules that are suitable for verification or interpretation by humans. With the proposed approach, concise symbolic rules with high accuracy can be extracted from a neural network. The network is first trained to achieve the required accuracy rate. Redundant connections of the network are then removed by a network pruning algorithm. The activation values of the hidden units in the network are analyzed, and classification rules are generated using the result of this analysis. The effectiveness of the proposed approach is clearly demonstrated by the experimental results on a set of standard data mining test problems     

Efficient and effective algorithms for training single-hidden-layer neural networks

Recently there have been renewed interests in single-hidden-layer neural networks (SHLNNs). This is due to its powerful modeling ability as well as the existence of some efficient learning algorithms. A prominent example of such algorithms is extreme learning machine (ELM), which assigns random values to the lower-layer weights. While ELM can be trained efficiently, it requires many more hidden units than is typically needed by the conventional neural networks to achieve matched classification accuracy. The use of a large number of hidden units translates to significantly increased test time, which is more valuable than training time in practice. In this paper, we propose a series of new efficient learning algorithms for SHLNNs. Our algorithms exploit both the structure of SHLNNs and the gradient information over all training epochs, and update the weights in the direction along which the overall square error is reduced the most. Experiments on the MNIST handwritten digit recognition task and the MAGIC gamma telescope dataset show that the algorithms proposed in this paper obtain significantly better classification accuracy than ELM when the same number of hidden units is used. For obtaining the same classification accuracy, our best algorithm requires only 1/16 of the model size and thus approximately 1/16 of test time compared with ELM. This huge advantage is gained at the expense of 5 times or less the training cost incurred by the ELM training.     

Adaptive and optimal control of a non-linear process using intelligent controllers

A time-optimal control for set point changes and an adaptive control for process parameter variations using neural network for a non-linear conical tank level process are proposed in this work. Time-optimal level control was formulated using dynamic programming algorithm and basic properties of the solutions were analysed. It was found that the control is of bang–bang type and there is only one switching. In this method, a mathematical step-by-step procedure is used to obtain the optimal valve position path with one switching and is trained by neural network, based on the back-propagation algorithm. The dynamic programming procedure allows the set point to be reached as fast as possible without overshoot. An adaptive system is also designed and proved to be useful in adjusting the trained parameter of the dynamic programming based neural network for the process parameter variations. A prototype of conical tank level system has been built and implementation of dynamic programming based neural network control algorithm for set point changes and implementation of adaptive control for process parameter variations are performed. Finally, the performance is compared with conventional control. The results prove the effectiveness of the proposed optimal and adaptive control schemes.