A fast and accurate online self-organizing scheme for parsimonious fuzzy neural networks

In this paper, we present a fast and accurate online self-organizing scheme for parsimonious fuzzy neural networks (FAOS-PFNN), where a novel structure learning algorithm incorporating a pruning strategy into new growth criteria is developed. The proposed growing procedure without pruning not only speeds up the online learning process but also facilitates a more parsimonious fuzzy neural network while achieving comparable performance and accuracy by virtue of the growing and pruning strategy. The FAOS-PFNN starts with no hidden neurons and parsimoniously generates new hidden units according to the proposed growth criteria as learning proceeds. In the parameter learning phase, all the free parameters of hidden units, regardless of whether they are newly created or originally existing, are updated by the extended Kalman filter (EKF) method. The effectiveness and superiority of the FAOS-PFNN paradigm is compared with other popular approaches like resource allocation network (RAN), RAN via the extended Kalman filter (RANEKF), minimal resource allocation network (MRAN), adaptive-network-based fuzzy inference system (ANFIS), orthogonal least squares (OLS), RBF-AFS, dynamic fuzzy neural networks (DFNN), generalized DFNN (GDFNN), generalized GAP-RBF (GGAP-RBF), online sequential extreme learning machine (OS-ELM) and self-organizing fuzzy neural network (SOFNN) on various benchmark problems in the areas of function approximation, nonlinear dynamic system identification, chaotic time-series prediction and real-world regression problems. Simulation results demonstrate that the proposed FAOS-PFNN algorithm can achieve faster learning speed and more compact network structure with comparably high accuracy of approximation and generalization.     

A fast and accurate face detector based on neural networks

Detecting faces in images with complex backgrounds is a difficult task. Our approach, which obtains state of the art results, is based on a neural network model: the constrained generative model (CGM). Generative, since the goal of the learning process is to evaluate the probability that the model has generated the input data, and constrained since some counter-examples are used to increase the quality of the estimation performed by the model. To detect side view faces and to decrease the number of false alarms, a conditional mixture of networks is used. To decrease the computational time cost, a fast search algorithm is proposed. The level of performance reached, in terms of detection accuracy and processing time, allows us to apply this detector to a real world application: the indexing of images and videos     

Convolutional neural network based heterogeneous transfer learning for remote-sensing scene classification

ABSTRACT

Deep convolutional neural network (CNN) transfer has recently shown strong performance in scene classification of high-resolution remote-sensing images. However, the majority of transfer learning solutions are categorized as homogeneous transfer learning, which ignores differences between target and source domains. In this paper, we propose a heterogeneous model to transfer CNNs to remote-sensing scene classification to correct input feature differences between target and source datasets. First, we extract filters from source images using the principal component analysis (PCA) method. Next, we convolute the target images with the extracted PCA filters to obtain an adopted target dataset. Then, a pretrained CNN is transferred to the adopted target dataset as a feature extractor. Finally, a classifier is used to accomplish remote-sensing scene classification. We conducted extensive experiments on the UC Merced dataset, the Brazilian coffee scene dataset and the Aerial Images Dataset to verify the effectiveness of the proposed heterogeneous model. The experimental results show that the proposed heterogeneous model outperforms the homogeneous model that uses pretrained CNNs as feature extractors by a wide margin and gains similar accuracies by fine-tuning a homogeneous transfer learning model with few training iterations.     

A fast, three-layer neural network for path finding

A path-planning algorithm is proposed to find a path based on local rules applied to a three-layer artificial neural network. Each layer consists of two-dimensionally arranged neurons with recurrent connections within a limited neighbourhood. The output of one layer determines the weights of the connections in the next layer. In principle, the method is based on a diffusion process, but is modified such that it does not suffer from several drawbacks involved in this algorithm. By application of a nonlinear transformation in layer 2, the diffusion front has the qualitative properties of a propagation wave. Therefore, limited resolution of the units is not critical, in contrast to classical diffusion algorithms. Furthermore, the algorithm generally does not suffer from the superposition of diffusion gradients when several paths are possible. The diffusion takes place in a space covered with 'obstacle potentials' which decrease the velocity of the diffusion front. In this way the path can maintain an adjustable safety margin in relation to the obstacles, for example, to cope with problems of incomplete knowledge of the obstacle's position. The algorithm thus combines the advantages of the diffusion algorithm, namely avoidance of local minima, of wave propagation, i.e. coping with limited resolution, and the potential field approach, i.e. maintaining a safety margin in relation to obstacles. The distributed architecture also allows for 'spatial interpolation' between the units (coarse coding), thereby providing smooth path forms. A comparison with paths developed by human subjects shows some similarity on the qualitative level, but there are also obvious differences.     

基于深度卷积神经网络的道路场景理解

在无人驾驶技术中,道路场景的理解是一个非常重要的环境感知任务,也是一个很具有挑战性的课题。提出了一个深层的道路场景分割网络（Road Scene Segmentation Network,RSSNet）,该网络为32层的全卷积神经网络,由卷积编码网络和反卷积解码网络组成。网络中采用批正则化层防止了深度网络在训练中容易出现的“梯度消失”问题;在激活层中采用了Maxout激活函数,进一步缓解了梯度消失,避免网络陷入饱和模式以及出现神经元死亡现象;同时在网络中适当使用Dropout操作,防止了模型出现过拟合现象;编码网络存储了特征图的最大池化索引并在解码网络中使用它们,保留了重要的边缘信息。实验证明,该网络能够大大提高训练效率和分割精度,有效识别道路场景图像中各像素的类别并对目标进行平滑分割,为无人驾驶汽车提供有价值的道路环境信息。     

Adaptive fuzzy color segmentation with neural network for road detections

In this paper a new color space, called the RGB color ratio space, is proposed and defined according to a reference color such that an image can be transformed from a conventional color space to the RGB color ratio space. Because a color in the RGB color ratio space is represented as three color ratios and intensity, the chrominance can be completely reserved (three color ratios) and the luminance can be de-correlated with the chrominance. Different from traditional distance measurement, a road color model is determined by an ellipse area in the RGB ratio space enclosed by the estimated boundaries. A proposed adaptive fuzzy logic in which fuzzy membership functions are defined according to estimated boundaries is introduced to implement clustering rules. Therefore, each pixel will have its own fuzzy membership function corresponding to its intensity. A basic neural network is trained and used to achieve parameters optimization. The low computation cost of the proposed segmentation method shows the feasibility for real time application. Experimental results for road detection demonstrate the robustness to intensity variation of the proposed approach.     

LRCN-RetailNet: A recurrent neural network architecture for accurate people counting

Multimedia Tools and Applications - Measuring and analyzing the flow of customers in retail stores is essential for a retailer to better comprehend customers’ behavior and support...     

Pooling spike neural network for fast rendering in global illumination

Neural Computing and Applications - The generation of photo-realistic images is a major topic in computer graphics. By using the principles of physical light propagation, images that are...     

Boosting a fast neural network for supervised land cover classification

It is demonstrated that the use of an ensemble of neural networks for routine land cover classification of multispectral satellite data can lead to a significant improvement in classification accuracy. Specifically, the AdaBoost.M1 algorithm is applied to a sequence of three-layer, feed-forward neural networks. In order to overcome the drawback of long training time for each network in the ensemble, the networks are trained with an efficient Kalman filter algorithm. On the basis of statistical hypothesis tests, classification performance on multispectral imagery is compared with that of maximum likelihood and support vector machine classifiers. Good generalization accuracies are obtained with computation times of the order of 1 h or less. The algorithms involved are described in detail and a software implementation in the ENVI/IDL image analysis environment is provided.     

Indoor scene segmentation algorithm based on full convolutional neural network

Neural Computing and Applications - With the leaps and bounds of computer performance and the advent of the era of big data, deep learning has drawn more and more attention from all walks of life....     

An optimized discrete neural network in embedded systems for road recognition

A type of optimized neural networks with limited precision weights (LPWNN) is presented in this paper. Such neural networks, which require less memory for storing the weights and less expensive floating point units in order to perform the computations involved, are better suited for embedded systems implementation than the real weight ones. Based on analyzing the learning capability of LPWNN, Quantize Back-propagation Step-by-Step (QBPSS) algorithm is proposed for such neural networks to overcome the effects of limited precision. Methods of designing and training LPNN are represented, including the quantization of non-linear activation function and the selection of learning rate, network architecture and weights precision. The optimized LPWNN performance has been evaluated by comparing to conventional neural networks with double-precision floating-point weights on road recognition of image for intelligent vehicle in ARM 9 embedded systems, and the results show the optimized LPWNN has 7 times faster than the conventional ones.     

全卷积网络结合改进的条件随机场循环神经网络用于SAR图像场景分类

传统合成孔径雷达（SAR）图像基于粗分割像素块提取相关特征,后接支持向量机（SVM）和马尔可夫随机场（MRF）或条件随机场（CRF）进行分类,该方法存在同一像素块内部不同类别像素的误差,而且只考虑邻近区域未充分用到全局信息和结构信息。故考虑基于像素点引入全卷积网络（FCN）,以ESAR卫星图像为样本,基于像素点级别构建卷积网络进行训练,得到各像素的初始类别分类概率。为了考虑全局像素类别的影响后接CRF-循环神经网络（CRF-RNN）,利用FCN得到的初始概率,结合CRF结构得到全局像素类别转移结果,之后进行RNN的迭代进一步优化实验结果。由于基于像素点和考虑了全局信息与结构信息,克服了传统分类的部分缺点,使正确率较传统SVM或CRF方法平均提高了约6.5个百分点。由于CRF-RNN的距离权重是用高斯核人为拟合的,不能随实际训练样本来改变和确定,故存在一定误差,针对该问题提出可训练的全图距离权重卷积网络来改进CRF-RNN,最终实验结果表明改进后方法的正确率较未改进的CRF-RNN又提高了1.04个百分点。     

Approximating support vector machine with artificial neural network for fast prediction

Support vector machine (SVM) is a powerful algorithm for classification and regression problems and is widely applied to real-world applications. However, its high computational load in the test phase makes it difficult to use in practice. In this paper, we propose hybrid neural network (HNN), a method to accelerate an SVM in the test phase by approximating the SVM. The proposed method approximates the SVM using an artificial neural network (ANN). The resulting regression function of the ANN replaces the decision function or the regression function of the SVM. Since the prediction of the ANN requires significantly less computation than that of the SVM, the proposed method yields faster test speed. The proposed method is evaluated by experiments on real-world benchmark datasets. Experimental results show that the proposed method successfully accelerates SVM in the test phase with little or no prediction loss.     

基于模糊神经网络的大场景人群密度估计方法 *

提出了一种估计大场景下密集人群密度的方法。该方法根据人类视觉的模糊性原理,认为用模糊集来划分人群密度范围比用确定的方法更符合人眼视知觉的认知方式,利用统计的方法确定灰度共生矩阵各指标对于各个密度类别的隶属函数;设计基于误差反向传播训练算法（BP）的模糊神经网络,计算样本模式对于各个密度类别的隶属度,并根据人群密度变化的时域连续性原理对人群密度范围进行合理估计。实验表明该方法提高了估计精度。     

Cost estimation in road construction using artificial neural network

Neural Computing and Applications - Road construction projects on the territory of the Republic of Croatia are characterized by the overrun of planned costs. The experience of the contractor on...     

Complex scene video frames alignment and multi-frame fusion deraining with deep neural network

Neural Computing and Applications - Rainy is one of the most common weather. It is difficult to get clear and accurate background information when shooting outdoors in the rain. At the same time,...     

Coherency-based fast voltage contingency ranking employing counterpropagation neural network

Power system security is one of the major concerns in competitive electricity markets driven by trade demands and regulations. If the system is found to be insecure, timely corrective measures need to be taken to prevent system collapse. This paper presents an approach based on a counterpropagation neural network (CPNN) to identify and rank the contingencies expected to reduce or eliminate the steady-state loadability margin of the system, making it prone to voltage collapse. It has been shown that unlike other artificial neural networks (ANN) paradigms, which start with random weights, CPNN is very sensitive to initial weights. To reduce the dimension and training time, a novel feature selection method, based on the coherency existing between load buses with respect to voltage dynamics, is employed to select significant input features for the CPNN. Once trained, the CPNN is found to rank voltage contingencies accurately for previously unknown system conditions very fast. Due to its fast training, the proposed CPNN will be particularly useful for power system planning studies, as a number of combinations can be tried within a small time frame. The effectiveness of the proposed approach has been demonstrated on IEEE 30-bus test system and a 75-bus practical Indian system.     

特征重校准融合密集神经网络的遥感场景分类

针对遥感影像场景中空间信息丰富以及冗余的地理特征会对网络训练时造成干扰等问题,提出一种采用特征重校准融合密集神经网络的遥感影像场景分类方法。通过缩聚与激发机制建立SE block,将SE block与其多尺度分支嵌入DenseNet-121中进行特征重校准,利用DenseNet中密集连接方式加强信息流的传递。该方法使得整体模型获得全局感受野的稳健特征表示,减少遥感场景特征的冗余映射。通过在两个公开遥感影像数据集UCMerced_LandUse和SIRI-WHU中进行实验,分类精度分别高达97.7%和98.9%,验证了该方法的有效性。     

Automatic scene generation using sentiment analysis and bidirectional recurrent neural network with multi-head attention

Neural Computing and Applications - Text generation is one of the complex tasks associated with natural language processing. For efficient text generation, syntax and semantics of the language have...     

A fast identification algorithm for box-cox transformation based radial basis function neural network

In this letter, a Box-Cox transformation-based radial basis function (RBF) neural network is introduced using the RBF neural network to represent the transformed system output. Initially a fixed and moderate sized RBF model base is derived based on a rank revealing orthogonal matrix triangularization (QR decomposition). Then a new fast identification algorithm is introduced using Gauss-Newton algorithm to derive the required Box-Cox transformation, based on a maximum likelihood estimator. The main contribution of this letter is to explore the special structure of the proposed RBF neural network for computational efficiency by utilizing the inverse of matrix block decomposition lemma. Finally, the Box-Cox transformation-based RBF neural network, with good generalization and sparsity, is identified based on the derived optimal Box-Cox transformation and a D-optimality-based orthogonal forward regression algorithm. The proposed algorithm and its efficacy are demonstrated with an illustrative example in comparison with support vector machine regression.