A comparison of standard spell checking algorithms and a novel binary neural approach

In this paper, we propose a simple, flexible, and efficient hybrid spell checking methodology based upon phonetic matching, supervised learning, and associative matching in the AURA neural system. We integrate Hamming Distance and n-gram algorithms that have high recall for typing errors and a phonetic spell-checking algorithm in a single novel architecture. Our approach is suitable for any spell checking application though aimed toward isolated word error correction, particularly spell checking user queries in a search engine. We use a novel scoring scheme to integrate the retrieved words from each spelling approach and calculate an overall score for each matched word. From the overall scores, we can rank the possible matches. We evaluate our approach against several benchmark spellchecking algorithms for recall accuracy. Our proposed hybrid methodology has the highest recall rate of the techniques evaluated. The method has a high recall rate and low-computational cost.     

面向图像识别的多层脉冲神经网络学习算法综述

相较于第1代和第2代神经网络,第3代神经网络的脉冲神经网络是一种更加接近于生物神经网络的模型,因此更具有生物可解释性和低功耗性。基于脉冲神经元模型,脉冲神经网络可以通过脉冲信号的形式模拟生物信号在神经网络中的传播,通过脉冲神经元的膜电位变化来发放脉冲序列,脉冲序列通过时空联合表达不仅传递了空间信息还传递了时间信息。当前面向模式识别任务的脉冲神经网络模型性能还不及深度学习,其中一个重要原因在于脉冲神经网络的学习方法不成熟,深度学习中神经网络的人工神经元是基于实数形式的输出,这使得其可以使用全局性的反向传播算法对深度神经网络的参数进行训练,脉冲序列是二值性的离散输出,这直接导致对脉冲神经网络的训练存在一定困难,如何对脉冲神经网络进行高效训练是一个具有挑战的研究问题。本文首先总结了脉冲神经网络研究领域中的相关学习算法,然后对其中主要的方法:直接监督学习、无监督学习的算法以及ANN2SNN的转换算法进行分析介绍,并对其中代表性的工作进行对比分析,最后基于对当前主流方法的总结,对未来更高效、更仿生的脉冲神经网络参数学习方法进行展望。     

A new convex objective function for the supervised learning of single-layer neural networks

This paper proposes a novel supervised learning method for single-layer feedforward neural networks. This approach uses an alternative objective function to that based on the MSE, which measures the errors before the neuron's nonlinear activation functions instead of after them. In this case, the solution can be easily obtained solving systems of linear equations, i.e., requiring much less computational power than the one associated with the regular methods. A theoretical study is included to proof the approximated equivalence between the global optimum of the objective function based on the regular MSE criterion and the one of the proposed alternative MSE function.Furthermore, it is shown that the presented method has the capability of allowing incremental and distributed learning. An exhaustive experimental study is also presented to verify the soundness and efficiency of the method. This study contains 10 classification and 16 regression problems. In addition, a comparison with other high performance learning algorithms shows that the proposed method exhibits, in average, the highest performance and low-demanding computational requirements.     

Improvement of time forecasting models using a novel hybridization of bootstrap and double bootstrap artificial neural networks

Hybrid models such as the Artificial Neural Network-Autoregressive Integrated Moving Average (ANN–ARIMA) model are widely used in forecasting. However, inaccuracies and inefficiency remain in evidence. To yield the ANN–ARIMA with a higher degree of accuracy, efficiency and precision, the bootstrap and the double bootstrap methods are commonly used as alternative methods through the reconstruction of an ANN–ARIMA standard error. Unfortunately, these methods have not been applied in time series-based forecasting models. The aims of this study are twofold. First, is to propose the hybridization of bootstrap model and that of double bootstrap mode called Bootstrap Artificial Neural Network-Autoregressive Integrated Moving Average (B-ANN–ARIMA) and Double Bootstrap Artificial Neural Network-Autoregressive Integrated Moving Average (DB-ANN–ARIMA), respectively. Second, is to investigate the performance of these proposed models by comparing them with ARIMA, ANN and ANN–ARIMA. Our investigation is based on three well-known real datasets, i.e., Wolf’s sunspot data, Canadian lynx data and, Malaysia ringgit/United States dollar exchange rate data. Statistical analysis on SSE, MSE, RMSE, MAE, MAPE and VAF is then conducted to verify that the proposed models are better than previous ARIMA, ANN and ANN–ARIMA models. The empirical results show that, compared with ARIMA, ANNs and ANN–ARIMA models, the proposed models generate smaller values of SSE, MSE, RMSE, MAE, MAPE and VAF for both training and testing datasets. In other words, the proposed models are better than those that we compare with. Their forecasting values are closer to the actual values. Thus, we conclude that the proposed models can be used to generate better forecasting values with higher degree of accuracy, efficiency and, precision in forecasting time series results becomes a priority.     

A novel compensation-based recurrent fuzzy neural network and its learning algorithm

Based on detailed study on several kinds of fuzzy neural networks, we propose a novel compensation-based recurrent fuzzy neural network (CRFNN) by adding recurrent element and compensatory element to the conventional fuzzy neural network. Then, we propose a sequential learning method for the structure identification of the CRFNN in order to confirm the fuzzy rules and their correlative parameters effectively. Furthermore, we improve the BP algorithm based on the characteristics of the proposed CRFNN to train the network. By modeling the typical nonlinear systems, we draw the conclusion that the proposed CRFNN has excellent dynamic response and strong learning ability. Supported by the National High-Tech Research and Development Program of China (Grant No. 2006AA05A107) and Special Fund of Jiangsu Province for Technology Transfer (Grant No. BA2007008)     

Chaos control and associative memory of a time-delay globally coupled neural network using symmetric map

A chaotic neural network called time-delay globally coupled neural network using symmetric map (TDSG) is proposed for information processing applications. Firstly, its rich dynamic behaviors are exhibited and the output stability is demonstrated by using a parameter modulated control method. Secondly, the associative memory of TDSG is investigated by the control method. It is observed that the stable output sequence only contains stored pattern and its reverse pattern and the TDSG finally converges to the stored pattern which has the smallest Hamming distance to the initial patterns with noise. At last, strong information recovery ability of the TDSG is illustrated by comparative experiments.     

A simulational comparison of intelligent control algorithms on a direct drive manipulator

This paper investigates the control of nonlinear systems by neural networks and fuzzy logic. As the control methods, Gaussian neuro-fuzzy variable structure (GNFVS), feedback error learning architecture (FELA) and direct inverse modeling architecture (DIMA) are studied, and their performances are comparatively evaluated on a two degrees of freedom direct drive robotic manipulator with respect to trajectory tracking performance, computational complexity, design complexity, RMS errors, necessary training time in learning phase and payload variations.     

A novel analysis and design of a neural network assisted nonlinear controller for a bioreactor

A novel approach is presented for the analysis and the design of a controller for a bioreactor. It is based on the model reference control theory, assisted by a neural network identifier. The control objectives specified in the paper require the controller to be a nonlinear one, however, it is shown that it is stable in the sense of bounded input bounded output and locally stabilizing in the sense of Lyapunov. The feasibility and the efficacy of the proposed approach are tested on the benchmark problem. Copyright © 1999 John Wiley & Sons, Ltd.     

A new discretization algorithm based on range coefficient of dispersion and skewness for neural networks classifier

In this paper we propose a new static, global, supervised, incremental and bottom-up discretization algorithm based on coefficient of dispersion and skewness of data range. It automates the discretization process by introducing the number of intervals and stopping criterion. The results obtained using this discretization algorithm show that the discretization scheme generated by the algorithm almost has minimum number of intervals and requires smallest discretization time. The feedforward neural network with conjugate gradient training algorithm is used to compute the accuracy of classification from the data discretized by this algorithm. The efficiency of the proposed algorithm is shown in terms of better discretization scheme and better accuracy of classification by implementing it on six different real data sets.     

A novel hybrid learning algorithm for full Bayesian approach of artificial neural networks

The Bayesian neural networks are useful tools to estimate the functional structure in the nonlinear systems. However, they suffer from some complicated problems such as controlling the model complexity, the training time, the efficient parameter estimation, the random walk, and the stuck in the local optima in the high-dimensional parameter cases. In this paper, to alleviate these mentioned problems, a novel hybrid Bayesian learning procedure is proposed. This approach is based on the full Bayesian learning, and integrates Markov chain Monte Carlo procedures with genetic algorithms and the fuzzy membership functions. In the application sections, to examine the performance of proposed approach, nonlinear time series and regression analysis are handled separately, and it is compared with the traditional training techniques in terms of their estimation and prediction abilities.     

A novel deep convolutional neural network-bootstrap integrated method for RUL prediction of rolling bearing

In this study, a novel deep convolutional neural network-bootstrap-based integrated prognostic approach for the remaining useful life (RUL) prediction of rolling bearing is developed. The proposed architecture includes two main parts: 1) a deep convolutional neural network–multilayer perceptron (i.e., DCNN–MLP) dual network is utilized to simultaneously extract informative representations hidden in both time series-based and image-based features and to predict the RUL of bearings, and 2) the proposed dual network is embedded into the bootstrap-based implementation framework to quantify the RUL prediction interval. Unlike other deep-learning-based prognostic approaches, the proposed DCNN-bootstrap integrated method has two innovative features: 1) both 1D time series-based and 2D image-based features of bearings, which can multi-dimensionally characterize the degradation of bearings, are comprehensively leveraged by the proposed dual network, and 2) the RUL prediction interval can be effectively quantified without relying on the bearing’s physical or statistical prior information based on bootstrap implementation paradigm. The proposed approach is experimentally validated with two case studies on rolling element bearings, and comparisons with other state-of-the-art techniques are also presented. Subsequently, our code will be open sourced.     

A hybrid of electromagnetism-like mechanism and back-propagation algorithms for recurrent neural fuzzy systems design

This article introduces a novel hybrid evolutionary algorithm for recurrent fuzzy neural systems design in applications of nonlinear systems. The hybrid learning algorithm, IEMBP-improved electromagnetism-like (EM) with back-propagation (BP) technique, combines the advantages of EM and BP algorithms which provides high-speed convergence, higher accuracy and less computational complexity (computation time in seconds). In addition, the IEMBP needs only a small population to outperform the standard EM that uses a larger population. For a recurrent neural fuzzy system, IEMBP simulates the ‘attraction’ and ‘repulsion’ of charged particles by considering each neural system parameters as a charged particle. The EM algorithm is modified in such a way that the competition selection is adopted and the random neighbourhood local search is replaced by BP without evaluations. Thus, the IEMBP algorithm combines the advantages of multi-point search, global optimisation and faster convergence. Finally, several illustration examples for nonlinear systems are shown to demonstrate the performance and effectiveness of IEMBP.     

A comparison of experimental designs in the development of a neural network simulation metamodel

In this case study, we investigate the effects of experimental design on the development of artificial neural networks as simulation metamodels. A simple, deterministic combat model developed within the paradigm of system dynamics provides the underlying simulation. The neural network metamodels are developed using six different experimental design approaches. These include a traditional full factorial design, a random sampling design, a central composite design, a modified Latin Hypercube design and designs supplemented with domain knowledge. The results from this case study show how much impact the experimental design chosen for the neural network training set can have on the predictive accuracy achieved by the metamodel. We compare the networks in terms of various performance measures. The neural network developed from the modified Latin Hypercube design supplemented with domain knowledge produces the best performance, outperforming networks developed from other designs of the same size.     

A comparison of dispatching rules and genetic algorithms for job shop schedules of standard hydraulic cylinders

Hydraulic cylinders perform straight-line reciprocating movements, and they have been used widely in various forms in many different industries. In this paper, we select a sample of the various types of standard hydraulic cylinders. Each cylinder’s near-optimal processing time and the processing order of the cylinder’s parts are investigated using two different techniques. First, we study typical procedures, known as ‘Dispatching Rules’, which would be used in a job shop to resolve scheduling problems. Second, we investigate another kind of technique, called ‘Genetic Algorithms’. The goal of this paper, we show that efficient scheduling solutions are calculated by using dispatching rules and genetic algorithms for manufacturing standard hydraulic cylinders, and we propose that a way to use dispatching rules in association with genetic algorithms should be considered for resolving job shop scheduling problems.     

A comparison of neural network and multiple regression analysis in modeling capital structure

Empirical studies of the variation in debt ratios across firms have used statistical models singularly to analyze the important determinants of capital structure. Researchers, however, rarely employ non-linear models to examine the determinants and make little effort to identify a superior prediction model. This study adopts multiple linear regressions and artificial neural networks (ANN) models with seven explanatory variables of corporation’s feature and three external macro-economic control variables to analyze the important determinants of capital structures of the high-tech and traditional industries in Taiwan, respectively. Results of this study show that the determinants of capital structure are different in both industries. The major different determinants are business-risk and growth opportunities. Based on the values of RMSE, the ANN models achieve a better fit and forecast than the regression models for debt ratio, and ANNs are cable of catching sophisticated non-linear integrating effects in both industries. It seems that the relationships between debt ratio and independent variables are not linear. Managers can apply these results for their dynamic adjustment of capital structure in achieving optimality and maximizing firm’s value.     

A new robust training algorithm for a class of single-hidden layer feedforward neural networks

A robust training algorithm for a class of single-hidden layer feedforward neural networks (SLFNs) with linear nodes and an input tapped-delay-line memory is developed in this paper. It is seen that, in order to remove the effects of the input disturbances and reduce both the structural and empirical risks of the SLFN, the input weights of the SLFN are assigned such that the hidden layer of the SLFN performs as a pre-processor, and the output weights are then trained to minimize the weighted sum of the output error squares as well as the weighted sum of the output weight squares. The performance of an SLFN-based signal classifier trained with the proposed robust algorithm is studied in the simulation section to show the effectiveness and efficiency of the new scheme.     

AKSNet: A novel convolutional neural network with adaptive kernel width and sparse regularization for machinery fault diagnosis

Convolutional kernels have significant affections on feature learning of convolutional neural network (CNN). However, it is still a challenging problem to determine appropriate kernel width. Moreover, some features learned by convolutional layers are still redundant and noisy. Thus, adaptive selection of kernel width and feature selection of feature maps are key techniques to improve feature learning performance of CNNs. In this paper, a new deep neural network (DNN) model, adaptive kernel sparse network (AKSNet) is proposed to extract multi-scale fault features from one-dimensional (1-D) vibration signals. Firstly, an adaptive kernel selection method is developed, where multiple branches with different kernels are used to extract multi-scale features from vibration signals. Channel-wise attention is developed to fuse features generated by these kernels to obtain different informative scales. Secondly, a spatial attention is used for dynamic receptive field to focus on salient region of feature maps. Thirdly, a sparse regularization layer is embedded in the deep network to further filter noise and highlight impaction of the feature maps. Finally, two cases are adopted to verify effectiveness of AKSNet-based feature learning for bearing fault diagnosis. Experimental results show that AKSNet can effectively extract features from multi-channel vibration signals and then improves fault diagnosis performance of the classifier significantly. AKSNet shows better recognition performance in comparison with that of shallow neural networks and other typical DNNs.     

A review on the design and optimization of antennas using machine learning algorithms and techniques

This paper presents a focused and comprehensive literature survey on the use of machine learning (ML) in antenna design and optimization. An overview of the conventional computational electromagnetics and numerical methods used to gain physical insight into the design of the antennas is first presented. The major aspects of ML are then presented, with a study of its different learning categories and frameworks. An overview and mathematical briefing of regression models built with ML algorithms is then illustrated, with a focus on those applied in antenna synthesis and analysis. An in‐depth overview on the different research papers discussing the design and optimization of antennas using ML is then reported, covering the different techniques and algorithms applied to generate antenna parameters based on desired radiation characteristics and other antenna specifications. Various investigated antennas are sorted based on antenna type and configuration to assist the readers who wish to work with a specific type of antennas using ML.     

A comparison between functional networks and artificial neural networks for the prediction of fishing catches

In recent years, functional networks have emerged as an extension of artificial neural networks (ANNs). In this article, we apply both network techniques to predict the catches of the Prionace Glauca (a class of shark) and the Katsowonus Pelamis (a variety of tuna, more commonly known as the Skipjack). We have developed an application that will help reduce the search time for good fishing zones and thereby increase the fleets competitivity. Our results show that, thanks to their superior learning and generalisation capacities, functional networks are more efficient than ANNs. Our data proceeds from remote sensors. Their spectral signatures allow us to calculate products that are useful for ecological modelling. After an initial phase of digital image processing, we created a database that provides all the necessary patterns to train both network types.     

A comparison of linear and neural network ARX models applied to a prediction of the indoor temperature of a building

A neural network auto regressive with exogenous input (NNARX) model is used to predict the indoor temperature of a residential building. Firstly, the optimal regressor of a linear ARX model is identified by minimising Akaikes final prediction error (FPE). This regressor is then used as the input vector of a fully connected feedforward neural network with one hidden layer of ten units and one output unit. Results show that the NNARX model outperforms the linear model considerably: the sum of the squared error (SSE) is 15.0479 with the ARX model and 2.0632 with the NNARX model. The optimal network topology is subsequently determined by pruning the fully connected network according to the optimal brain surgeon (OBS) strategy. With this procedure near 73% of connections were removed and, as a result, the performance of the network has been improved: the SSE is equal to 0.9060.