前馈型神经元网络中的放电频率传递分析

本文建立了一个基于Hodgkin-Huxley神经元的前馈神经元网络模型,研究了平均放电频率在前馈神经元网络中的传递情况。研究结果显示,适当的层间连接概率与输人噪声强度能够提髙前馈神经元网络的同步效率,进而增强网络稳定传递放电频率的性能。此外,通过引入并调节突触时滞,发现适当的时滞对神经元耦合系统的完全同步和前馈神经元网络内信息传输有明显的促进作用。     

Correlated firing in a feedforward network with Mexican-hat-type connectivity

We report on deterministic and stochastic evolutions of firing states through a feedforward neural network with Mexican-hat-type connectivity. The prevalence of columnar structures in a cortex implies spatially localized connectivity between neural pools. Although feedforward neural network models with homogeneous connectivity have been intensively studied within the context of the synfire chain, the effect of local connectivity has not yet been studied so thoroughly. When a neuron fires independently, the dynamics of macroscopic state variables (a firing rate and spatial eccentricity of a firing pattern) is deterministic from the law of large numbers. Possible stable firing states, which are derived from deterministic evolution equations, are uniform, localized, and nonfiring. The multistability of these three states is obtained where the excitatory and inhibitory interactions among neurons are balanced. When the presynapse-dependent variance in connection efficacies is incorporated into the network, the variance generates common noise. Then the evolution of the macroscopic state variables becomes stochastic, and neurons begin to fire in a correlated manner due to the common noise. The correlation structure that is generated by common noise exhibits a nontrivial bimodal distribution. The development of a firing state through neural layers does not converge to a certain fixed point but keeps on fluctuating.     

Mean instantaneous firing frequency is always higher than the firing rate

Frequency coding is considered one of the most common coding strategies employed by neural systems. This fact leads, in experiments as well as theoretical studies, to construction of so-called transfer functions, where the output firing frequency is plotted against the input intensity. The term "firing frequency" can be understood differently in different contexts. Basically, it means that the number of spikes over an interval of preselected length is counted and then divided by the length of the interval, but due to the obvious limitations, the length of observation cannot be arbitrarily long. Then firing frequency is defined as reciprocal to the mean interspike interval. In parallel, an instantaneous firing frequency can be defined as reciprocal to the length of current interspike interval, and by taking a mean of these, the definition can be extended to introduce the mean instantaneous firing frequency. All of these definitions of firing frequency are compared in an effort to contribute to a better understanding of the input-output properties of a neuron.     

前馈环形振荡器的结构与相位噪声研究

基于高速串行通信系统中锁相环和时钟数据恢复电路的需求,研究了前馈环形振荡器的结构与工作原理;在传统结构的基础上,将前馈路径耦合至主路径反相器的源极,可以提高输出信号的边沿速率;最后基于Hajimiri模型的脉冲灵敏度函数进行分析,提出的结构有效降低了热噪声和闪烁噪声的引入.在28 nm CMOS工艺下设计了单源极前馈型...     

Establishing impacts of the inputs in a feedforward neural network

Artificial neural network models are now being widely used in various areas of statistical research. Nevertheless, there is a certain degree of reluctance amongst members of the business profession in applying neural networks to business analysis. One of the major causes of scepticism is the inability of the models to provide explanation on how they reach their decisions. The current experiment is concerned with solving this problem by developing a framework for establishing the impacts of the input variables on the network output. The framework was tested on a feedforward neural network model for turnover forecasting which was developed in co-operation with a British retailer using real world marketing data. The results obtained are compared with those from a sensitivity analysis.     

Visualizing the function computed by a feedforward neural network

A method for visualizing the function computed by a feedforward neural network is presented. It is most suitable for models with continuous inputs and a small number of outputs, where the output function is reasonably smooth, as in regression and probabilistic classification tasks. The visualization makes readily apparent the effects of each input and the way in which the functions deviate from a linear function. The visualization can also assist in identifying interactions in the fitted model. The method uses only the input-output relationship and thus can be applied to any predictive statistical model, including bagged and committee models, which are otherwise difficult to interpret. The visualization method is demonstrated on a neural network model of how the risk of lung cancer is affected by smoking and drinking.     

Duality of rate coding and temporal coding in multilayered feedforward networks

A functional role for precise spike timing has been proposed as an alternative hypothesis to rate coding. We show in this article that both the synchronous firing code and the population rate code can be used dually in a common framework of a single neural network model. Furthermore, these two coding mechanisms are bridged continuously by several modulatable model parameters, including shared connectivity, feedback strength, membrane leak rate, and neuron heterogeneity. The rates of change of these parameters are closely related to the response time and the timescale of learning.     

Precision constrained stochastic resonance in a feedforward neural network

Stochastic resonance (SR) is a phenomenon in which the response of a nonlinear system to a subthreshold information-bearing signal is optimized by the presence of noise. By considering a nonlinear system (network of leaky integrate-and-fire (LIF) neurons) that captures the functional dynamics of neuronal firing, we demonstrate that sensory neurons could, in principle harness SR to optimize the detection and transmission of weak stimuli. We have previously characterized this effect by use of signal-to-noise ratio (SNR). Here in addition to SNR, we apply an entropy-based measure (Fisher information) and compare the two measures of quantifying SR. We also discuss the performance of these two SR measures in a full precision floating point model simulated in Java and in a precision limited integer model simulated on a field programmable gate array (FPGA). We report in this study that stochastic resonance which is mainly associated with floating point implementations is possible in both a single LIF neuron and a network of LIF neurons implemented on lower resolution integer based digital hardware. We also report that such a network can improve the SNR and Fisher information of the output over a single LIF neuron.     

Integration of feedforward neural network and finite element in the draw-bend springback prediction

To achieve accurate results, current nonlinear elastic recovery applications of finite element (FE) analysis have become more complicated for sheet metal springback prediction. In this paper, an alternative modelling method able to facilitate nonlinear recovery was developed for springback prediction. The nonlinear elastic recovery was processed using back-propagation networks in an artificial neural network (ANN). This approach is able to perform pattern recognition and create direct mapping of the elastically-driven change after plastic deformation. The FE program for the sheet metal springback experiment was carried out with the integration of ANN. The results obtained at the end of the FE analyses were found to have improved in comparison to the measured data.     

Document segmentation using textural features summarization and feedforward neural network

Document Segmentation is a process that aims to filter documents while identifying certain regions of interest. Generally, the regions of interest include texts, graphics (image occupied regions) and the background. This paper presents a novel top-bottom approach to perform document segmentation using texture features that are extracted from the specified/selected documents. A mask of suitable size is used to summarize textural features, and statistical parameters are captured as blocks in document images. Four textural features that are extracted from masks using the gray level co-occurrence matrix (glcm) include entropy, contrast, energy and homogeneity. Furthermore, two statistical parameters extracted from corresponding masks are the modal and median pixel values. The extracted attributes allow the classification of each mask or block as text, graphics, and background. A feedforward network is trained on the 6 extracted attributes, using documents obtained from a public database ; an error rate of 15.77 % is achieved. Furthermore, it is shown that this novel approach produces promising performance in segmenting documents and is expected to be significantly efficient for content-based information retrieval systems. Detection of duplicate documents within large databases is another potential area of application.     

The layer-wise method and the backpropagation hybrid approach tolearning a feedforward neural network

Feedforward neural networks (FNNs) have been proposed to solve complex problems in pattern recognition and classification and function approximation. Despite the general success of learning methods for FNNs, such as the backpropagation (BP) algorithm, second-order optimization algorithms and layer-wise learning algorithms, several drawbacks remain to be overcome. In particular, two major drawbacks are convergence to a local minima and long learning time. We propose an efficient learning method for a FNN that combines the BP strategy and optimization layer by layer. More precisely, we construct the layer-wise optimization method using the Taylor series expansion of nonlinear operators describing a FNN and propose to update weights of each layer by the BP-based Kaczmarz iterative procedure. The experimental results show that the new learning algorithm is stable, it reduces the learning time and demonstrates improvement of generalization results in comparison with other well-known methods.     

A formal selection and pruning algorithm for feedforward artificialneural network optimization

A formal selection and pruning technique based on the concept of local relative sensitivity index is proposed for feedforward neural networks. The mechanism of backpropagation training algorithm is revisited and the theoretical foundation of the improved selection and pruning technique is presented. This technique is based on parallel pruning of weights which are relatively redundant in a subgroup of a feedforward neural network. Comparative studies with a similar technique proposed in the literature show that the improved technique provides better pruning results in terms of reduction of model residues, improvement of generalization capability and reduction of network complexity. The effectiveness of the improved technique is demonstrated in developing neural network models of a number of nonlinear systems including three bit parity problem, Van der Pol equation, a chemical processes and two nonlinear discrete-time systems using the backpropagation training algorithm with adaptive learning rate.     

Adaptive sliding mode approach for learning in a feedforward neural network

An adaptive learning algorithm is proposed for a feedforward neural network. The design principle is based on the sliding mode concept. Unlike the existing algorithms, the adaptive learning algorithm developed does not require a prioriknowledge of upper bounds of bounded signals. The convergence of the algorithm is established and conditions given. Simulations are presented to show the effectiveness of the algorithm.     

径向基网络在密相气力输送中的应用

在水平管道中,用压缩空气对玻璃珠进行密相气力输送压损实验研究。管道压降是密相气力输送系统的关键参数之一,它的大小基本决定了动力的大小。传统的方法完全靠经验公式和经验来决定动力大小,误差较大。本文提出了基于径向基网络的密相气力输送管道压降模型,对不同流态下管道压降进行了仿真。结果表明,RBF网络能对不同流态下的管道压降进行较好的仿真。RBF网络的收敛速度快,可实现密相气力输送参数的在线控制。     

Short-term potentiation effect on pattern recall in sparsely coded neural network

It has been shown in studies of biological synaptic plasticity that synaptic efficacy can change in a very short time window, compared to the time scale associated with typical neural events. This time scale is small enough to possibly have an effect on pattern recall processes in neural networks. We study properties of a neural network which uses a cyclic Hebb rule. Then we add the short term potentiation of synapses in the recall phase. We show that this approach preserves the ability of the network to recognize the patterns stored by the network and that the network does not respond to other patterns at the same time. We show that this approach dramatically increases the capacity of the network at the cost of a longer pattern recall process. We discuss that the network possesses two types of recall. The fast recall does not need synaptic plasticity to recognize a pattern, while the slower recall utilizes synaptic plasticity. This is something that we all experience in our daily lives: some memories can be recalled promptly whereas recollection of other memories requires much more time.     

A weight initialization method for improving training speed in feedforward neural network

An algorithm for determining the optimal initial weights of feedforward neural networks based on the Cauchy's inequality and a linear algebraic method is developed. The algorithm is computational efficient. The proposed method ensures that the outputs of neurons are in the active region and increases the rate of convergence. With the optimal initial weights determined, the initial error is substantially smaller and the number of iterations required to achieve the error criterion is significantly reduced. Extensive tests were performed to compare the proposed algorithm with other algorithms. In the case of the sunspots prediction, the number of iterations required for the network initialized with the proposed method was only 3.03% of those started with the next best weight initialization algorithm.     

Fruit classification by biogeography‐based optimization and feedforward neural network

Accurate fruit classification is difficult to accomplish because of the similarities among the various categories. In this paper, we proposed a novel fruit‐classification system, with the goal of recognizing fruits in a more efficient way. Our methodology included the following steps. First, a four‐step pre‐processing was employed. Second, the features (colour, shape, and texture) were extracted. Third, we utilized principal component analysis to remove excessive features. Fourth, a novel fruit‐classification system based on biogeography‐based optimization (BBO) and feedforward neural network (FNN) was proposed, with the short name of BBO‐FNN. The experiment employed over 1653 chromatic fruit images (18 categories) by fivefold stratified cross‐validation. The results showed that the proposed BBO‐FNN yielded an overall accuracy of 89.11%, which was higher than the five state‐of‐the‐art methods: genetic algorithm‐FNN, artificial bee colony‐FNN, particle swarm optimization‐FNN, kernel support vector machine, and ant colony optimization‐FNN. Also, the BBO‐FNN achieved the same accuracy as fitness‐scaling chaotic artificial bee colony‐FNN, but it performed much faster than the latter. The proposed BBO‐FNN was effective in fruit‐classification in terms of classification accuracy and computation time. This indicated that it can be applied in credible use.     

Post-processing of coded images by neural network cancellation of the unmasked noise

This paper presents a new application of neural networks for the post-processing of coded images. It is based on a model of the human visual system. The image affected by coding noise is decomposed into perceptual channel components. The image restoration stage is realized by filtering the perceptual components of the channels for which the noise power is not masked by the image power. This operation, referred as cancellation of the unmasked noise, is performed using a multi-layer perceptron (MLP) network. Different network structures have been considered for this purpose. Simulation results of the processing scheme show significant improvements in both visual and objective (SNR) quality for post-processed images affected by DCT or subband coding noise.     

Metaheuristics for the feedforward artificial neural network (ANN) architecture optimization problem

This article deals with evolutionary artificial neural network (ANN) and aims to propose a systematic and automated way to find out a proper network architecture. To this, we adapt four metaheuristics to resolve the problem posed by the pursuit of optimum feedforward ANN architecture and introduced a new criteria to measure the ANN performance based on combination of training and generalization error. Also, it is proposed a new method for estimating the computational complexity of the ANN architecture based on the number of neurons and epochs needed to train the network. We implemented this approach in software and tested it for the problem of identification and estimation of pollution sources and for three separate benchmark data sets from UCI repository. The results show the proposed computational approach gives better performance than a human specialist, while offering many advantages over similar approaches found in the literature.