Classification of freeway traffic patterns for incident detectionusing constructive probabilistic neural networks

This paper proposes a new technique for freeway incident detection using a constructive probabilistic neural network (CPNN). The CPNN incorporates a clustering technique with an automated training process. The work reported in this paper was conducted on Ayer Rajah Expressway (AYE) in Singapore for incident detection model development, and subsequently on I-880 freeway in California, for model adaptation. The model developed achieved incident detection performance of 92% detection rate and 0.81% false alarm rate on AYE, and 91.30% detection rate and 0.27% false alarm rate on I-880 freeway using the proposed adaptation method. In addition to its superior performance, the network pruning method employed facilitated model size reduction by a factor of 11 compared to a conventional probabilistic neural network. A more impressive size reduction by a factor of 50 was achieved after the model was adapted for the new site. The results from this paper suggest that CPNN is a better adaptive classifier for incident detection problem with a changing site traffic environment.     

Learning only when necessary: better memories of correlated patterns in networks with bounded synapses

Learning in a neuronal network is often thought of as a linear superposition of synaptic modifications induced by individual stimuli. However, since biological synapses are naturally bounded, a linear superposition would cause fast forgetting of previously acquired memories. Here we show that this forgetting can be avoided by introducing additional constraints on the synaptic and neural dynamics. We consider Hebbian plasticity of excitatory synapses. A synapse is modified only if the postsynaptic response does not match the desired output. With this learning rule, the original memory performances with unbounded weights are regained, provided that (1) there is some global inhibition, (2) the learning rate is small, and (3) the neurons can discriminate small differences in the total synaptic input (e.g., by making the neuronal threshold small compared to the total postsynaptic input). We prove in the form of a generalized perceptron convergence theorem that under these constraints, a neuron learns to classify any linearly separable set of patterns, including a wide class of highly correlated random patterns. During the learning process, excitation becomes roughly balanced by inhibition, and the neuron classifies the patterns on the basis of small differences around this balance. The fact that synapses saturate has the additional benefit that nonlinearly separable patterns, such as similar patterns with contradicting outputs, eventually generate a subthreshold response, and therefore silence neurons that cannot provide any information.     

Nonlinear enhancement of noisy speech, using continuous attractor dynamics formed in recurrent neural networks

Here, formation of continuous attractor dynamics in a nonlinear recurrent neural network is used to achieve a nonlinear speech denoising method, in order to implement robust phoneme recognition and information retrieval. Formation of attractor dynamics in recurrent neural network is first carried out by training the clean speech subspace as the continuous attractor. Then, it is used to recognize noisy speech with both stationary and nonstationary noise. In this work, the efficiency of a nonlinear feedforward network is compared to the same one with a recurrent connection in its hidden layer. The structure and training of this recurrent connection, is designed in such a way that the network learns to denoise the signal step by step, using properties of attractors it has formed, along with phone recognition. Using these connections, the recognition accuracy is improved 21% for the stationary signal and 14% for the nonstationary one with 0db SNR, in respect to a reference model which is a feedforward neural network.     

A swarm intelligent method for traffic light scheduling: application to real urban traffic networks

Traffic lights play an important role nowadays for solving complex and serious urban traffic problems. How to optimize the schedule of hundreds of traffic lights has become a challenging and exciting problem. This paper proposes an inner and outer cellular automaton mechanism combined with particle swarm optimization (IOCA-PSO) method to achieve a dynamic and real-time optimization scheduling of urban traffic lights. The IOCA-PSO method includes the inner cellular model (ICM), the outer cellular model (OCM), and the fitness function. Our work can be divided into following parts: (1) Concise basic transition rules and affiliated transition rules are proposed in ICM, which can help the proposed phase cycle planning (PCP) algorithm achieve a globally sophisticated scheduling and offer effective solutions for different traffic problems; (2) Benefited from the combination of cellular automaton (CA) and particle swarm optimization (PSO), the proposed inner and outer cellular PSO (IOPSO) algorithm in OCM offers a strong search ability to find out the optimal timing control; (3) The proposed fitness function can evaluate and conduct the optimization of traffic lights’ scheduling dynamically for different aims by adjusting parameters. Extensive experiments show that, compared with the PSO method, the genetic algorithm method and the RANDOM method in real cases, IOCA-PSO presents distinct improvements under different traffic conditions, which shows a high adaptability of the proposed method in urban traffic network scales under different traffic flow states, intersection numbers, and vehicle numbers.     

Distributed-information neural control: the case of dynamic routingin traffic networks

Large-scale traffic networks can be modeled as graphs in which a set of nodes are connected through a set of links that cannot be loaded above their traffic capacities. Traffic flows may vary over time. Then the nodes may be requested to modify the traffic flows to be sent to their neighboring nodes. In this case, a dynamic routing problem arises. The decision makers are realistically assumed 1) to generate their routing decisions on the basis of local information and possibly of some data received from other nodes, typically, the neighboring ones and 2) to cooperate on the accomplishment of a common goal, that is, the minimization of the total traffic cost. Therefore, they can be regarded as the cooperating members of informationally distributed organizations, which, in control engineering and economics, are called team organizations. Team optimal control problems cannot be solved analytically unless special assumptions on the team model are verified. In general, this is not the case with traffic networks. An approximate resolutive method is then proposed, in which each decision maker is assigned a fixed-structure routing function where some parameters have to be optimized. Among the various possible fixed-structure functions, feedforward neural networks have been chosen for their powerful approximation capabilities. The routing functions can also be computed (or adapted) locally at each node. Concerning traffic networks, we focus attention on store-and-forward packet switching networks, which exhibit the essential peculiarities and difficulties of other traffic networks. Simulations performed on complex communication networks point out the effectiveness of the proposed method.     

Intelligent agent construction using the attentive characteristic patterns of chaotic neural networks

Humans learn from incidents in their own life and reflects these in subsequent actions as their own experiences. These experiences are memorized in the brain and recollected when necessary. This research incorporates this type of intelligent information processing mechanism and applies it to an autonomous agent. In the proposed system, the reinforcement Q-learning method is used. Autoassociative chaotic neural networks are also used as mutual associative memory systems. However, an agent cannot retrieve all stored patterns exactly, especially in the case of too many stored patterns and a strong correlation among them. To solve this problem, we propose to use types of attentive parameters and attentive characteristic patterns. The attentive characteristic pattern is part of the stored patterns. When robots concentrate their attention on a specific part of a stored pattern, i.e., the attentive characteristic pattern, whole stored patterns are retrieved easily and completely. Finally, the effectiveness of the proposed method is verified through a simulation applied to plural maze-searching problems.     

Implementing the induction-motor drive with four-switch inverter: An application of neural networks

This paper reports a four switch based three-phase voltage source inverter using space vector pulse width modulation (SVPWM), and designed with a three-layer feed forward back propagation based artificial neural network (ANN). The input–output samples, obtained using simulations in Matlab Simulink, were used for the extensive training of the neural network. Matlab interface with National Instruments’ NI-USB-6259 BNC was used for implementing the designed scheme with calculated weights and biases. The designed ANN based SVPWM model receives command voltage and reference speed as the inputs and generates pulse width modulated waves for the four-switch three-phase inverter bridge. The V/f ratio can be controlled by controlling the input parameters of the ANN generating PWM pulses. The simulations and experimental results, and harmonic analysis with the designed ANN structure are presented at different base speeds. The designed model was tested in under modulation, over modulation and unity modulation mode of operation and tuned to give minimum total harmonic distortion. Harmonic results at different modulation indexes are also presented. The ANN based implementation reduces the complexity of control system and overall cost reduction is achieved by the combination of FSTPI and ANN.     

Nonlinear internal model control using neural networks: an application for machining processes

This paper presents a novel approach to the control of the cutting force on the basis of the internal model control (IMC) principle. The main goal is to control a single output variable, the cutting force, by changing a single input variable, the feedrate. A neural model is used as an internal model to determine the control inputs (feedrate) necessary to keep the cutting force constant. Three approaches, the fuzzy logic controller (FLC), the direct inverse controller (DIC) and the IMC, based on artificial neural networks (IMC-NN), are simulated and their performances are assessed in terms of several performance measurements. The results demonstrate that IMC-NN strategy provides a better disturbance rejection than FLC for the cases analysed.     

The application of neural networks to the papermaking industry

This paper describes the application of neural network techniques to the papermaking industry, particularly for the prediction of paper "curl". Paper curl is an important quality measure that can only be measured reliably off-line after manufacture, making it difficult to control. Here, we predict, before paper manufacture from characteristics of the current reel, whether the paper curl will be acceptable and the level of curl. For both issues the case of predicting the probability that paper will be "out-of-specification" and that of predicting the level of curl, we include confidence intervals indicating to the machine operator whether the predictions should be trusted. The results and the associated discussion describe a successful application of neural networks to a difficult, but important, real-world task taken from the papermaking industry. In addition the techniques described are widely applicable to industry where direct prediction of a quality measure and its acceptability are desirable.     

Generation of optimal artificial neural networks using a pattern search algorithm: application to approximation of chemical systems

A pattern search optimization method is applied to the generation of optimal artificial neural networks (ANNs). Optimization is performed using a mixed variable extension to the generalized pattern search method. This method offers the advantage that categorical variables, such as neural transfer functions and nodal connectivities, can be used as parameters in optimization. When used together with a surrogate, the resulting algorithm is highly efficient for expensive objective functions. Results demonstrate the effectiveness of this method in optimizing an ANN for the number of neurons, the type of transfer function, and the connectivity among neurons. The optimization method is applied to a chemistry approximation of practical relevance. In this application, temperature and a chemical source term are approximated as functions of two independent parameters using optimal ANNs. Comparison of the performance of optimal ANNs with conventional tabulation methods demonstrates equivalent accuracy by considerable savings in memory storage. The architecture of the optimal ANN for the approximation of the chemical source term consists of a fully connected feedforward network having four nonlinear hidden layers and 117 synaptic weights. An equivalent representation of the chemical source term using tabulation techniques would require a 500 x 500 grid point discretization of the parameter space.     

Pruning artificial neural networks: An example using land cover classification of multi-sensor images

The use of three techniques for pruning artificial neural networks (magnitude-based pruning, optimum brain damage and optimal brain surgeon) is investigated, using microwave SAR and optical SPOT data to classify land cover in a test area located in eastern England. Results show that it is possible to reduce network size significantly without compromising overall classification accuracy; indeed, accuracy may rise as the number of links decreases. However, individual class accuracies and the spatial distribution of the pixels forming the individual classes may change significantly. If the network is pruned too severely some classes may be eliminated altogether. In terms of maintaining overall classification accuracy the optimal brain surgeon algorithm gave the best results, and magnitude-based pruning also gave good results despite its simplicity. The optimum brain damage algorithm performed least well of the three methods tested.     

An optimization approach to design of cellular neural networks

In this paper, we consider the problem of realizing associative memories via cellular neural networks (CNNs). After introducing qualitative properties of the CNN model, we formulate the synthesis of CNNs that can store given binary vectors with improved performance as a constrained optimization problem. Next, we observe that this problem's constraints can be transformed into simple inequalities involving linear matrix inequalities. Finally, we reformulate the synthesis problem as a generalized eigenvalue problem, which can be efficiently solved by recently developed interior point methods. The validity of the proposed approach is illustrated by a design example.     

An application of neural networks for harmonic coefficients and relative phase shifts detection

The varying the phase shifts will completely change the shape of the distorted wave, and may thus greatly affect the ability of the neural network to recognize harmonics. In this study, feed forward neural networks were used for the detection of the harmonic coefficients and relative phase shifts. The distorted wave including uniform distributed 5th, 7th, 11th, 13th, 17th, 19th, 23rd, 25th harmonics with up to 20° relative phase shifts were simulated and used. Two neural networks were used for this purpose. One of the neural networks was used for the detection of the 5th, 7th, 11th, 13th harmonic coefficients and the other was used for the detection of the relative phase shifts of these harmonics. Scaled conjugate gradient algorithm was used as training algorithm for the weights update of the neural networks. The results show that these neural networks are applicable to detect each harmonic coefficient and relative phase shift effectively.     

Reliability estimation using a genetic algorithm-based artificial neural network: An application to a load-haul-dump machine

In this study, a neural network-based model for forecasting reliability was developed. A genetic algorithm was applied for selecting neural network parameters like learning rate (η) and momentum (μ). The input variables of the neural network model were selected by maximizing the mean entropy value. The developed model was validated by applying two benchmark data sets. A comparative study reveals that the proposed method performs better than existing methods on benchmark data sets. A case study was conducted on a load-haul-dump (LHD) machine operated at a coal mine in Alaska, USA. Past time-to-failure data for the LHD machine were collected, and cumulative time-to-failure was calculated for reliability modeling. The results demonstrate that the developed model performs well with high accuracy (R² = 0.94) in the failure prediction of a LHD machine.     

Multi-label semantic concept detection in videos using fusion of asymmetrically trained deep convolutional neural networks and foreground driven concept co-occurrence matrix

Describing visual contents in videos by semantic concepts is an effective and realistic approach that can be used in video applications such as annotation, indexing, retrieval and ranking. In these applications, video data needs to be labelled with some known set of labels or concepts. Assigning semantic concepts manually is not feasible due to the large volume of ever-growing video data. Hence, automatic semantic concept detection of videos is a hot research area. Recently Deep Convolutional Neural Networks (CNNs) used in computer vision tasks are showing remarkable performance. In this paper, we present a novel approach for automatic semantic video concept detection using deep CNN and foreground driven concept co-occurrence matrix (FDCCM) which keeps foreground to background concept co-occurrence values, built by exploiting concept co-occurrence relationship in pre-labelled TRECVID video dataset and from a collection of random images extracted from Google Images. To deal with the dataset imbalance problem, we have extended this approach by making a fusion of two asymmetrically trained deep CNNs and used FDCCM to further improve concept detection. The performance of the proposed approach is compared with state-of-the-art approaches for the video concept detection over the widely used TRECVID data set and is found to be superior to existing approaches.     

An intelligent model to predict the life condition of crude oil pipelines using artificial neural networks

Neural Computing and Applications - Inefficient scheduling of a pipeline system may lead to severe degradation and substantial economic losses. Earlier studies mostly focussed on corrosion and...     

Large-scale simulations of self-organizing neural networks on parallel computers: application to biological modelling

In this contribution we report about a study of a very versatile neural network algorithm known as “Self-organizing Feature Maps” and based on earlier work of Kohonen [1,2]. In its original version, the algorithm addresses a fundamental issue of brain organization, namely how topographically ordered maps of sensory information can be formed by learning.

This algorithm is investigated for a large number of neurons (up to 16 K) and for an input space of dimension d900. To meet the computational demands this algorithm was implemented on two parallel machines, on a self-built Transputer systolic ring and on a Connection Machine CM-2.

We will present below

1. (i) a simulation based on the feature map algorithm modelling part of the synaptic organization in the “hand-region” of the somatosensory cortex,

2. (ii) a study of the influence of the dimension of the input-space on the learning process,

3. (iii) a simulation of the extended algorithm, which explicitly includes lateral interactions, and

4. (iv) a comparison of the transputer-based “coarse-grained” implementation of the model, and the “fine-grained” implementation of the same system on the Connection Machine.

     

Nonlinear normalization of input patterns to speaker variability in speech recognition neural networks

The issue of input variability resulting from speaker changes is one of the most crucial factors influencing the effectiveness of speech recognition systems. A solution to this problem is adaptation or normalization of the input, in a way that all the parameters of the input representation are adapted to that of a single speaker, and a kind of normalization is applied to the input pattern against the speaker changes, before recognition. This paper proposes three such methods in which some effects of the speaker changes influencing speech recognition process is compensated. In all three methods, a feed-forward neural network is first trained for mapping the input into codes representing the phonetic classes and speakers. Then, among the 71 speakers used in training, the one who is showing the highest percentage of phone recognition accuracy is selected as the reference speaker so that the representation parameters of the other speakers are converted to the corresponding speech uttered by him. In the first method, the error back-propagation algorithm is used for finding the optimal point of every decision region relating to each phone of each speaker in the input space for all the phones and all the speakers. The distances between these points and the corresponding points related to the reference speaker are employed for offsetting the speaker change effects and the adaptation of the input signal to the reference speaker. In the second method, using the error back-propagation algorithm and maintaining the reference speaker data as the desirable speaker output, we correct all the speech signal frames, i.e., the train and the test datasets, so that they coincide with the corresponding speech of the reference speaker. In the third method, another feed-forward neural network is applied inversely for mapping the phonetic classes and speaker information to the input representation. The phonetic output retrieved from the direct network along with the reference speaker data are given to the inverse network. Using this information, the inverse network yields an estimation of the input representation adapted to the reference speaker. In all three methods, the final speech recognition model is trained using the adapted training data, and is tested by the adapted testing data. Implementing these methods and combining the final network results with un-adapted network based on the highest confidence level, an increase of 2.1, 2.6 and 3% in phone recognition accuracy on the clean speech is obtained from the three methods, respectively.     

Correction to: Robust facial landmark extraction scheme using multiple convolutional neural networks

Multimedia Tools and Applications - In the original publication, the author name “Seungmin Rho” was incorrectly spelled as “Seumgmin Rho”. The correct author name is given...