A novel optimized GA–Elman neural network algorithm

Neural Computing and Applications - The Elman neural network has good dynamic properties and strong global stability, being most widely used to deal with nonlinear, dynamic, and complex data....     

Modular design of a hybrid genetic algorithm for a flexible job–shop scheduling problem

The aim of the job–shop scheduling problem is to optimize the task planning in an industrial plant satisfying time and technological constraints. The existing algorithmic and mathematical methods for solving this problem usually have high computational complexities making them intractable. Flexible job–shop scheduling becomes even more complex, since it allows one to assign each operation to a resource from a set of suitable ones. Alternative heuristic methods are only able to satisfy part of the constraints applicable to the problem. Moreover, these solutions usually offer little flexibility to adapt them to new requirements. This paper describes research within heuristic methods that combines genetic algorithms with repair heuristics. Firstly, it uses a genetic algorithm to provide a non-optimal solution for the problem, which does not satisfy all its constraints. Then, it applies repair heuristics to refine this solution. There are different types of heuristics, which correspond to the different types of constraints. A heuristic is intended to evaluate and slightly modify a solution that violates a constraint in a way that avoids or mitigates such violation. This approach improves the adaptability of the solution to a problem, as some changes can be addressed just modifying the considered chromosome or heuristics. The proposed solution has been tested in order to analyse its level of constraint satisfaction and its makespan, which are two of the main parameters considered in these types of problems. The paper discusses this experimentation showing the improvements over existing methods.     

Gait modification and optimization using neural network–genetic algorithm approach: Application to knee rehabilitation

Gait modification strategies play an important role in the overall success of total knee arthroplasty. There are a number of studies based on multi-body dynamic (MBD) analysis that have minimized knee adduction moment to offload knee joint. Reducing the knee adduction moment, without consideration of the actual contact pressure, has its own limitations. Moreover, MBD-based framework that mainly relies on iterative trial-and-error analysis, is fairly time consuming. This study embedded a time-delay neural network (TDNN) in a genetic algorithm (GA) as a cost effective computational framework to minimize contact pressure. Multi-body dynamic and finite element analyses were performed to calculate gait kinematics/kinetics and the resultant contact pressure for a number of experimental gait trials. A TDNN was trained to learn the nonlinear relation between gait parameters (inputs) and contact pressures (output). The trained network was then served as a real-time cost function in a GA-based global optimization to calculate contact pressure associated with each potential gait pattern. Two optimization problems were solved: first, knee flexion angle was bounded within the normal patterns and second, knee flexion angle was allowed to be increased beyond the normal walking. Designed gait patterns were evaluated through multi-body dynamic and finite element analyses.The TDNN-GA resulted in realistic gait patterns, compared to literature, which could effectively reduce contact pressure at the medial tibiofemoral knee joint. The first optimized gait pattern reduced the knee contact pressure by up to 21% through modifying the adjacent joint kinematics whilst knee flexion was preserved within normal walking. The second optimized gait pattern achieved a more effective pressure reduction (25%) through a slight increase in the knee flexion at the cost of considerable increase in the ankle joint forces. The proposed approach is a cost-effective computational technique that can be used to design a variety of rehabilitation strategies for different joint replacement with multiple objectives.     

Near‐surface air temperature estimation from ASTER data based on neural network algorithm

An algorithm based on the radiance transfer model (MODTRAN4) and a dynamic learning neural network for estimation of near‐surface air temperature from ASTER data are developed in this paper. MODTRAN4 is used to simulate radiance transfer from the ground with different combinations of land surface temperature, near surface air temperature, emissivity and water vapour content. The dynamic learning neural network is used to estimate near surface air temperature. The analysis indicates that near surface air temperature cannot be directly and accurately estimated from thermal remote sensing data. If the land surface temperature and emissivity were made as prior knowledge, the mean and the standard deviation of estimation error are both about 1.0 K. The mean and the standard deviation of estimation error are about 2.0 K and 2.3 K, considering the estimation error of land surface temperature and emissivity. Finally, the comparison of estimation results with ground measurement data at meteorological stations indicates that the RM‐NN can be used to estimate near surface air temperature from ASTER data.     

Annealing stochastic approximation Monte Carlo algorithm for neural network training

We propose a general-purpose stochastic optimization algorithm, the so-called annealing stochastic approximation Monte Carlo (ASAMC) algorithm, for neural network training. ASAMC can be regarded as a space annealing version of the stochastic approximation Monte Carlo (SAMC) algorithm. Under mild conditions, we show that ASAMC can converge weakly at a rate of Ω toward a neighboring set (in the space of energy) of the global minimizers. ASAMC is compared with simulated annealing, SAMC, and the BFGS algorithm for training MLPs on a number of examples. The numerical results indicate that ASAMC outperforms the other algorithms in both training and test errors. Like other stochastic algorithms, ASAMC requires longer training time than do the gradient-based algorithms. It provides, however, an efficient approach to train MLPs for which the energy landscape is rugged. Action Editor: Risto Miikkulainen.     

A solving method based on neural network for a class of multi-leader–follower games

In this paper, we present a new solving approach for a class of multi-leader–follower games. For the problem studied, we firstly propose a neural network model. Then, based on Lyapunov and LaSalle theories, we prove that the trajectory of the neural network model can converge to the equilibrium point, which corresponds to the Nash equilibrium of the problem studied. The numerical results show that the proposed neural network approach is feasible to the problem studied.

     

Channel estimation based on neural network in space time block coded MIMO–OFDM system

In this study, we propose feed-forward multilayered perceptron (MLP) neural network trained with the Levenberg–Marquardt algorithm to estimate channel parameters in MIMO–OFDM systems. Bit error rate (BER) and mean square error (MSE) performances of least square (LS) and least mean square error (LMS) algorithms are also compared to our proposed neural network to evaluate the performances. Neural network channel estimator has got much better performance than LS and LMS algorithms. Furthermore it doesn?t need channel statistics and sending pilot tones, contrary to classical algorithms.     

Implementation of hybrid ANN–PSO algorithm on FPGA for harmonic estimation

Harmonic estimation is the main process in active filters for harmonic reduction. A hybrid Adaptive Neural Network–Particle Swarm Optimization (ANN–PSO) algorithm is being proposed for harmonic isolation. Originally Fourier Transformation is used to analyze a distorted wave. In order to improve the convergence rate and processing speed an Adaptive Neural Network Algorithm called Adaline has then been used. A further improvement has been provided to reduce the error and increase the fineness of harmonic isolation by combining PSO algorithm with Adaline algorithm. The inertia weight factor of PSO is combined along with the weight factor of Adaline and trained in Neural Network environment for better results. ANN–PSO provides uniform convergence with the convergence rate comparable that of Adaline algorithm. The proposed ANN–PSO algorithm is implemented on an FPGA. To validate the performance of ANN–PSO; results are compared with Adaline algorithm and presented herein.     

A fast learning algorithm of neural network with tunable activation function

This paper presents a modified structure of a neural network with tunable activation function and provides a new learning algorithm for the neural network training. Simulation results of XOR problem, Feigenbaum function, and Henon map show that the new algorithm has better performance than BP (back propagation) algorithm in terms of shorter convergence time and higher convergence accuracy. Further modifications of the structure of the neural network with the faster learning algorithm demonstrate simpler structure with even faster convergence speed and better convergence accuracy.     

Algebraic conditions of stability for Hopfield neural network

Using the relationship between the resistance, capacitance and current in Hopfield neural network, and the properties of sigmoid function, this paper gives the terse, explicit algebraical criteria of global exponential stability, global asymptotical stability and instability. Then this paper makes clear the essence of the stability that Hopfield defined, and provides a theoretical foundation for the design of a network.     

An efficient hybrid learning algorithm for neural network–based speech recognition systems on FPGA chip

This paper implemented an artificial neural network (ANN) on a field programmable gate array (FPGA) chip for Mandarin speech measurement and recognition of nonspecific speaker. A three-layer hybrid learning algorithm (HLA), which combines genetic algorithm (GA) and steepest descent method, was proposed to fulfill a faster global search of optimal weights in ANN. Some other popular evolutionary algorithms, such as differential evolution, particle swarm optimization and improve GA, were compared to the proposed HLA. It can be seen that the proposed HLA algorithm outperforms the other algorithms. Finally, the designed system was implemented on an FPGA chip with an SOC architecture to measure and recognize the speech signals.     

FPGA-realization of a self-tuning PID controller for X–Y table with RBF neural network identification

Microsystem Technologies - Based on field programmable gate array (FPGA) technology, a realization of a servo/motion control system with the self-tuning PID controller for X–Y table is...     

A computationally efficient algorithm for joint range–DOA–frequency estimation of near-field sources

Near-field source localization using passive sensor arrays plays an important role in array signal processing areas. Although many algorithms have been developed to deal with this issue, most of them suffer from either parameter match or heavy loss of the aperture or high computational complexity problems. To overcome these problems, a new algorithm is proposed in this paper to jointly estimate the ranges, directions-of-arrival (DOAs), and frequencies of multiple near-field narrow-band sources. Simulation results verify that the proposed algorithm can resolve these problems and give much better performance.     

Artificial neural networks–based backcalculation of the structural properties of a typical flexible pavement

Pavement evaluation is one of the foremost phases in all pavement engineering activities. In the backcalculation process, the researcher or the engineer varies the structural properties of the layers until the theoretical (calculated) deflections and the obtained (measured) deflections from FWD tests are closely matched to each other within a tolerable limit. However, this process is substantially time-consuming and poses some difficulties due to inherent inaccuracies in the results. In this study, synthetically derived deflections from a typical flexible pavement are used to estimate asphaltic concrete layer’s elastic modulus, Poisson’s ratio and thickness. Furthermore, artificial neural network (ANN) is utilized to determine the structural parameters, and it can be clearly seen that satisfactory results are obtained. ANN estimation of the three pavement layer characteristic parameters, that is, layer elastic modulus, Poisson’s ratio and layer thickness, was carried out for the first time in the study.     

New criteria for globally exponential stability of delayed Cohen–Grossberg neural network

This paper is concerned with analysis problem for the global exponential stability of the Cohen–Grossberg neural networks with discrete delays and with distributed delays. We first prove the existence and uniqueness of the equilibrium point under mild conditions, assuming neither differentiability nor strict monotonicity for the activation function. Then, we employ Lyapunov functions to establish some sufficient conditions ensuring global exponential stability of equilibria for the Cohen–Grossberg neural networks with discrete delays and with distributed delays. Our results are not only presented in terms of system parameters and can be easily verified and also less restrictive than previously known criteria. A comparison between our results and the previous results admits that our results establish a new set of stability criteria for delayed neural networks.     

Control of robotic manipulators—a neural network approach

A neural-network-based scheme is used for the control of a robotic manipulator. The main idea is that, by using a neural network to learn the characteristics of the robot system (or specifically its inverse dynamics), accurate trajectory following and good performance results are obtained. However, the traditional back-propagation algorithm commonly used for control and identification of nonlinear systems suffers from a slow rate of convergence. We investigate the effect of adusting the slope of the activation function (the node nonlinearity) on the performance of a back-propagation algorithm. It is shown that learning speed is increased significantly by making the slope of non-linearity adaptive. The results demonstrate that the proposed method gives better error minimization and faster convergence. The suggested method is applied to a two-link robotic manipulator. The resulting controller is sufficiently robust with respect to the changing conditions.     

A Bradley–Terry artificial neural network model for individual ratings in group competitions

A common statistical model for paired comparisons is the Bradley–Terry model. This research re-parameterizes the Bradley–Terry model as a single-layer artificial neural network (ANN) and shows how it can be fitted using the delta rule. The ANN model is appealing because it makes using and extending the Bradley–Terry model accessible to a broader community. It also leads to natural incremental and iterative updating methods. Several extensions are presented that allow the ANN model to learn to predict the outcome of complex, uneven two-team group competitions by rating individuals—no other published model currently does this. An incremental-learning Bradley–Terry ANN yields a probability estimate within less than 5% of the actual value training over 3,379 multi-player online matches of a popular team- and objective-based first-person shooter.     

Two-level evolutionary algorithm for discovering relations between nodes’ features in a complex network

Complex network theory offers an efficient mathematical framework for modelling natural phenomena. However, these studies focus mainly on the topological characteristics of networks, while the actual reasons behind the networks’ formation remain overlooked. This paper proposes a new approach to complex network analysis. By searching for the optimal functional definition of the network's edge set, it allows an examination of the influences of the physical properties of the nodes on the network's structure and behaviour (i.e. changes of the network's structure when the physical properties of nodes change). A two-level evolutionary algorithm is proposed for this purpose, whereby the search for a suitable function form is achieved at the first level, while the second level is used for optimal function fitting. In this way, not only the features with the largest influences are identified, but also the intensities of their influences are estimated. Synthetic networks are examined in order to show the superiority of the proposed approach over traditional machine learning algorithms, while the applicability of the proposed method is demonstrated on a real-world study of the behaviour of biological cells.     

Integration of top-down and bottom-up visual processing using a recurrent convolutional–deconvolutional neural network for semantic segmentation

Semantic segmentation has a wide array of applications such as scene understanding, autonomous driving, and robot manipulation tasks. While existing segmentation models have achieved good performance using bottom-up deep neural processing, this paper describes a novel deep learning architecture that integrates top-down and bottom-up processing. The resulting model achieves higher accuracy at a relatively low computational cost. In the proposed model, higher-level top-down information is transmitted to the lower layers through recurrent connections in an encoder and a decoder, and the recurrent connection weights are trained using backpropagation. Experiments on several benchmark datasets demonstrate that this use of top-down information improves the mean intersection over union by more than 3% compared with a state-of-the-art bottom-up only network using the CamVid, SUN-RGBD and PASCAL VOC 2012 benchmark datasets. Additionally, the proposed model is successfully applied to a dataset designed for robotic grasping tasks.