Observations on using genetic algorithms for dynamic load-balancing

Load-balancing problems arise in many applications, but, most importantly, they play a special role in the operation of parallel and distributed computing systems. Load-balancing deals with partitioning a program into smaller tasks that can be executed concurrently and mapping each of these tasks to a computational resource such as a processor (e.g., in a multiprocessor system) or a computer (e.g., in a computer network). By developing strategies that can map these tasks to processors in a way that balances out the load, the total processing time will be reduced with improved processor utilization. Most of the research on load-balancing focused on static scenarios that, in most of the cases, employ heuristic methods. However, genetic algorithms have gained immense popularity over the last few years as a robust and easily adaptable search technique. The work proposed here investigates how a genetic algorithm can be employed to solve the dynamic load-balancing problem. A dynamic load-balancing algorithm is developed whereby optimal or near-optimal task allocations can “evolve” during the operation of the parallel computing system. The algorithm considers other load-balancing issues such as threshold policies, information exchange criteria, and interprocessor communication. The effects of these and other issues on the success of the genetic-based load-balancing algorithm as compared with the first-fit heuristic are outlined     

改进的遗传算法对神经网络优化的分类

针对数据挖掘中的分类问题,本文应用基于基因重组的遗传算法优化神经网络,并训练权值来实现分类.实验结果表明改进的遗传算法有效地提高了神经网络数据分类的准确性.     

用改进的遗传算法训练神经网络构造分类器

针对基本遗传算法存在容易早熟和局部搜索能力弱等缺陷,提出了改进的遗传算法,引入交叉概率和变异概率与个体的适度值相联系,改进了操作算子,而且在交叉操作后又引入模拟退火机制,提高遗传算法的局部搜索能力。同时,用改进的遗传算法和基本的遗传算法训练神经网络构造分类器,实验结果表明,改进的遗传算法在最好个体适度值和最好分类准确性等方面性能更好。     

Fuzzy identification using fuzzy neural networks with stable learning algorithms

In general, fuzzy neural networks cannot match nonlinear systems exactly. Unmodeled dynamic leads parameters drift and even instability problem. According to system identification theory, robust modification terms must be included in order to guarantee Lyapunov stability. This paper suggests new learning laws for Mamdani and Takagi-Sugeno-Kang type fuzzy neural networks based on input-to-state stability approach. The new learning schemes employ a time-varying learning rate that is determined from input-output data and model structure. Stable learning algorithms for the premise and the consequence parts of fuzzy rules are proposed. The calculation of the learning rate does not need any prior information such as estimation of the modeling error bounds. This offer an advantage compared to other techniques using robust modification.     

Research on using genetic algorithms to optimize Elman neural networks

There is a function of dynamic mapping when processing non-linear complex data with Elman neural networks. Because Elman neural network inherits the feature of back-propagation neural network to some extent, it has many defects; for example, it is easy to fall into local minimum, the fixed learning rate, the uncertain number of hidden layer neuron and so on. It affects the processing accuracy. So we optimize the weights, thresholds and numbers of hidden layer neurons of Elman networks by genetic algorithm. It improves training speed and generalization ability of Elman neural networks to get the optimal algorithm model. It has been proved by instance analysis that new algorithm was superior to the traditional model in terms of convergence rate, predicted value error, number of trainings conducted successfully, etc. It indicates the effect of the new algorithm and deserves further popularization.     

Ten steps modeling of electrolysis processes by using neural networks

Neural networks have been developed to model the electrolysis of wastes polluted with phenolic compounds, including phenol, 4-chlorophenol, 2,4-dichlorophenol, 2,4,6-trichlorophenol, 4-nitrophenol and 2,4-dinitrophenol. They enable the prediction the Chemical Oxygen Demand of a treated waste as a function of the initial characteristics (pollutant concentration, pH), operation conditions (temperature, current density) and current charge passed. A consistent set of experimental data was obtained by electrochemical oxidation with conductive diamond electrodes, used to treat synthetic aqueous wastes.Several modeling strategies based on simple and stacked neural networks, with different transfer functions into the hidden and output layers, have been considered to obtain a good accuracy of the model. Global errors during the training stage were under 3% and those of the validation stage were under 4%, demonstrating that the neural network based technique is appropriate for modeling the system.The generalization capability of the neural networks was also tested in realistic conditions where Chemical Oxygen Demand was predicted with errors around 5%. Therefore, the developed neural models can be used in industry to determine the required treatment period, to obtain the discharge limits in batch electrolysis processes, and it is a first step in the development of process control strategies.The ten step methodology was applied to the neural network based process modeling.     

On fuzzy modeling using fuzzy neural networks with theback-propagation algorithm

A fuzzy modeling method using fuzzy neural networks with the backpropagation algorithm is presented. The method can identify the fuzzy model of a nonlinear system automatically. The feasibility of the method is examined using simple numerical data.     

Nearest-neighbor algorithms for load-balancing in parallel computers

With nearest-neighbor load-balancing algorithms, a processor makes balancing decisions based on localized workload information and manages workload migrations within its neighborhood. The paper compares a couple of fairly well-known nearest-neighbor algorithms, the dimension-exchange (DE) and the diffusion (DF) methods and their several variants—the average dimension-exchange (ADE), optimally tuned dimension-exchange (ODE), local average diffusion (ADF) and optimally tuned diffusion (ODF). The measures of interest are their efficiency in driving any initial workload distribution to a uniform distribution and their ability in controlling the growth of the variance among the processors' workloads. The comparison is made with respect to both one-port and all-port communication architectures and in consideration of various implementation strategies including synchronous/asynchronous invocation policies and static/dynamic random workload behaviors. It turns out that the dimension-exchange method outperforms the diffusion method in the one-port communication model. In particular, the ODE algorithm is best suited for statically synchronous implementations of a load-balancing process regardless of its underlying communication models. The strength of the diffusion method is in asynchronous implementations in the all-port communication model; the ODF algorithm performs best in that case. The underlying communication networks considered assume the most popular topologies, the mesh and the torus and their special cases: the hypercube and the k-ary n-cube.     

A hybrid approach to vehicle routing using neural networks and genetic algorithms

A competitive neural network model and a genetic algorithm are used to improve the initialization and construction phase of a parallel insertion heuristic for the vehicle routing problem with time windows. The neural network identifies seed customers that are distributed over the entire geographic area during the initialization phase, while the genetic algorithm finds good parameter settings in the route construction phase that follows. Computational results on a standard set of problems are also reported.     

Diffusion learning algorithms for feedforward neural networks

The problem of training feedforward neural networks is considered. To solve it, new algorithms are proposed. They are based on the asymptotic analysis of the extended Kalman filter (EKF) and on a separable network structure. Linear weights are interpreted as diffusion random variables with zero expectation and a covariance matrix proportional to an arbitrarily large parameter λ. Asymptotic expressions for the EKF are derived as λ→∞. They are called diffusion learning algorithms (DLAs). It is shown that they are robust with respect to the accumulation of rounding errors in contrast to their prototype EKF with a large but finite λ and that, under certain simplifying assumptions, an extreme learning machine (ELM) algorithm can be obtained from a DLA. A numerical example shows that the accuracy of a DLA may be higher than that of an ELM algorithm.     

New recursive-least-squares algorithms for nonlinear active controlof sound and vibration using neural networks

In recent years, a few articles describing the use of neural networks for nonlinear active control of sound and vibration were published. Using a control structure with two multilayer feedforward neural networks (one as a nonlinear controller and one as a nonlinear plant model), steepest descent algorithms based on two distinct gradient approaches were introduced for the training of the controller network. The two gradient approaches were sometimes called the filtered-x approach and the adjoint approach. Some recursive-least-squares algorithms were also introduced, using the adjoint approach. In this paper, an heuristic procedure is introduced for the development of recursive-least-squares algorithms based on the filtered-x and the adjoint gradient approaches. This leads to the development of new recursive-least-squares algorithms for the training of the controller neural network in the two networks structure. These new algorithms produce a better convergence performance than previously published algorithms. Differences in the performance of algorithms using the filtered-x and the adjoint gradient approaches are discussed in the paper. The computational load of the algorithms discussed in the paper is evaluated for multichannel systems of nonlinear active control. Simulation results are presented to compare the convergence performance of the algorithms, showing the convergence gain provided by the new algorithms.     

Object-oriented modeling and simulation of neural networks

The field of neural networks is being investigated by many researchers in order to provide solutions to difficult problems in the area of manufacturing systems. Computer simulation of neural networks is an important part of this investigation. This paper applies concepts from an important trend in software engineering research, namely object-oriented programming, to model neural networks.The design and implementation of a software object library is crucial to obtaining the full benefits of object-oriented programming. In this paper we discuss the design and implementation of a foundation library of software objects for the purpose of simulating and validating different network architectures and learning rules. The library contains objects that implement various types of nodes and learning rules. We discuss the results of our experiments to illustrate the benefits of using an object-oriented approach to modeling neural networks.     

Adaptation algorithms for 2-D feedforward neural networks

The generalized weight adaptation algorithms presented by J.G. Kuschewski et al. (1993) and by S.H. Zak and H.J. Sira-Ramirez (1990) are extended for 2-D madaline and 2-D two-layer feedforward neural nets (FNNs).     

Local routing algorithms based on Potts neural networks

A feedback neural approach to static communication routing in asymmetric networks is presented, where a mean field formulation of the Bellman-Ford method for the single unicast problem is used as a common platform for developing algorithms for multiple unicast, multicast and multiple multicast problems. The appealing locality and update philosophy of the Bellman-Ford algorithm is inherited. For all problem types the objective is to minimize a total connection cost, defined as the sum of the individual costs of the involved arcs, subject to capacity constraints. The methods are evaluated for synthetic problem instances by comparing to exact solutions for cases where these are accessible, and else with approximate results from simple heuristics. In general, the quality of the results are better than those of the heuristics. Furthermore, the computational demands are modest, even when the distributed nature of the the approach is not exploited numerically.     

Sine-fitting multiharmonic algorithms implemented by artificial neural networks

A new method designed to perform high-accuracy spectral analysis, based on ADALINE artificial neural networks (ANNs), is proposed. The proposed network is able to accurately calculate the fundamental frequency and the harmonic content of an input signal. The method is especially useful in high-precision digital measurement systems in which periodical signals are involved, i.e. digital watt meters. Most of these systems use spectral analysis algorithms as an intermediate step for the computation of the magnitudes of interest. The traditional spectral analysis methods require synchronous sampling, which introduce limitations to the sampling circuitry. Sine-fitting multiharmonics algorithms resolve the hardware limitations concerning the synchronous sampling but have some limitations with regard to the phase of the array of samples. The new implementation of sine-fitting multiharmonics algorithms based on ANN eliminates these limitations.     

On-line learning algorithms for locally recurrent neural networks

This paper focuses on online learning procedures for locally recurrent neural nets with emphasis on multilayer perceptron (MLP) with infinite impulse response (IIR) synapses and its variations which include generalized output and activation feedback multilayer networks (MLN). We propose a new gradient-based procedure called recursive backpropagation (RBP) whose online version, causal recursive backpropagation (CRBP), has some advantages over other online methods. CRBP includes as particular cases backpropagation (BP), temporal BP, Back-Tsoi algorithm (1991) among others, thereby providing a unifying view on gradient calculation for recurrent nets with local feedback. The only learning method known for locally recurrent nets with no architectural restriction is the one by Back and Tsoi. The proposed algorithm has better stability and faster convergence with respect to the Back-Tsoi algorithm. The computational complexity of the CRBP is comparable with that of the Back-Tsoi algorithm, e.g., less that a factor of 1.5 for usual architectures and parameter settings. The superior performance of the new algorithm, however, easily justifies this small increase in computational burden. In addition, the general paradigms of truncated BPTT and RTRL are applied to networks with local feedback and compared with CRBP. CRBP exhibits similar performances and the detailed analysis of complexity reveals that CRBP is much simpler and easier to implement, e.g., CRBP is local in space and in time while RTRL is not local in space.     

Performance modeling of Cartesian product networks

This paper presents a comprehensive performance model for fully adaptive routing in wormhole-switched Cartesian product networks. Besides the generality of the model which makes it suitable to be used for any product graph, experimental (simulation) results show that the proposed model exhibits high accuracy even in heavy traffic and saturation region, where other models have severe problems to predict the performance of the network. Most popular interconnection network can be defined as a Cartesian product of two or more networks including the mesh, hypercube, and torus networks. Torus and mesh networks are the most popular topologies used in recent supercomputing parallel machines. They have been widely used for realizing on-chip network in recent on-chip multicore and multiprocessors system.     

Two-stage intonation modeling using feedforward neural networks for syllable based text-to-speech synthesis

This paper proposes a two-stage feedforward neural network (FFNN) based approach for modeling fundamental frequency (F₀) values of a sequence of syllables. In this study, (i) linguistic constraints represented by positional, contextual and phonological features, (ii) production constraints represented by articulatory features and (iii) linguistic relevance tilt parameters are proposed for predicting intonation patterns. In the first stage, tilt parameters are predicted using linguistic and production constraints. In the second stage, F₀ values of the syllables are predicted using the tilt parameters predicted from the first stage, and basic linguistic and production constraints. The prediction performance of the neural network models is evaluated using objective measures such as average prediction error (μ), standard deviation (σ) and linear correlation coefficient (γ_X,Y). The prediction accuracy of the proposed two-stage FFNN model is compared with other statistical models such as Classification and Regression Tree (CART) and Linear Regression (LR) models. The prediction accuracy of the intonation models is also analyzed by conducting listening tests to evaluate the quality of synthesized speech obtained after incorporation of intonation models into the baseline system. From the evaluation, it is observed that prediction accuracy is better for two-stage FFNN models, compared to the other models.     

Performance analysis of modeling framework for prediction in wind farms employing artificial neural networks

This paper introduces the concept and practice of Neural Network architectures for wind speed prediction in wind farms. The wind speed prediction method has been analyzed by using back propagation network and radial basis function network. Artificial neural network is used to develop suitable architecture for predicting wind speed in wind farms. The key of wind speed prediction is rational selection of forecasting model and effective optimization of model performance. To verify the effectiveness of neural network architecture, simulations were conducted on real time wind data with different heights of wind mill. Due to fluctuation and nonlinearity of wind speed, accurate wind speed prediction plays a major role in the operational control of wind farms. The key advantages of Radial Basis Function Network include higher accuracy, reduction of training time and minimal error. The experimental results show that compared to existing approaches, proposed radial basis function network performs better in terms of minimization of errors.     

Parallel processing, neural networks and genetic algorithms

In an earlier paper[1] some recent developments in computational technology to structural engineering were described. The developments included: parallel and distributed computing; neural networks; and genetic algorithms. In this paper, the authors concentrate on parallel implementations of neural networks and genetic algorithms. In the final section of the paper the authors show how a parallel finite element analysis may be undertaken in an efficient manner by preprocessing of the finite element model using a genetic algorithm utilizing a neural network predictor. This preprocessing is the partitioning of the finite element mesh into sub-domains to ensure load balancing and minimum interprocessor communication during the parallel finite element analysis on a MIMD distributed memory computer. © 1998 Published by Elsevier Science Limited. All rights reserved.