Intelligent data structures selection using neural networks

It is well known that abstract data types represent the core for any software application, and a proper use of them is an essential requirement for developing a robust and efficient system. Data structures are essential in obtaining efficient algorithms, having a major importance in the software development process. Selecting and creating the appropriate data structure for implementing an abstract data type can greatly impact the performance and the efficiency of the software systems. It is not a trivial problem for a software developer, as it is hard to anticipate all the use scenarios of the deployed application, and a static selection before the system’s execution is, generally, not accurate. In this paper, we are focusing on the problem of dynamic selection of efficient data structures for abstract data types implementation using a supervised learning approach. In order to dynamically select the most suitable representation for an aggregate according to the software system’s current execution context, a neural network will be used. We experimentally evaluate the proposed technique on a case study, emphasizing the advantages of the proposed model in comparison with existing similar approaches.     

Learning chaotic attractors by neural networks

An algorithm is introduced that trains a neural network to identify chaotic dynamics from a single measured time series. During training, the algorithm learns to short-term predict the time series. At the same time a criterion, developed by Diks, van Zwet, Takens, and de Goede (1996) is monitored that tests the hypothesis that the reconstructed attractors of model-generated and measured data are the same. Training is stopped when the prediction error is low and the model passes this test. Two other features of the algorithm are (1) the way the state of the system, consisting of delays from the time series, has its dimension reduced by weighted principal component analysis data reduction, and (2) the user-adjustable prediction horizon obtained by "error propagation"-partially propagating prediction errors to the next time step. The algorithm is first applied to data from an experimental-driven chaotic pendulum, of which two of the three state variables are known. This is a comprehensive example that shows how well the Diks test can distinguish between slightly different attractors. Second, the algorithm is applied to the same problem, but now one of the two known state variables is ignored. Finally, we present a model for the laser data from the Santa Fe time-series competition (set A). It is the first model for these data that is not only useful for short-term predictions but also generates time series with similar chaotic characteristics as the measured data.     

Chaotic neural networks and chaotic annealing

A simple model of single neuron with chaotic dynamics is proposed. Neural networks coupled by such neurons have the property of temporal retrieval of stored patterns in a chaotic way. The network is also studied from the viewpoint of optimization. A chaotic annealing technique is developed to search for the global minima of the energy with transient chaos.     

Memristor-based chaotic neural networks for associative memory

In chaotic neural networks, the rich dynamic behaviors are generated from the contributions of spatio-temporal summation, continuous output function, and refractoriness. However, a large number of spatio-temporal summations in turn make the physical implementation of a chaotic neural network impractical. This paper proposes and investigates a memristor-based chaotic neural network model, which adequately utilizes the memristor with unique memory ability to realize the spatio-temporal summations in a simple way. Furthermore, the associative memory capabilities of the proposed memristor-based chaotic neural network have been demonstrated by conventional methods, including separation of superimposed pattern, many-to-many associations, and successive learning. Thanks to the nanometer scale size and automatic memory ability of the memristors, the proposed scheme is expected to greatly simplify the structure of chaotic neural network and promote the hardware implementation of chaotic neural networks.     

Intelligent sensor system for discrimination of material type using neural networks

Neural network based classification of material type even with the variation in the sensor parameter is investigated in this paper. The sensor is developed by means of a lightweight plunger probe and an optical mouse sensor. An experimental prototype was developed which involves bouncing or hopping of the plunger based impact probe freely on the plain surface of an object under test. The experiment is conducted to obtain the bouncing signals for plain surface of an objects kept at different distances from the probe. During the bouncing of the probe, time varying signals are generated from optical mouse that are recorded in data files on PC. Some dominant unique features are then extracted using signal processing tools to optimize neural network based classifier. The time and features of bouncing signal are related to the material type, and each material has a unique set of such properties. It is found that the sensor system is intelligent due to its ability to classify the material type even with the variation in the sensor parameter (distance between the sensor probe and plain objects). The classifiers are developed using two neural networks configurations, namely a well-known Multi-layer Perceptron Neural Networks (MLP NN), and Radial Basis Function Neural Networks (RBF NN). MLP NN and RBF NN models are designed to maximize accuracy under the constraints of minimum network dimension.The optimal parameters of MLP NN and RBF NN models based on various performance measures that include percentage classification accuracy (PCLA) on the testing data, and area under Receiver Operating Characteristics (ROC), and are determined. For the sensor data set, the PCLA of both the classifiers are found reasonable consistently in respect of rigorous testing using different data partitions. The areas under the ROC curves are close to unity. Performances of the two classifiers have been compared. It has been found that the RBF NN is more robust to noise, and epochs required for training are very less as compared to that for MLP NN.     

Anti-synchronization of stochastic perturbed delayed chaotic neural networks

This paper studies the anti-synchronization of a class of stochastic perturbed chaotic delayed neural networks. By employing the Lyapunov functional method combined with the stochastic analysis as well as the feedback control technique, several sufficient conditions are established that guarantee the mean square exponential anti-synchronization of two identical delayed networks with stochastic disturbances. These sufficient conditions, which are expressed in terms of linear matrix inequalities (LMIs), can be solved efficiently by the LMI toolbox in Matlab. Two numerical examples are exploited to demonstrate the feasibility and applicability of the proposed synchronization approaches.     

Global impulsive exponential anti-synchronization of delayed chaotic neural networks

In this paper, the impulsive exponential anti-synchronization for chaotic delayed neural networks is investigated. By establishing an integral delay inequality and using the inequality method, some sufficient conditions ensuring impulsive exponential anti-synchronization of two chaotic delayed networks are derived. To illustrate the effectiveness of the new scheme, a numerical example is given.     

A method for analyzing the spatiotemporal changes of chaotic neural networks

A chaotic neural network proposed (CNN) by Aihara et al. is able to recollect stored patterns dynamically. But there are difficult cases such as its long time processing of association, and difficult to recall a specific stored pattern during the dynamical associations. We have proposed to find the optimal parameters using meta-heuristics methods to improve association performance, for example, the shorter recalling time and higher recollection rates of stored patterns in our previous works. However, the relationship between the different values of parameters of chaotic neurons and the association performance of CNN was not investigated clearly. In this paper, we propose a method to analyze the spatiotemporal changes of internal states in CNN and, by the method, analyze how the change of values of internal parameters of chaotic neurons affects the characteristics of chaotic neurons when multiple patterns are stored in the CNN. Quantile–Quantile plot, least square approximation, hierarchical clustering, and Hilbert transform are used to investigate the similarity of internal states of chaotic neurons, and to classify the neurons. Simulation results showed that how different values of an internal parameter yielded different behaviors of chaotic neurons and it suggests the optimal parameter which generates higher association performance may concern with the stored patterns of the CNN.     

On the exponential synchronization of stochastic impulsive chaotic delayed neural networks

In this paper, the exponential synchronization of stochastic impulsive chaotic delayed neural networks is investigated. Based on the Lyapunov function method, time-varying delay feedback control technique and the efficient modified Halanay inequality for stochastic differential equations, several sufficient conditions are presented to guarantee the exponential synchronization in mean square between two identical chaotic delayed neural networks with stochastic and impulsive perturbations. These conditions are expressed in terms of linear matrix inequalities (LMIs), which can easily be checked by utilizing the numerically efficient Matlab LMI toolbox. Comparing with the existing works that consider single perturbation (stochastic or impulsive one), the proposed method can provide a more general framework for the synchronization of multi-perturbation chaotic systems, which is favorable for practical application in secure communication. Finally, numerical simulations verify the effectiveness of the proposed method.     

Neuron-synapse IC chip-set for large-scale chaotic neural networks.

We propose a neuron-synapse integrated circuit (IC) chip-set for large-scale chaotic neural networks. We use switched-capacitor (SC) circuit techniques to implement a three-internal-state transiently-chaotic neural network model. The SC chaotic neuron chip faithfully reproduces complex chaotic dynamics in real numbers through continuous state variables of the analog circuitry. We can digitally control most of the model parameters by means of programmable capacitive arrays embedded in the SC chaotic neuron chip. Since the output of the neuron is transfered into a digital pulse according to the all-or-nothing property of an axon, we design a synapse chip with digital circuits. We propose a memory-based synapse circuit architecture to achieve a rapid calculation of a vast number of weighted summations. Both of the SC neuron and the digital synapse circuits have been fabricated as IC forms. We have tested these IC chips extensively, and confirmed the functions and performance of the chip-set. The proposed neuron-synapse IC chip-set makes it possible to construct a scalable and reconfigurable large-scale chaotic neural network with 10000 neurons and 10000/sup 2/ synaptic connections.     

Range estimation of construction costs using neural networks with bootstrap prediction intervals

Modeling of construction costs is a challenging task, as it requires representation of complex relations between factors and project costs with sparse and noisy data. In this paper, neural networks with bootstrap prediction intervals are presented for range estimation of construction costs. In the integrated approach, neural networks are used for modeling the mapping function between the factors and costs, and bootstrap method is used to quantify the level of variability included in the estimated costs. The integrated method is applied to range estimation of building projects. Two techniques; elimination of the input variables, and Bayesian regularization were implemented to improve generalization capabilities of the neural network models. The proposed modeling approach enables identification of parsimonious mapping function between the factors and cost and, provides a tool to quantify the prediction variability of the neural network models. Hence, the integrated approach presents a robust and pragmatic alternative for conceptual estimation of costs.     

Dynamic characteristic of a multiple chaotic neural network and its application

Based on chaotic neural network, a multiple chaotic neural network algorithm combining two different chaotic dynamics sources in each neuron is proposed. With the effect of self-feedback connection and non-linear delay connection weight, the new algorithm can contain more powerful chaotic dynamics to search the solution domain globally in the beginning searching period. By analyzing the dynamic characteristic and the influence of cooling schedule in simulated annealing, a flexible parameter tuning strategy being able to promote chaotic dynamics convergence quickly is introduced into our algorithm. We show the effectiveness of the new algorithm in two difficult combinatorial optimization problems, i.e., a traveling salesman problem and a maximum clique problem.     

Intelligent tutoring systems: Formalization as automata and interface design using neural networks

This article proposes a mathematical model of Intelligent Tutoring Systems (ITS), based on observations of the behaviour of these systems. One of the most important problems of pedagogical software is to establish a common language between the knowledge areas involved in their development, basically pedagogical, computing and domain areas. A mathematical model, like the one proposed here, can facilitate the integration of these different areas, as it defines the elements that constitute the system and defines the technological tools to implement it. The article presents an example demonstrating how the formalization was used to design the adaptive mechanism of an ITS to adapt its Interface Module to some student characteristics.     

On sequential construction of binary neural networks

A new technique called sequential window learning (SWL), for the construction of two-layer perceptrons with binary inputs is presented. It generates the number of hidden neurons together with the correct values for the weights, starting from any binary training set. The introduction of a new type of neuron, having a window-shaped activation function, considerably increases the convergence speed and the compactness of resulting networks. Furthermore, a preprocessing technique, called hamming clustering (HC), is proposed for improving the generalization ability of constructive algorithms for binary feedforward neural networks. Its insertion in the sequential window learning is straightforward. Tests on classical benchmarks show the good performances of the proposed techniques, both in terms of network complexity and recognition accuracy.     

Hermite混沌神经网络异步加密算法

基于最佳均方逼近,采用Hermite正交多项式做为神经网络隐层的激励函数,引入一种新型的Hermite神经网络模型．通过神经网络权值和混沌初值产生性能接近于理论值的混沌序列,从中提取与明文等长的序列进行排序,将排序结果对明文置换后即可得密文．加密与解密信息完全隐藏于神经网络产生的混沌序列中,与混沌初值无显式关系,且只需改变混沌初值,便可实现“一次一密”异步加密,其安全性取决于混沌序列的复杂性和无法预测性．理论分析和加密实例表明,该加密算法简单易行,克服了混沌同步加密的诸多缺陷,具有良好的安全性．     

Oscillatory and chaotic dynamics in neural networks under varyingoperating conditions

This paper studies the effects of a time-dependent operating environment on the dynamics of a neural network. In the previous paper Wang et al. (1990) studied an exactly solvable model of a higher order neural network. We identified a bifurcation parameter for the system, i.e., the rescaled noise level, which represents the combined effects of incomplete connectivity, interference among stored patterns, and additional stochastic noise. When this bifurcation parameter assumes different but static (time-independent) values, the network shows a spectrum of dynamics ranging from fixed points, to oscillations, to chaos. This paper shows that varying operating conditions described by the time-dependence of the rescaled noise level give rise to many more interesting dynamical behaviours, such as disappearances of fixed points and transitions between periodic oscillations and deterministic chaos. These results suggest that a varying environment, such as the one studied in the present model, may be used to facilitate memory retrieval if dynamic states are used for information storage in a neural network     

Guaranteed cost synchronization of chaotic cellular neural networks with time-varying delay

Synchronization of cellular neural networks with time-varying delay is discussed in this letter. Based on Razumikhin theorem, a guaranteed cost synchronous controller is given. Unlike Lyapunov-Krasovskii analysis process, there is no constraint on the change rate of time delay. The saturated terms emerging in the Razumikhin analysis are amplified by zoned discussion and maximax synthesis rather than by Lipschitz condition and vector inequality, which will bring more conservatism. Then the controller criterion is transformed from quadratic matrix inequality form into linear matrix inequality form, with the help of a sufficient and necessary transformation condition. The minimization of the guaranteed cost is studied, and a further criterion for getting the controller is presented. Finally, the guaranteed cost synchronous control and its corresponding minimization problem are illustrated with examples of chaotic time-varying delay cellular neural networks.     

Job-shop scheduling using neural networks

Job-shop scheduling cannot easily be analytically accomplished, so, it is done by computer simulation using heuristic priority rules. The SLACK rule for calculating the margins of jobs to their due-dates is effective in meeting the due-dates. However, the calculated margins are not precise because the actual margin is shortened due to conflicts with other jobs. The authors propose a method for estimating the margins by using a neural network. It is found that the method is effective for improving the average lateness to due-dates but not the maximum lateness. This paper proposes a method for adding a second neural network for judging the reliability of the estimated margins composed to the first one and for switching to the margins calculated by the SLACK rule when the reliability is low. The proposed method is verified by scheduling simulations to be effective in decreasing the maximum lateness to due-dates as much as the average lateness.     

Algebraic condition of synchronization for multiple time-delayed chaotic Hopfield neural networks

In this paper, an easy and efficient method is brought forward to design the feedback control for the synchronization of two multiple time-delayed chaotic Hopfield neural networks, whose activation functions and delayed activation functions can have different forms of mapping. Without many complex restrictions and Lyapunov analytic process, the feedback control is given based on the M-matrix theory, the system parameters and the feedback section coefficients. All the results are simulated by Matlab and Simulink, which shows the simplicity and validity of the control. As shown in the simulation results, the error systems converge to zero rapidly.     

Real-coded chaotic quantum-inspired genetic algorithm for training of fuzzy neural networks

In this paper, a novel approach to adjusting the weightings of fuzzy neural networks using a Real-coded Chaotic Quantum-inspired genetic Algorithm (RCQGA) is proposed. Fuzzy neural networks are traditionally trained by using gradient-based methods, which may fall into local minimum during the learning process. To overcome the problems encountered by the conventional learning methods, RCQGA algorithms are adopted because of their capabilities of directed random search for global optimization. It is well known, however, that the searching speed of the conventional quantum genetic algorithms (QGA) is not satisfactory. In this paper, a real-coded chaotic quantum-inspired genetic algorithm (RCQGA) is proposed based on the chaotic and coherent characters of Q-bits. In this algorithm, real chromosomes are inversely mapped to Q-bits in the solution space. Q-bits probability-guided real cross and chaos mutation are applied to the evolution and searching of real chromosomes. Chromosomes consisting of the weightings of the fuzzy neural network are coded as an adjustable vector with real number components that are searched by the RCQGA. Simulation results have shown that faster convergence of the evolution process in searching for an optimal fuzzy neural network can be achieved. Examples of nonlinear functions approximated by using the fuzzy neural network via the RCQGA are demonstrated to illustrate the effectiveness of the proposed method.