Bayesian selection probability estimation for probabilistic Boolean networks

A Bayesian approach to estimate selection probabilities of probabilistic Boolean networks is developed in this study. The concepts of inverse Boolean function and updatable set are introduced to specify states which can be used to update a Bayesian posterior distribution. The analysis on convergence of the posteriors is carried out by exploiting the combination of semi‐tensor product technique and state decomposition algorithm for Markov chain. Finally, some numerical examples demonstrate the proposed estimation algorithm.     

Discrete probability estimation for classification usingcertainty-factor-based neural networks

Traditional probability estimation often demands a large amount of data for a problem of industrial scale. Neural networks have been used as an effective alternative for estimating input-output probabilities. In this paper, the certainty-factor-based neural network (CFNet) is explored for probability estimation in discrete domains. A new analysis presented here shows that the basis functions learned by the CFNet can bear precise semantics for dependencies. In the simulation study, the CFNet outperforms both the backpropagation network and the system based on the Rademacher-Walsh expansion. In the real-data experiments on splice junction and breast cancer data sets, the CFNet outperforms other neural networks and symbolic systems.     

A general probability estimation approach for neural comp

We describe an analytical framework for the adaptations of neural systems that adapt its internal structure on the basis of subjective probabilities constructed by computation of randomly received input signals. A principled approach is provided with the key property that it defines a probability density model that allows studying the convergence of the adaptation process. In particular, the derived algorithm can be applied for approximation problems such as the estimation of probability densities or the recognition of regression functions. These approximation algorithms can be easily extended to higher-dimensional cases. Certain neural network models can be derived from our approach (e.g., topological feature maps and associative networks).     

A fast algorithm for nonparametric probability density estimation

A fast algorithm for the well-known Parzen window method to estimate density functions from the samples is described. The computational efforts required by the conventional and straightforward implementation of this estimation procedure limit its practical application to data of low dimensionality. The proposed algorithm makes the computation of the same density estimates with a substantial reduction of computer time possible, especially for data of high dimensionality. Some simulation experiments are presented which demonstrate the efficiency of the method. They indicate the computational savings that may be achieved through the use of this fast algorithm for artificially generated sets of data.     

A novel multilayer neural networks training algorithm thatminimizes the probability of classification error

A multilayer neural networks training algorithm that minimizes the probability of classification error is proposed. The claim is made that such an algorithm possesses some clear advantages over the standard backpropagation (BP) algorithm. The convergence analysis of the proposed procedure is performed and convergence of the sequence of criterion realizations with probability of one is proven. An experimental comparison with the BP algorithm on three artificial pattern recognition problems is given.     

Drought estimation with neural networks

An artificial neural network model is presented to derive streamflow precipitation data. It is tested with actual data coming from a nearby river, referred to a basin area of 356 km² and a time period of 11 years. A feedforward multilayer perception with linear output has been built to deal with this problem. The dynamics are caught by the filter structure of the input layer.

A special study on crossing properties, based on training sample selection,is made to measure the performance of the network for drought analysis. Sample selection leads to increased accuracy within the sample range and degraded performance for points that are clearly out. Predicted number of droughts, average drought length and deficit are compared with the actual data. The results show that very simple neural network models can give fine results.     

Self-organizing mixture networks for probability density estimation

A self-organizing mixture network (SOMN) is derived for learning arbitrary density functions. The network minimizes the Kullback-Leibler information metric by means of stochastic approximation methods. The density functions are modeled as mixtures of parametric distributions. A mixture needs not to be homogenous, i.e., it can have different density profiles. The first layer of the network is similar to Kohonen's self-organizing map (SOM), but with the parameters of the component densities as the learning weights. The winning mechanism is based on maximum posterior probability, and updating of the weights is limited to a small neighborhood around the winner. The second layer accumulates the responses of these local nodes, weighted by the learned mixing parameters. The network possesses a simple structure and computational form, yet yields fast and robust convergence. The network has a generalization ability due to the relative entropy criterion used. Applications to density profile estimation and pattern classification are presented. The SOMN can also provide an insight to the role of neighborhood function used in the SOM.     

一种概率模型无线传感器网络覆盖算法*

针对传统覆盖算法在求解覆盖度时计算量较大、算法复杂度过高,从而导致算法效率过低,提出一种基于概率模型的覆盖算法。首先该概率模型通过调度覆盖区域内的节点状态来实现对覆盖区域监测,保证了所关注目标节点被传感器节点均匀覆盖的同时又优化了网络资源;其次对不同的覆盖区域利用概率期望值及相应定理求出满足覆盖条件下最少传感器节点数。仿真实验结果表明,该算法在保证网络覆盖质量要求时能够有效地减少活跃节点的数量,延长了网络的生存时间。     

Bayesian nonlinear model selection and neural networks: a conjugateprior approach

In order to select the best predictive neural-network architecture in a set of several candidate networks, we propose a general Bayesian nonlinear regression model comparison procedure, based on the maximization of an expected utility criterion. This criterion selects the model under which the training set achieves the highest level of internal consistency, through the predictive probability distribution of each model. The density of this distribution is computed as the model posterior predictive density and is asymptotically approximated from the assumed Gaussian likelihood of the data set and the related conjugate prior density of the parameters. The use of such a conjugate prior allows the analytic calculation of the parameter posterior and predictive posterior densities, in an empirical Bayes-like approach. This Bayesian selection procedure allows us to compare general nonlinear regression models and in particular feedforward neural networks, in addition to embedded models as usual with asymptotic comparison tests.     

Globally optimal bounding ellipsoid algorithm for parameter estimation using artificial neural networks

This paper develops a real-time implementation of a globally optimal bounding ellipsoid (GOBE) algorithm for parameter estimation of linear-in-parameter models with unknown but bounded (UBB)errors. A recently proposed recursively optimal bounding ellipsoid (ROBE) algorithm is introduced, and a GOBE algorithm is derived through repeating this ROBE algorithm. An analogue artificial neural network (ANN) is provided to implement the GOBE algorithm in real time. Convergence analyses on the ROBE, the GOBE algorithms, and the analogue ANN implementation of the GOBE algorithm are presented. No persistent excitation condition is required to ensure the convergence. Simulation results show the good performances of these algorithms and the ANN implementation.     

Robust learning algorithm for multiplicative neuron model artificial neural networks

The two most commonly used types of artificial neural networks (ANNs) are the multilayer feed-forward and multiplicative neuron model ANNs. In the literature, although there is a robust learning algorithm for the former, there is no such algorithm for the latter. Because of its multiplicative structure, the performance of multiplicative neuron model ANNs is affected negatively when the dataset has outliers. On this issue, a robust learning algorithm for the multiplicative neuron model ANNs is proposed that uses Huber's loss function as fitness function. The training of the multiplicative neuron model is performed using particle swarm optimization. One principle advantage of this algorithm is that the parameter of the scale estimator, which is an important factor affecting the value of Huber's loss function, is also estimated with the proposed algorithm. To evaluate the performance of the proposed method, it is applied to two well-known real world time series datasets, and also a simulation study is performed. The algorithm has superior performance both when it is applied to real world time series datasets and the simulation study when compared with other ANNs reported in the literature. Another of its advantages is that, for datasets with outliers, the results are very close to the results obtained from the original datasets. In other words, we demonstrate that the algorithm is unaffected by outliers and has a robust structure.     

A neural networks algorithm for data path synthesis

This paper presents a deterministic parallel algorithm to solve the data path allocation problem in high-level synthesis. The algorithm is driven by a motion equation that determines the neurons firing conditions based on the modified Hopfield neural network model of computation. The method formulates the allocation problem using the clique partitioning problem, an NP-complete problem, and handles multicycle functional units as well as structural pipelining. The algorithm has a running time complexity of O(1) for a circuit with n operations and c shared resources. A sequential simulator was implemented on a Linux Pentium PC under X-Windows. Several benchmark examples have been implemented and favorable design comparisons to other synthesis systems are reported.     

Conditional probability density function estimation with sigmoidalneural networks

Previous developments in conditional density estimation have used neural nets to estimate statistics of the distribution or the marginal or joint distributions of the input-output variables. We modify the joint distribution estimating sigmoidal neural network to estimate the conditional distribution. Thus, the probability density of the output conditioned on the inputs is estimated using a neural network. We derive and implement the learning laws to train the network. We show that this network has computational advantages over a brute force ratio of joint and marginal distributions. We also compare its performance to a kernel conditional density estimator in a larger scale (higher dimensional) problem simulating more realistic conditions.     

A training algorithm for binary feedforward neural networks

The authors present a new training algorithm to be used on a four-layer perceptron-type feedforward neural network for the generation of binary-to-binary mappings. This algorithm is called the Boolean-like training algorithm (BLTA) and is derived from original principles of Boolean algebra followed by selected extensions. The algorithm can be implemented on analog hardware, using a four-layer binary feedforward neural network (BFNN). The BLTA does not constitute a traditional circuit building technique. Indeed, the rules which govern the BLTA allow for generalization of data in the face of incompletely specified Boolean functions. When compared with techniques which employ descent methods, training times are greatly reduced in the case of the BLTA. Also, when the BFNN is used in conjunction with A/D converters, the applicability of the present algorithm can be extended to accept real-valued inputs.     

A complex-valued RTRL algorithm for recurrent neural networks

A complex-valued real-time recurrent learning (CRTRL) algorithm for the class of nonlinear adaptive filters realized as fully connected recurrent neural networks is introduced. The proposed CRTRL is derived for a general complex activation function of a neuron, which makes it suitable for nonlinear adaptive filtering of complex-valued nonlinear and nonstationary signals and complex signals with strong component correlations. In addition, this algorithm is generic and represents a natural extension of the real-valued RTRL. Simulations on benchmark and real-world complex-valued signals support the approach.     

State estimation for delayed neural networks

In this letter, the state estimation problem is studied for neural networks with time-varying delays. The interconnection matrix and the activation functions are assumed to be norm-bounded. The problem addressed is to estimate the neuron states, through available output measurements, such that for all admissible time-delays, the dynamics of the estimation error is globally exponentially stable. An effective linear matrix inequality approach is developed to solve the neuron state estimation problem. In particular, we derive the conditions for the existence of the desired estimators for the delayed neural networks. We also parameterize the explicit expression of the set of desired estimators in terms of linear matrix inequalities (LMIs). Finally, it is shown that the main results can be easily extended to cope with the traditional stability analysis problem for delayed neural networks. Numerical examples are included to illustrate the applicability of the proposed design method.     

A posteriori minimax estimation with likelihood constraints

Consideration was given to the problem of guaranteed estimation of the parameters of an uncertain stochastic regression. The loss function is the conditional mean squared error relative to the available observations. The uncertainty set is a subset of the probabilistic distributions lumped on a certain compact with additional linear constraints generated by the likelihood function. Solution of this estimation problem comes to determining the saddle point defining both the minimax estimator and the set of the corresponding worst distributions. The saddle point is the solution of a simpler finite-dimensional dual optimization problem. A numerical algorithm to solve this problem was presented, and its precision was determined. Model examples demonstrated the impact of the additional likelihood constraints on the estimation performance.     

Sunflower biomass estimation using a scattering model and a neural network algorithm

Inversion of biomass for sunflower fields using radar backscattering data has been carried out with neural network algorithms. An electromagnetic model is used to generate the scattering coefficients for training and testing of the net. The model is validated with experimental data obtained from the Montespertoli test site during the Remote Sensing Campaign Mac-Europe 91. The inversion results show that the neural network is capable of performing the retrieval with good accuracy. By optimizing the structural complexity of the net, a better inversion result is obtained.     

Confidence estimation methods for neural networks: a practicalcomparison

Feedforward neural networks, particularly multilayer perceptrons, are widely used in regression and classification tasks. A reliable and practical measure of prediction confidence is essential. In this work three alternative approaches to prediction confidence estimation are presented and compared. The three methods are the maximum likelihood, approximate Bayesian, and the bootstrap technique. We consider prediction uncertainty owing to both data noise and model parameter misspecification. The methods are tested on a number of controlled artificial problems and a real, industrial regression application, the prediction of paper "curl". Confidence estimation performance is assessed by calculating the mean and standard deviation of the prediction interval coverage probability. We show that treating data noise variance as a function of the inputs is appropriate for the curl prediction task. Moreover, we show that the mean coverage probability can only gauge confidence estimation performance as an average over the input space, i.e., global performance and that the standard deviation of the coverage is unreliable as a measure of local performance. The approximate Bayesian approach is found to perform better in terms of global performance.     

State estimation for discrete-time neural networks with time-varying delay

This article deals with the problem of delay-dependent state estimation for discrete-time neural networks with time-varying delay. Our objective is to design a state estimator for the neuron states through available output measurements such that the error state system is guaranteed to be globally exponentially stable. Based on the linear matrix inequality approach, a delay-dependent condition is developed for the existence of the desired state estimator via a novel Lyapunov functional. The obtained condition has less conservativeness than the existing ones, which is demonstrated by a numerical example.