在非线性建模中神经网络模型的构建与选择

主要研究了如何将平时单独使用的数学方法和统计学方法根据它们各自的优点综合运用,以提高非线性建模过程中神经网络模型构建和选择的效率。所使用的统计学工具包括矩阵的条件数,假设检验,交叉验证。文中对每个方法进行综合分析,进而判断它们分别应用在神经网络模型构建与选择过程的哪个阶段是最有效的。在此基础上,提出了一个系统的神经网络模型的构建与选择程序,并最终通过仿真试验来说明这个程序的有效性。     

Dynamic recurrent neural networks: a dynamical analysis

In this paper, we explore the dynamical features of a neural network model which presents two types of adaptative parameters: the classical weights between the units and the time constants associated with each artificial neuron. The purpose of this study is to provide a strong theoretical basis for modeling and simulating dynamic recurrent neural networks. In order to achieve this, we study the effect of the statistical distribution of the weights and of the time constants on the network dynamics and we make a statistical analysis of the neural transformation. We examine the network power spectra (to draw some conclusions over the frequential behaviour of the network) and we compute the stability regions to explore the stability of the model. We show that the network is sensitive to the variations of the mean values of the weights and the time constants (because of the temporal aspects of the learned tasks). Nevertheless, our results highlight the improvements in the network dynamics due to the introduction of adaptative time constants and indicate that dynamic recurrent neural networks can bring new powerful features in the field of neural computing.     

B样条神经网络的算法设计及应用

B样条神经网络是一个三层前向神经网络,可广泛用于非线性系统的建模、控制和模式识别等领域。本文讨论了一维和多维B样条神经网络的结构和程序算法,用仿真实例分析了算法中参数的选取和算法的有效性,并举例说明其在动态非线性系统故障诊断中的应用。     

A neural network model for the wire bonding process

A crucial step in manufacturing microcircuits is the wire bonding process in which a very thin gold wire must be formed to connect two surfaces in the microcircuit. The quality of the wire bond can be measured by visual inspection and a pull test—both of which are high-reliability, high-cost approaches to statistical process control. Westinghouse wanted to develop a high-reliability, low-cost quality assurance system. In this paper, we report on a year-long study to construct a neural network model that is capable of predicting the quality of wire bonds. The results of our modeling efforts reveal that neural networks are useful tools for statistical process control problems.     

Intelligent decision-making for liver fibrosis stadialization based on tandem feature selection and evolutionary-driven neural network

Hepatic fibrosis represents the principal pointer to the development of liver diseases. The correct evaluation of its degree, based on both recent non-invasive procedures and machine learning models, is of current major concern. One of the latest medical imaging methodologies for assessing it is the Fibroscan, supported by biochemical and clinical examinations. Since the complex interaction between the Fibroscan stiffness indicator and the biochemical and clinical results is hard to be manually managed towards the liver fibrosis stadialization, well-performing machine learning algorithms have been proposed to support an automatic diagnosis. We propose in this paper a tandem feature selection mechanism and evolutionary-driven neural network as a computer-based support for liver fibrosis stadialization in chronic hepatitis C. A synergetic system, based on both specific statistical tools and the sensitivity analysis provided by neural networks is used for reducing the dimension of the database from twenty-five to just six attributes. An evolutionary-trained neural network is developed afterwards for the classification of the liver fibrosis stages. The tandem approach is direct and simple, resulting from embedding the feature selection system into the method structure, in order to dynamically concentrate the search only on the most relevant attributes. Experimental results and a thorough statistical analysis clearly demonstrated the efficiency of the proposed intelligent system in comparison with other machine learning techniques reported in literature.     

A hybrid expert system for finite element modeling of fuselage frames

A hybrid expert system which integrates expert system with neural networks is developed for finite element modeling of fuselage frame of aircraft structure. Importance order parameters are introduced to quantify the modeling control. Expert knowledge of importance order parameters, node setting and element selection in fuselage frame modeling is presented. A neural network is employed to structure type classification. The modeling procedures of fuselage frame can be carried out automatically and efficiently by this system. Example shows the frame 1022 of MD-82 passenger aircraft which is modeled by this system automatically and successfully.     

Stochastic Petri net modeling of wave sequences in cardiac arrhythmias

We describe a methodology for modeling heart rhythms observed in electrocardiograms. In particular, we present a procedure to derive simple dynamic models that capture the cardiac mechanisms which control the particular timing sequences of P and R waves characteristic of different arrhythmias. By treating the cardiac electrophysiology at an aggregate level, simple network models of the wave generating system under a variety of diseased conditions can be developed. These network models are then systematically converted to stochastic Petri nets which offer a compact mathematical framework to express the dynamics and statistical variability of the wave generating mechanisms. Models of several arrhythmias are included in order to illustrate the methodology.     

Neural Network Modeling of the Level of Observation Decision in an Acute Psychiatric Ward

Patients in an acute psychiatric ward need to be observed with varying levels of closeness. We report a series of experiments in which neural networks were trained to model this “level of observation” decision. One hundred eighty-seven such clinical decisions were used to train and test the networks which were evaluated by a multitrialv-fold cross-validation procedure. One neural network modeling approach was to break down the decision process into four subproblems, each of which was solved by a perceptron unit. This resulted in a hierarchical perceptron network having a structure that was equivalent to a sparsely connected two-layer perceptron. Neural network approaches were compared with nearest neighbor, linear regression, and naive Bayes classifiers. The hierarchical and sparse neural networks were the most accurate classifiers. This shows that the decision process is nonlinear, that neural nets can be more accurate than other statistical approaches, and that hierarchical decomposition is a useful methodology for neural network design.     

基于神经网络的统计机器翻译的预调序模型

长距离调序是统计机器翻译的一个主要挑战。之前的研究工作表明预调序是解决这个问题的一个可能的途径。在该工作中,我们沿着预调序这个研究方向,将神经网络建模结合到线性排序的框架之下,提出了一个基于神经网络的预调序模型。这个的预调序模型能够利用从海量未标注数据中抽取的句法和语意信息,从而更好的对不同语言之间的语序差异进行预测。我们在中文到英文以及日文到英文的机器翻译任务上进行了实验,实验结果表明了该方法的有效性。
     

输入层自构造神经网络及其在光谱分析中的应用

为了解决红外光谱定量分析中的特征提取和校正规模问题,提出了一种输入层自构造神经网络。这种网络能够利用训练数据的某些先验知识,自动选择输入层神经元的个数。在学习过程中,输入神经元个数从最小值1开始,根据网络误差的变化逐步增加,最终确定最佳神经元数量。这种网络模型将特征提取和参数学习过程融为一体,有利于提高建模效率。利用仿真红外光谱的定量分析实验表明,这种网络模型不仅能够对光谱数据实现高效率的波长选择,并具有抑制随机噪声和非线性干扰的能力。     

On the complexity of computing and learning with multiplicative neural networks.

In a great variety of neuron models, neural inputs are combined using the summing operation. We introduce the concept of multiplicative neural networks that contain units that multiply their inputs instead of summing them and thus allow inputs to interact nonlinearly. The class of multiplicative neural networks comprises such widely known and well-studied network types as higher-order networks and product unit networks. We investigate the complexity of computing and learning for multiplicative neural networks. In particular, we derive upper and lower bounds on the Vapnik-Chervonenkis (VC) dimension and the pseudo-dimension for various types of networks with multiplicative units. As the most general case, we consider feedforward networks consisting of product and sigmoidal units, showing that their pseudo-dimension is bounded from above by a polynomial with the same order of magnitude as the currently best-known bound for purely sigmoidal networks. Moreover, we show that this bound holds even when the unit type, product or sigmoidal, may be learned. Crucial for these results are calculations of solution set components bounds for new network classes. As to lower bounds, we construct product unit networks of fixed depth with super-linear VC dimension. For sigmoidal networks of higher order, we establish polynomial bounds that, in contrast to previous results, do not involve any restriction of the network order. We further consider various classes of higher-order units, also known as sigma-pi units, that are characterized by connectivity constraints. In terms of these, we derive some asymptotically tight bounds. Multiplication plays an important role in both neural modeling of biological behavior and computing and learning with artificial neural networks. We briefly survey research in biology and in applications where multiplication is considered an essential computational element. The results we present here provide new tools for assessing the impact of multiplication on the computational power and the learning capabilities of neural networks.     

Adaptive importance sampling to accelerate training of a neural probabilistic language model.

Previous work on statistical language modeling has shown that it is possible to train a feedforward neural network to approximate probabilities over sequences of words, resulting in significant error reduction when compared to standard baseline models based on n-grams. However, training the neural network model with the maximum-likelihood criterion requires computations proportional to the number of words in the vocabulary. In this paper, we introduce adaptive importance sampling as a way to accelerate training of the model. The idea is to use an adaptive n-gram model to track the conditional distributions produced by the neural network. We show that a very significant speedup can be obtained on standard problems.     

Neural Net Based Hybrid Modeling of the Methanol Synthesis Process

A Hybrid modeling approach, combining an analytical model with a radial basis function neural network is introduced in this paper. The modeling procedure is combined with genetic algorithm based feature selection designed to select informative variables from the set of available measurements. By only using informative inputs, the model's generalization ability can be enhanced. The approach proposed is applied to modeling of the liquid–phase methanol synthesis. It is shown that a hybrid modeling approach exploiting available a priori knowledge and experimental data can considerably outperform a purely analytical approach.     

Applying the multivariate time-rescaling theorem to neural population models

Statistical models of neural activity are integral to modern neuroscience. Recently interest has grown in modeling the spiking activity of populations of simultaneously recorded neurons to study the effects of correlations and functional connectivity on neural information processing. However, any statistical model must be validated by an appropriate goodness-of-fit test. Kolmogorov-Smirnov tests based on the time-rescaling theorem have proven to be useful for evaluating point-process-based statistical models of single-neuron spike trains. Here we discuss the extension of the time-rescaling theorem to the multivariate (neural population) case. We show that even in the presence of strong correlations between spike trains, models that neglect couplings between neurons can be erroneously passed by the univariate time-rescaling test. We present the multivariate version of the time-rescaling theorem and provide a practical step-by-step procedure for applying it to testing the sufficiency of neural population models. Using several simple analytically tractable models and more complex simulated and real data sets, we demonstrate that important features of the population activity can be detected only using the multivariate extension of the test.     

卷积神经网络特征重要性分析及增强特征选择模型

卷积神经网络等深度神经网络凭借着其强大的表达能力、突出的分类性能,已在不同领域内得到了广泛应用.当面对高维特征时,深度神经网络通常被认为具有较好的鲁棒性,能够隐含地对特征进行选择,但由于网络参数巨大,如果数据量达不到足够的规模,则会导致学习不充分,因而可能无法达到最优的特征选择.而神经网络的黑箱特性使得无法观测神经网络选择了哪些特征,也无法评估其特征选择的能力.为此,以卷积神经网络为例,首先研究如何显式地表达神经网络中的特征重要性,提出了基于感受野的特征贡献度分析方法;其次,将神经网络特征选择与传统特征评价方法进行对比分析发现,在非海量样本的情况下,传统特征评价方法对高重要性特征和噪声特征的识别能力反而能够超过神经网络.因此,进一步地提出了卷积神经网络增强特征选择模型,将传统特征评价方法对特征重要性的理解结合到神经网络的学习过程中,以辅助深度神经网络进行特征选择.在基于文本的社交媒体用户属性建模任务下进行了对比实验,结果验证了该模型的有效性.     

Neural network technique for fuzzy multiobjective linear programming

Neural Network(NN) is well-known as one of powerful computing tools to solve optimization problems. Due to the massive computing unit-neurons and parallel mechanism of neural network approach we can solve the large-scale problem efficiently and optimal solution can be gotten. In this paper, we intoroduce improvement of the two-phase approach for solving fuzzy multiobjectve linear programming problem with both fuzzy objectives and constraints and we propose a new neural network technique for solving fuzzy multiobjective linear programming problems. The procedure and efficiency of this approach are shown with numerical simulations.     

IG-based genetically optimized fuzzy polynomial neural networks with fuzzy set-based polynomial neurons

In this study, we introduce and investigate a new topology of fuzzy-neural networks—fuzzy polynomial neural networks (FPNN) that is based on a genetically optimized multiplayer perceptron with fuzzy set-based polynomial neurons (FSPNs). We also develop a comprehensive design methodology involving mechanisms of genetic optimization and information granulation. In the sequel, the genetically optimized FPNN (gFPNN) is formed with the use of fuzzy set-based polynomial neurons (FSPNs) composed of fuzzy set-based rules through the process of information granulation. This granulation is realized with the aid of the C-means clustering (C-Means). The design procedure applied in the construction of each layer of an FPNN deals with its structural optimization involving the selection of the most suitable nodes (or FSPNs) with specific local characteristics (such as the number of input variable, the order of the polynomial, the number of membership functions, and a collection of specific subset of input variables) and address main aspects of parametric optimization. Along this line, two general optimization mechanisms are explored. The structural optimization is realized via genetic algorithms (GAs) and HCM method whereas in case of the parametric optimization we proceed with a standard least square estimation (learning). Through the consecutive process of structural and parametric optimization, a flexible neural network is generated in a dynamic fashion. The performance of the designed networks is quantified through experimentation where we use two modeling benchmarks already commonly utilized within the area of fuzzy or neurofuzzy modeling.     

On multivariate network analysis of statistical data sets with different measures of association

The main goal of the present paper is the development of a general framework of multivariate network analysis of statistical data sets. A general method of multivariate network construction, on the basis of measures of association, is proposed. In this paper we consider Pearson correlation network, sign similarity network, Fechner correlation network, Kruskal correlation network, Kendall correlation network, and the Spearman correlation network. The problem of identification of the threshold graph in these networks is discussed. Different multiple decision statistical procedures are proposed. It is shown that a statistical procedure used for threshold graph identification in one network can be efficiently used for any other network. Our approach allows us to obtain statistical procedures with desired properties for any network.     

Predictive neuro-control of uncertain systems: design and use of a neuro-optimizer

We consider the problem of predictive control of uncertain stochastic discrete I/O systems. Given a model identification procedure able to give accurate output system estimates, e.g. a neural network approximation, we use another feedforward neural network to generate at each time step a constrained optimal control. Dynamic backpropagation is used to improve when necessary the controller network parameters. Both system and controller neural structures are first selected off-line by a statistical Bayesian procedure in order to make the predictive control minimizing process more efficient. The issue of stochastic stability of the closed-loop is considered. We developed this approach for the tracking control of such uncertain systems as biotechnological processes. Actual and simulated predictive neuro-control case studies in this field of application are proposed as illustrations. A comparison with a more classic quasi-Newton-based approach is also proposed, showing the interest of this neuro-control approach.     

A simulation study of artificial neural networks for nonlinear time-series forecasting

This study presents an experimental evaluation of neural networks for nonlinear time-series forecasting. The effects of three main factors — input nodes, hidden nodes and sample size, are examined through a simulated computer experiment. Results show that neural networks are valuable tools for modeling and forecasting nonlinear time series while traditional linear methods are not as competent for this task. The number of input nodes is much more important than the number of hidden nodes in neural network model building for forecasting. Moreover, large sample is helpful to ease the overfitting problem.Scope and purposeInterest in using artificial neural networks for forecasting has led to a tremendous surge in research activities in the past decade. Yet, mixed results are often reported in the literature and the effect of key modeling factors on performance has not been thoroughly examined. The lack of systematic approaches to neural network model building is probably the primary cause of inconsistencies in reported findings. In this paper, we present a systematic investigation of the application of neural networks for nonlinear time-series analysis and forecasting. The purpose is to have a detailed examination of the effects of certain important neural network modeling factors on nonlinear time-series modeling and forecasting.