Batch-to-batch control of fed-batch processes using control-affine feedforward neural network

A control strategy for fed-batch processes is proposed based on control affine feed-forward neural network (CAFNN). Many fed-batch processes can be considered as a class of control affine nonlinear systems. CAFNN is constructed by a special structure to fit the control affine system. It is similar to a multi-layer feed-forward neural network, but it has its own particular feature to model the fed-batch process. CAFNN can be trained by a modified Levenberg–Marquardt (LM) algorithm. However, due to model-plant mismatches and unknown disturbances, the optimal control policy calculated based on the CAFNN model may not be optimal when applied to the fed-batch process. In terms of the repetitive nature of fed-batch processes, iterative learning control (ILC) can be used to improve the process performance from batch to batch. Due to the special structure of CAFNN, the gradient information of CAFNN can be computed analytically and applied to the batch-to-batch ILC. Under the ILC strategy from batch to batch, endpoint product qualities of fed-batch processes can be improved gradually. The proposed control scheme is illustrated on a simulated fed-batch ethanol fermentation process.     

Document segmentation using textural features summarization and feedforward neural network

Document Segmentation is a process that aims to filter documents while identifying certain regions of interest. Generally, the regions of interest include texts, graphics (image occupied regions) and the background. This paper presents a novel top-bottom approach to perform document segmentation using texture features that are extracted from the specified/selected documents. A mask of suitable size is used to summarize textural features, and statistical parameters are captured as blocks in document images. Four textural features that are extracted from masks using the gray level co-occurrence matrix (glcm) include entropy, contrast, energy and homogeneity. Furthermore, two statistical parameters extracted from corresponding masks are the modal and median pixel values. The extracted attributes allow the classification of each mask or block as text, graphics, and background. A feedforward network is trained on the 6 extracted attributes, using documents obtained from a public database ; an error rate of 15.77 % is achieved. Furthermore, it is shown that this novel approach produces promising performance in segmenting documents and is expected to be significantly efficient for content-based information retrieval systems. Detection of duplicate documents within large databases is another potential area of application.     

A feedforward multi-layer neural network for machine cell formation in computer integrated manufacturing

Neural-network applications have been one of the better alternatives either for simulating massive data in parallel, or embedding human subjective decisions into existing quantitative models, thereby spawning a qualitative model. This paper introduces a linear classifier with a classical feedforward neural network in forming machine cells or groups for Computer Integrated Manufacturing. The proposed method, through experiment, has been proven to outperform conventional methods such as Part Family Analysis (PFA) and BLOCPLAN, among others. A single-layer perceptron, along with multi-layer feedforward network where applicable, have been employed in forming the part families. The underlying philosophy is the Group Technology (GT). The developed models and algorithms are illustrated with a numerical example.     

A smart pixel-based feedforward neural network

A novel smart pixel-based neural network was realized experimentally. The matrix multiplication is split into positive and negative components and computed optically. The necessary subtraction, binarization, and transmission of the resulting matrices is accomplished via a prototype smart pixel spatial light modulator. The result is a neural network that performs truly parallel computation without requiring the use of an external processor.     

Optimizing feedforward artificial neural network architecture

Despite the fact that feedforward artificial neural networks (ANNs) have been a hot topic of research for many years there still are certain issues regarding the development of an ANN model, resulting in a lack of absolute guarantee that the model will perform well for the problem at hand. The multitude of different approaches that have been adopted in order to deal with this problem have investigated all aspects of the ANN modelling procedure, from training data collection and pre/post-processing to elaborate training schemes and algorithms. Increased attention is especially directed to proposing a systematic way to establish an appropriate architecture in contrast to the current common practice that calls for a repetitive trial-and-error process, which is time-consuming and produces uncertain results.This paper proposes such a methodology for determining the best architecture and is based on the use of a genetic algorithm (GA) and the development of novel criteria that quantify an ANN's performance (both training and generalization) as well as its complexity. This approach is implemented in software and tested based on experimental data capturing workpiece elastic deflection in turning. The intention is to present simultaneously the approach's theoretical background and its practical application in real-life engineering problems. Results show that the approach performs better than a human expert, at the same time offering many advantages in comparison to similar approaches found in literature.     

Measurement of machine performance degradation using a neural network model

Machines degrade as a result of aging and wear, which decreases performance reliability and increases the potential for faults and failures. The impact of machine faults and failures on factory productivity is an important concern for manufacturing industries. Economic impacts relating to machine availability and reliability, as well as corrective (reactive) maintenance costs, have prompted facilities and factories to improve their maintenance techniques and operations to monitor machine degradation and detect faults. This paper presents an innovative methodology that can change maintenance practice from that of reacting to breakdowns, to one of preventing breakdowns, thereby reducing maintenance costs and improving productivity. To analyze the machine behavior quantitatively, a pattern discrimination model (PDM) based on a cerebellar model articulation controller (CMAC) neural network was developed. A stepping motor and a PUMA 560 robot were used to study the feasibility of the developed technique. Experimental results have shown that the developed technique can analyze machine degradation quantitatively. This methodology could help operators set up machines for a given criterion, determine whether the machine is running correctly, and predict problems before they occur. As a result, maintenance hours could be used more effectively and productively.     

Numerical solution of a calculus of variations problem using the feedforward neural network architecture

It is demonstrated, through theory and numerical example, how it is possible to construct directly and noniteratively a feedforward neural network to solve a calculus of variations problem. The method, using the piecewise linear and cubic sigmoid transfer functions, is linear in storage and processing time. The L₂ norm of the network approximation error decreases quadratically with the piecewise linear transfer function and quartically with the piecewise cubic sigmoid as the number of hidden layer neurons increases. The construction requires imposing certain constraints on the values of the input, bias, and output weights, and the attribution of certain roles to each of these parameters.

All results presented used the piecewise linear and cubic sigmoid transfer functions. However, the noniterative approach should also be applicable to the use of hyperbolic tangents and radial basis functions.     

Precision constrained stochastic resonance in a feedforward neural network

Stochastic resonance (SR) is a phenomenon in which the response of a nonlinear system to a subthreshold information-bearing signal is optimized by the presence of noise. By considering a nonlinear system (network of leaky integrate-and-fire (LIF) neurons) that captures the functional dynamics of neuronal firing, we demonstrate that sensory neurons could, in principle harness SR to optimize the detection and transmission of weak stimuli. We have previously characterized this effect by use of signal-to-noise ratio (SNR). Here in addition to SNR, we apply an entropy-based measure (Fisher information) and compare the two measures of quantifying SR. We also discuss the performance of these two SR measures in a full precision floating point model simulated in Java and in a precision limited integer model simulated on a field programmable gate array (FPGA). We report in this study that stochastic resonance which is mainly associated with floating point implementations is possible in both a single LIF neuron and a network of LIF neurons implemented on lower resolution integer based digital hardware. We also report that such a network can improve the SNR and Fisher information of the output over a single LIF neuron.     

Visualizing the function computed by a feedforward neural network

A method for visualizing the function computed by a feedforward neural network is presented. It is most suitable for models with continuous inputs and a small number of outputs, where the output function is reasonably smooth, as in regression and probabilistic classification tasks. The visualization makes readily apparent the effects of each input and the way in which the functions deviate from a linear function. The visualization can also assist in identifying interactions in the fitted model. The method uses only the input-output relationship and thus can be applied to any predictive statistical model, including bagged and committee models, which are otherwise difficult to interpret. The visualization method is demonstrated on a neural network model of how the risk of lung cancer is affected by smoking and drinking.     

Establishing impacts of the inputs in a feedforward neural network

Artificial neural network models are now being widely used in various areas of statistical research. Nevertheless, there is a certain degree of reluctance amongst members of the business profession in applying neural networks to business analysis. One of the major causes of scepticism is the inability of the models to provide explanation on how they reach their decisions. The current experiment is concerned with solving this problem by developing a framework for establishing the impacts of the input variables on the network output. The framework was tested on a feedforward neural network model for turnover forecasting which was developed in co-operation with a British retailer using real world marketing data. The results obtained are compared with those from a sensitivity analysis.     

Design of highly effective multilayer feedforward neural network by using genetic algorithm

This paper presents a highly effective and precise neural network method for choosing the activation functions (AFs) and tuning the learning parameters (LPs) of a multilayer feedforward neural network by using a genetic algorithm (GA). The performance of the neural network mainly depends on the learning algorithms and the network structure. The backpropagation learning algorithm is used for tuning the network connection weights, and the LPs are obtained by the GA to provide both fast and reliable learning. Also, the AFs of each neuron in the network are automatically chosen by a GA. The present study consists of 10 different functions to accomplish a better convergence of the desired input–output mapping. Test studies are performed to solve a set of two-dimensional regression problems for the proposed genetic-based neural network (GNN) and conventional neural network having sigmoid AFs and constant learning parameters. The proposed GNN has also been tested by applying it to three real problems in the fields of environment, medicine, and economics. Obtained results prove that the proposed GNN is more effective and reliable when compared with the classical neural network structure.     

混合前馈型神经网络在入侵检测中的应用研究

提出一种基于混沌神经元的混合前馈型神经网络,用于检测复杂的网络入侵模式．这种神经网络具有混沌神经元的延时、收集、思维和分类的功能,避免了MLP神经网络仅能识别网络中当前的滥用入侵行为的弱点．对混合网络进行训练后,将该网络用于滥用入侵检测．使用DARPA数据集对该方法进行评估,结果表明该方法可有效地提高对具备延时特性的Probe和DOS入侵的检测性能．     

Recurrent neural network training with feedforward complexity

This paper presents a training method that is of no more than feedforward complexity for fully recurrent networks. The method is not approximate, but rather depends on an exact transformation that reveals an embedded feedforward structure in every recurrent network. It turns out that given any unambiguous training data set, such as samples of the state variables and their derivatives, we need only to train this embedded feedforward structure. The necessary recurrent network parameters are then obtained by an inverse transformation that consists only of linear operators. As an example of modeling a representative nonlinear dynamical system, the method is applied to learn Bessel's differential equation, thereby generating Bessel functions within, as well as outside the training set.     

Capabilities of a four-layered feedforward neural network: fourlayers versus three

Neural-network theorems state that only when there are infinitely many hidden units is a four-layered feedforward neural network equivalent to a three-layered feedforward neural network. In actual applications, however, the use of infinitely many hidden units is impractical. Therefore, studies should focus on the capabilities of a neural network with a finite number of hidden units, In this paper, a proof is given showing that a three-layered feedforward network with N-1 hidden units can give any N input-target relations exactly. Based on results of the proof, a four-layered network is constructed and is found to give any N input-target relations with a negligibly small error using only (N/2)+3 hidden units. This shows that a four-layered feedforward network is superior to a three-layered feedforward network in terms of the number of parameters needed for the training data.     

A class-modular feedforward neural network for handwriting recognition

Since the conventional feedforward neural networks for character recognition have been designed to classify a large number of classes with one large network structure, inevitably it poses the very complex problem of determining the optimal decision boundaries for all the classes involved in a high-dimensional feature space. Limitations also exist in several aspects of the training and recognition processes. This paper introduces the class modularity concept to the feedforward neural network classifier to overcome such limitations. In the class-modular concept, the original K-classification problem is decomposed into K 2-classification subproblems. A modular architecture is adopted which consists of K subnetworks, each responsible for discriminating a class from the other K−1 classes. The primary purpose of this paper is to prove the effectiveness of class-modular neural networks in terms of their convergence and recognition power. Several cases have been studied, including the recognition of handwritten numerals (10 classes), English capital letters (26 classes), touching numeral pairs (100 classes), and Korean characters in postal addresses (352 classes). The test results confirmed the superiority of the class-modular neural network and the interesting aspects on further investigations of the class modularity paradigm.     

Adaptive sliding mode approach for learning in a feedforward neural network

An adaptive learning algorithm is proposed for a feedforward neural network. The design principle is based on the sliding mode concept. Unlike the existing algorithms, the adaptive learning algorithm developed does not require a prioriknowledge of upper bounds of bounded signals. The convergence of the algorithm is established and conditions given. Simulations are presented to show the effectiveness of the algorithm.     

前馈神经网络的混沌BP 混合学习算法

简要分析由Logistic映射产生的混沌数以及不同混沌序列之间的概率统计特性，为混沌全局性搜索提供了依据．将一种快速BP算法与混沌优化相结合，提出了混沌BP混合算法，由于混沌Logistic映射的遍历性、随机性，使得混合算法收敛速度快，且具有全局性，采用混合算法对XOR问题和非线性函数进行仿真，结果表明该算法明显优于标准BP算法和快速BP算法。