基于智能解析余度的容错飞控系统设计

常规的解析余度容错方法容易受到不确定因素和随机干扰的影响,本文以飞行控制系统为研究对象,提出基于智能解析余度的容错飞行控制系统设计方案,使用径向基神经网络的在线学习和全局逼近的性能,建立飞行控制系统传感器之间的解析余度关系,利用不相同传感器之间的解析关系进行残差分析从而进行传感器的故障隔离与信号重构.这样有效地抑制了测量噪声和模型不确定性.应用某型飞机进行仿真,实现了传感器的在线故障隔离与重构,验证了该方法的有效性.     

A new approach to simultaneous localization and map building with implicit model learning using neuro evolutionary optimization

This paper presents Neuro-Evolutionary Optimization SLAM (NeoSLAM) a novel approach to SLAM that uses a neural network (NN) to autonomously learn both a nonlinear motion model and the noise statistics of measurement data. The NN is trained using evolutionary optimization to learn the residual error of the motion model, which is then added to the odometry data to obtain the full motion model estimate. Stochastic optimization is used, to accommodate any kind of cost function. Prediction and correction are performed simultaneously within our neural framework, which implicitly integrates the motion and sensor models. An evolutionary programming (EP) algorithm is used to progressively refine the neural model until it generates a trajectory that is most consistent with the actual sensor measurements. During this learning process, NeoSLAM does not require any prior knowledge of motion or sensor models and shows consistently good performance regardless of the robot and the sensor noise type. Furthermore, NeoSLAM does not require the data association step at loop closing which is crucial in most other SLAM algorithms, but can still generate an accurate map. Experiments in various complex environments with widely-varying types of noise show that the learning capability of NeoSLAM ensures performance that is consistently less sensitive to noise and more accurate than that of other SLAM methods.     

A randomized model ensemble approach for reconstructing signals from faulty sensors

On-line sensor monitoring aims at detecting anomalies in sensors and reconstructing their correct signals during operation. The techniques used for signal reconstruction are commonly based on auto-associative regression models. In full scale implementations however, the number of sensors to be monitored is often too large to be handled effectively by a single reconstruction model. In this paper we propose to tackle the problem by resorting to a pool (ensemble) of reconstruction models, each one handling an individual group of signals. This approach involves two main technical steps: firstly, a procedure for constructing signal groups, and secondly a procedure for combining the outputs of the reconstruction models associated to the groups. For the signal grouping step, a wrapper optimization search is proposed to identify the optimal number of groups in the ensemble and the size of the groups. For the model output aggregation step, a simple arithmetic average is adopted. Ensemble accuracy and robustness is achieved by promoting diversity between the signal groups through the use of the Random Feature Selection Ensemble (RFSE) technique in combination with the Bootstrapping AGGregatING (BAGGING) technique for training data selection. The individual reconstruction models are based on Principal Components Analysis (PCA). The proposed approach has been applied to a real case study concerning 215 signals monitored at a Finnish nuclear pressurized water reactor. The results obtained have been compared with those achieved by an equivalent ensemble of models based on a grouping directly optimized by a Multi-Objective Genetic Algorithm (MOGA).     

基于流形正则化域适应随机权神经网络的湿式球磨机负荷参数软测量

针对湿式球磨机多工况运行过程中标签样本难以获取和工况改变导致的原测量模型失准问题,本文引入域适应随机权神经网络（Domain adaptive random weight neural network,DARWNN）,实现待测工况中少量标签样本与原工况样本共同进行迁移学习.DARWNN网络解决了不同工况间难以共同进行机器学习的问题,但其只考虑经验风险,而未考虑结构风险,从而泛化性能较差,预测精度较低.在此基础上,本文引入流形正则化,并构建基于流形正则化的域适应随机权神经网络（Domain adaptive manifold regularization random weight neural network,DAMRRWNN）,以保持数据几何结构,提高相应模型性能.实验结果表明,所提方法可以有效提高DARWNN的学习精度,解决多工况情况下湿式球磨机负荷参数软测量问题.     

一种基于FCMAC神经网络的优化方法

提出了一种利用FCMAC(Fuzzy Cerebellar Model Articulation Controller)神经网络进行优化的方法。该方法由学习过程和优化过程两部分组成。对于许多没有模型可参考的实际过程，使用该方法只需要传感器的观测信息就能进行优化。仿真结果证明了该方法的有效性与优越性，进而提出了在实际应用中进行优化的一种方案。     

改进的弹性网模型在深度神经网络中的应用

由于具有较高的模型复杂度，深层神经网络容易产生过拟合问题，为了减少该问题对网络性能的不利影响，提出一种基于改进的弹性网模型的深度学习优化方法。首先，考虑到变量之间的相关性，对弹性网模型中的L1范数的不同变量进行自适应加权，从而得到L2范数与自适应加权的L1范数的线性组合。其次，将改进的弹性网络模型与深度学习的优化模型相结合，给出在这种新正则项约束下求解神经网络参数的过程。然后，推导出改进的弹性网模型在神经网络优化中具有群组选择能力和Oracle性质，进而从理论上保证该模型是一种更加鲁棒的正则化方法。最后，在多个回归问题和分类问题的实验中，相对于L1、L2和弹性网正则项，该方法的回归测试误差可分别平均降低87.09、88.54和47.02，分类测试准确度可分别平均提高3.98、2.92和3.58个百分点。由此，在理论和实验两方面验证了改进的弹性网模型可以有效地增强深层神经网络的泛化能力，提升优化算法的性能，解决深度学习的过拟合问题。     

Uncertainty-aware soft sensor using Bayesian recurrent neural networks

Data-driven soft sensors have been widely used to measure key variables for industrial processes. Soft sensors using deep learning models have attracted considerable attention and shown superior predictive performance. However, if a soft sensor encounters an unexpected situation in inferring data or if noisy input data is used, the estimated value derived by a standard soft sensor using deep learning may at best be untrustworthy. This problem can be mitigated by expressing a degree of uncertainty about the trustworthiness of the estimated value produced by the soft sensor. To address this issue of uncertainty, we propose using an uncertainty-aware soft sensor that uses Bayesian recurrent neural networks (RNNs). The proposed soft sensor uses a RNN model as a backbone and is then trained using Bayesian techniques. The experimental results demonstrated that such an uncertainty-aware soft sensor increases the reliability of predictive uncertainty. In comparisons with a standard soft sensor, it shows a capability to use uncertainties for interval prediction without compromising predictive performance.     

Leave-one-out manifold regularization

The manifold regularization (MR) based semi-supervised learning could explore structural relationships from both labeled and unlabeled data. However, the model selection of MR seriously affects its predictive performance due to the inherent additional geometry regularizer of labeled and unlabeled data. In this paper, two continuous and two inherent discrete hyperparameters are selected as optimization variables, and a leave-one-out cross-validation (LOOCV) based Predicted REsidual Sum of Squares (PRESS) criterion is first presented for model selection of MR to choose appropriate regularization coefficients and kernel parameters. Considering the inherent discontinuity of the two hyperparameters, the minimization process is implemented by using a improved Nelder-Mead simplex algorithm to solve the inherent discrete and continues hybrid variables set. The manifold regularization and model selection algorithm are applied to six synthetic and real-life benchmark dataset. The proposed approach, leveraged by effectively exploiting the embedded intrinsic geometric manifolds and unbiased LOOCV estimation, outperforms the original MR and supervised learning approaches in the empirical study.     

Data-driven soft sensor development based on deep learning technique

In industrial process control, some product qualities and key variables are always difficult to measure online due to technical or economic limitations. As an effective solution, data-driven soft sensors provide stable and reliable online estimation of these variables based on historical measurements of easy-to-measure process variables. Deep learning, as a novel training strategy for deep neural networks, has recently become a popular data-driven approach in the area of machine learning. In the present study, the deep learning technique is employed to build soft sensors and applied to an industrial case to estimate the heavy diesel 95% cut point of a crude distillation unit (CDU). The comparison of modeling results demonstrates that the deep learning technique is especially suitable for soft sensor modeling because of the following advantages over traditional methods. First, with a complex multi-layer structure, the deep neural network is able to contain richer information and yield improved representation ability compared with traditional data-driven models. Second, deep neural networks are established as latent variable models that help to describe highly correlated process variables. Third, the deep learning is semi-supervised so that all available process data can be utilized. Fourth, the deep learning technique is particularly efficient dealing with massive data in practice.     

Modeling plasma surface modification of textile fabrics using artificial neural networks

In this paper, a neural network approach is used to understand the effects of fabric features and plasma processing parameters on fabric surface wetting properties. In this approach, fourteen features characterizing woven structures and two plasma parameters are taken as input variables, and the water contact angle cosine and the capillarity height of woven fabrics as output variables. In order to reduce the complexity of the model and effectively learn the network structure from a small number of data, a fuzzy logic based method is used for selecting the most relevant parameters which are taken as input variables of the reduced neural network models. With these relevant parameters, we can effectively control the plasma treatment by selecting the most appropriate fabric materials. Two techniques are used for improving the generalization capability of neural networks: (i) early stopping and (ii) Bayesian regularization. A methodology for optimizing such models is described. The learning abilities and prediction capabilities of the neural net models are compared in terms of different statistical performance criteria. Moreover, a connection weight method is used to determine the relative importance of each input variable in the networks. The obtained results show that neural network models could predict the process performance with reasonable accuracy. However, the neural model trained using Bayesian regularization provides the best results. Thus, it can be concluded that Bayesian network promises to be a valuable quantitative tool to evaluate, understand, and predict woven fabric surface modification by atmospheric air-plasma treatment.     

A time-efficient convolutional neural network model in human activity recognition

Activity recognition represents the task of classifying data derived from different sensor types into one of predefined activity classes. The most popular and beneficial sensors in the area of action recognition are inertial sensors such as accelerometer and gyroscope. Convolutional neural network (CNN) as one of the best deep learning methods has recently attracted much attention to the problem of activity recognition, where 1D kernels capture local dependency over time in a series of observations measured at inertial sensors (3-axis accelerometers and gyroscopes) while in 2D kernels apart from time dependency, dependency between signals from different axes of same sensor and also over different sensors will be considered. Most convolutional neural networks used for recognition task are built using convolution and pooling layers followed by a few number of fully connected layers but large and deep neural networks have high computational costs. In this paper, we propose a new architecture that consists solely of convolutional layers and find that with removing the pooling layers and instead adding strides to convolution layers, the computational time will decrease notably while the model performance will not change or in some cases will even improve. Also both 1D and 2D convolutional neural networks with and without pooling layer will be investigated and their performance will be compared with each other and also with some other hand-crafted feature based methods. The third point that will be discussed in this paper is the impact of applying fast fourier transform (FFT) to inputs before training learning algorithm. It will be shown that this preprocessing will enhance the model performance. Experiments on benchmark datasets demonstrate the high performance of proposed 2D CNN model with no pooling layers.

     

基于深度学习特征提取与多目标优化集成修剪的选择性集成学习软测量方法

集成学习已成为一种广泛使用的软测量建模框架,但是建立高性能的集成学习软测量模型依然面临特征选择不当、基模型多样性不足、基模型估计性能不佳等诸多挑战.为此,提出一种基于堆栈自编码器多样性生成机制的选择性集成学习高斯过程回归(selective ensemble of stacked autoencoder based Gaussian process regression, SESAEGPR)软测量建模方法.该方法充分发挥深度学习在特征提取方面的优势,通过构建多样性的堆栈自编码器(stacked autoencoder, SAE)网络,建立基于隐特征的高斯过程回归(Gaussian process regression, GPR)基模型.基于模型性能提升率和进化多目标优化对SAEGPR基模型实施两次集成修剪,以降低集成模型复杂度、保持甚至进一步提升模型估计性能,最后,引入PLS Stacking集成策略实现基模型融合.所提出方法显著优于传统全局和全集成软测量建模方法,其有效性和优越性通过青霉素发酵过程和Tennessee Eastman化工过程得到验证.     

Categorization in unsupervised neural networks: the Eidos model

Proulx and Begin (1995) recently explained the power of a learning rule that combines Hebbian and anti-Hebbian learning in unsupervised auto-associative neural networks. Combined with the brain-state-in-a-box transmission rule, this learning rule defines a new model of categorization: the Eidos model. To test this model, a simulated neural network, composed of 35 interconnected units, is subjected to an alphabetical characters recognition task. The results indicate the necessity of adding two parameters to the model: a restraining parameter and a forgetting parameter. The study shows the outstanding capacity of the model to categorize highly altered stimuli after a suitable learning process. Thus, the Eidos model seems to be an interesting option to achieve categorization in unsupervised neural networks.     

An improved kernel-based incremental extreme learning machine with fixed budget for nonstationary time series prediction

In order to curb the model expansion of the kernel learning methods and adapt the nonlinear dynamics in the process of the nonstationary time series online prediction, a new online sequential learning algorithm with sparse update and adaptive regularization scheme is proposed based on kernel-based incremental extreme learning machine (KB-IELM). For online sparsification, a new method is presented to select sparse dictionary based on the instantaneous information measure. This method utilizes a pruning strategy, which can prune the least “significant” centers, and preserves the important ones by online minimizing the redundancy of dictionary. For adaptive regularization scheme, a new objective function is constructed based on basic ELM model. New model has different structural risks in different nonlinear regions. At each training step, new added sample could be assigned optimal regularization factor by optimization procedure. Performance comparisons of the proposed method with other existing online sequential learning methods are presented using artificial and real-word nonstationary time series data. The results indicate that the proposed method can achieve higher prediction accuracy, better generalization performance and stability.

     

Data supplement for a soft sensor using a new generative model based on a variational autoencoder and Wasserstein GAN

In industrial process control, measuring some variables is difficult for environmental or cost reasons. This necessitates employing a soft sensor to predict these variables by using the collected data from easily measured variables. The prediction accuracy and computational speed in the modeling procedure of soft sensors could be improved with adequate training samples. However, the rough environment of some industrial fields makes it difficult to acquire enough samples for soft sensor modeling. Generative adversarial networks (GANs) and the variational autoencoder (VAE) are two prominent methods that have been employed for learning generative models. In the current work, the VA-WGAN combining VAE with Wasserstein generative adversarial networks (WGAN) as a generative model is established to produce new samples for soft sensors by using the decoder of VAE as the generator in WGAN. An actual industrial soft sensor with insufficient data is used to verify the data generation capability of the proposed model. According to the experimental results, the samples obtained with the proposed model more closely resemble the true samples compared with the other four common generative models. Moreover, the insufficiency of the training data and the prediction precision of soft sensors could be improved via these constructed samples.     

Application of the recurrent multilayer perceptron in modelingcomplex process dynamics

A nonlinear dynamic model is developed for a process system, namely a heat exchanger, using the recurrent multilayer perceptron network as the underlying model structure. The perceptron is a dynamic neural network, which appears effective in the input-output modeling of complex process systems. Dynamic gradient descent learning is used to train the recurrent multilayer perceptron, resulting in an order of magnitude improvement in convergence speed over a static learning algorithm used to train the same network. In developing the empirical process model the effects of actuator, process, and sensor noise on the training and testing sets are investigated. Learning and prediction both appear very effective, despite the presence of training and testing set noise, respectively. The recurrent multilayer perceptron appears to learn the deterministic part of a stochastic training set, and it predicts approximately a moving average response of various testing sets. Extensive model validation studies with signals that are encountered in the operation of the process system modeled, that is steps and ramps, indicate that the empirical model can substantially generalize operational transients, including accurate prediction of instabilities not in the training set. However, the accuracy of the model beyond these operational transients has not been investigated. Furthermore, online learning is necessary during some transients and for tracking slowly varying process dynamics. Neural networks based empirical models in some cases appear to provide a serious alternative to first principles models.     

基于深度学习LDAMP网络的量子状态估计

设计出一种基于学习去噪的近似消息传递(Learned denoising-based approximate message passing, LDAMP)的深度学习网络,将其应用于量子状态的估计.该网络将去噪卷积神经网络与基于去噪的近似消息传递算法相结合,利用量子系统输出的测量值作为网络输入,通过设计出的带有去噪卷积神经网络的LDAMP网络重构出原始密度矩阵,从大量的训练样本中提取各种不同类型密度矩阵的结构特征,来实现对量子本征态、叠加态以及混合态的估计.在对4个量子位的量子态估计的具体实例中,分别在无和有测量噪声干扰情况下,对基于LDAMP网络的量子态估计进行了仿真实验性能研究,并与基于压缩感知的交替方向乘子法和三维块匹配近似消息传递等算法进行估计性能对比研究.数值仿真实验结果表明,所设计的LDAMP网络可以在较少的测量的采样率下,同时完成对4种量子态的更高精度估计.     

A neural network predictive control system for paper mill wastewater treatment

This paper presents a neural network predictive control scheme for studying the coagulation process of wastewater treatment in a paper mill. A multi-layer back-propagation neural network is employed to model the nonlinear relationships between the removal rates of pollutants and the chemical dosages, in order to adapt the system to a variety of operating conditions and acquire a more flexible learning ability. The system includes a neural network emulator of the reaction process, a neural network controller, and an optimization procedure based on a performance function that is used to identify desired control inputs. The gradient descent algorithm method is used to realize the optimization procedure. The results indicate that reasonable forecasting and control performances have been achieved through the developed system.     

Regularized online sequential learning algorithm for single-hidden layer feedforward neural networks

Online learning algorithms have been preferred in many applications due to their ability to learn by the sequentially arriving data. One of the effective algorithms recently proposed for training single hidden-layer feedforward neural networks (SLFNs) is online sequential extreme learning machine (OS-ELM), which can learn data one-by-one or chunk-by-chunk at fixed or varying sizes. It is based on the ideas of extreme learning machine (ELM), in which the input weights and hidden layer biases are randomly chosen and then the output weights are determined by the pseudo-inverse operation. The learning speed of this algorithm is extremely high. However, it is not good to yield generalization models for noisy data and is difficult to initialize parameters in order to avoid singular and ill-posed problems. In this paper, we propose an improvement of OS-ELM based on the bi-objective optimization approach. It tries to minimize the empirical error and obtain small norm of network weight vector. Singular and ill-posed problems can be overcome by using the Tikhonov regularization. This approach is also able to learn data one-by-one or chunk-by-chunk. Experimental results show the better generalization performance of the proposed approach on benchmark datasets.