Output-relevant Variational autoencoder for Just-in-time soft sensor modeling with missing data

Main challenges for developing data-based models lie in the existence of high-dimensional and possibly missing observations that exist in stored data from industry process. Variational autoencoder (VAE) as one of the deep learning methods has been applied for extracting useful information or features from high-dimensional dataset. Considering that existing VAE is unsupervised, an output-relevant VAE is proposed for extracting output-relevant features in this work. By using correlation between process variables, different weight is correspondingly assigned to each input variable. With symmetric Kullback–Leibler (SKL) divergence, the similarity is evaluated between the stored samples and a query sample. According to the values of the SKL divergence, data relevant for modeling are selected. Subsequently, Gaussian process regression (GPR) is utilized to establish a model between the input and the corresponding output at the query sample. In addition, owing to the common existence of missing data in output data set, the parameters and missing data in the GPR are estimated simultaneously. A practical debutanizer industrial process is utilized to illustrate the effectiveness of the proposed method.     

基于改进降噪自编码器的馈线终端失效率预测

配电网中馈线终端设备由于运行环境恶劣,往往面临意外失效问题。本文针对海量馈线终端装置的失效率预测问题,使用堆叠降噪自编码器实现基于馈线终端的各个关键元件的失效率预测;采用基于Dropout的模型正则化方法防止自编码器训练过程中出现过拟合现象,同时采用Adadelta算法对堆叠自编码器进行优化,在保证预测准确率的同时提高学习速率,实现馈线终端故障失效率的高效准确预测;最后基于馈线终端装置现场数据进行仿真验证。仿真结果验证了本文方法对失效率预测的准确性和泛化能力。     

基于深度学习的miRNA靶位点预测研究综述

MicroRNAs(miRNAs)是一类长约22～23碱基(nt)的单链非编码RNA,在生物进化方面有着重要意义.成熟的miRNA会通过其种子序列(5'第2-8位核苷酸)与message RNAs(mRNAs)的3'UTR区域靶位点进行完全或不完全配对,实现切割mRNA及抑制mRNA翻译等功能.由于miRNA结合mRN...     

Densely connected convolutional network block based autoencoder for panorama map compression

As a novel virtual reality (VR) format, panorama maps are attracting increasing attention, while the compression of panorama images is still a concern. In this paper, a densely connected convolutional network block (dense block) based autoencoder is proposed to compress panorama maps. In the proposed autoencoder, dense blocks are specially designed to reuse feature maps and reduce redundancy of features. Meanwhile, a loss function, which imports a position-dependent weight item for each pixel, is proposed to train and adjust network parameters, in order to make the autoencoder fit to properties of panorama maps. Based on the proposed autoencoder and the weighted loss function, a greedy block-wise training scheme is also designed to avoid gradient vanishing problem and speed up training. During training process, the autoencoder is divided into several sub-nets. After each sub-net is trained separately, the whole network is fine-tuned to achieve the best performance. Experimental results demonstrate that the proposed autoencoder, compared with JPEG, saves up to 79.69 % bit rates, and obtains 7.27dB gain in BD-WS-PSNR or 0.0789 gain in BD-WS-SSIM. The proposed autoencoder also outperforms JPEG 2000, HEVC and VVC in both BD-WS-PSNR and BD-WS-SSIM. Meanwhile, subjective results show that the proposed autoencoder can recover details of panorama images, and reconstruct maps with high visual quality.     

A generative neural network model for the quality prediction of work in progress products

One of the key challenges in manufacturing processes is improving the accuracy of quality monitoring and prediction. This paper proposes a generative neural network model for automatically predicting work-in-progress product quality. Our approach combines an unsupervised feature-extraction step with a supervised learning method. An autoencoding neural network is trained using raw manufacturing process data to extract rich information from production line recordings. Then, the extracted features are reformed as time-series and are fed into a multi-layer perceptron for predicting product quality. Finally, the outputs are decoded into a forecast quality measure. We evaluate the performance of the generative model on a case study from a powder metallurgy process. Our experimental results suggest that our method can precisely capture the defective products.     

Defective wafer detection using a denoising autoencoder for semiconductor manufacturing processes

Defective wafer detection is essential to avoid loss of yield due to process abnormalities in semiconductor manufacturing. For most complex processes in semiconductor manufacturing, various sensors are installed on equipment to capture process information and equipment conditions, including pressure, gas flow, temperature, and power. Because defective wafers are rare in current practice, supervised learning methods usually perform poorly as there are not enough defective wafers for fault detection (FD). The existing methods of anomaly detection often rely on linear excursion detection, such as principal component analysis (PCA), k-nearest neighbor (kNN) classifier, or manual inspection of equipment sensor data. However, conventional methods of observing equipment sensor readings directly often cannot identify the critical features or statistics for detection of defective wafers. To bridge the gap between research-based knowledge and semiconductor practice, this paper proposes an anomaly detection method that uses a denoise autoencoder (DAE) to learn a main representation of normal wafers from equipment sensor readings and serve as the one-class classification model. Typically, the maximum reconstruction error (MaxRE) is used as a threshold to differentiate between normal and defective wafers. However, the threshold by MaxRE usually yields a high false positive rate of normal wafers due to the outliers in an imbalanced data set. To resolve this difficulty, the Hampel identifier, a robust method of outlier detection, is adopted to determine a new threshold for detecting defective wafers, called MaxRE without outlier (MaxREwoo). The proposed method is illustrated using an empirical study based on the real data of a wafer fabrication. Based on the experimental results, the proposed DAE shows great promise as a viable solution for on-line FD in semiconductor manufacturing.     

Fooling intrusion detection systems using adversarially autoencoder

Due to the increasing cyber-attacks, various Intrusion Detection Systems (IDSs) have been proposed to identify network anomalies. Most existing machine learning-based IDSs learn patterns from the features extracted from network traffic flows, and the deep learning-based approaches can learn data distribution features from the raw data to differentiate normal and anomalous network flows. Although having been used in the real world widely, the above methods are vulnerable to some types of attacks. In this paper, we propose a novel attack framework, Anti-Intrusion Detection AutoEncoder (AIDAE), to generate features to disable the IDS. In the proposed framework, an encoder transforms features into a latent space, and multiple decoders reconstruct the continuous and discrete features, respectively. Additionally, a generative adversarial network is used to learn the flexible prior distribution of the latent space. The correlation between continuous and discrete features can be kept by using the proposed training scheme. Experiments conducted on NSL-KDD, UNSW-NB15, and CICIDS2017 datasets show that the generated features indeed degrade the detection performance of existing IDSs dramatically.     

基于端到端深度学习的智能车自动转向研究*

为解决由图像直接计算出控制量的端到端深度学习算法中感知器和控制器难以区分的问题,对其网络结构进行了改进。通过预训练一个自编码器,得到良好的道路特征编码后,将编码器作为感知器和和转角预测控制器一起进行端到端的训练。训练结果表明,改进后的自动转向网络模型收敛的更快,预测的角度在测试集上能较好的跟随实际角度变化而变化。利用解码器和特征图反向传播法分别还原出道路图片,可视化了该自动转向模型重点关注的道路特征。     

A Bi-population Cooperative Optimization Algorithm Assisted by an Autoencoder for Medium-scale Expensive Problems

This study presents an autoencoder-embedded optimization (AEO) algorithm which involves a bi-population cooperative strategy for medium-scale expensive problems (MEPs). A huge search space can be compressed to an informative low-dimensional space by using an autoencoder as a dimension reduction tool. The search operation conducted in this low space facilitates the population with fast convergence towards the optima. To strike the balance between exploration and exploitation during optimization, two phases of a tailored teaching-learning-based optimization (TTLBO) are adopted to coevolve solutions in a distributed fashion, wherein one is assisted by an autoencoder and the other undergoes a regular evolutionary process. Also, a dynamic size adjustment scheme according to problem dimension and evolutionary progress is proposed to promote information exchange between these two phases and accelerate evolutionary convergence speed. The proposed algorithm is validated by testing benchmark functions with dimensions varying from 50 to 200. As indicated in our experiments, TTLBO is suitable for dealing with medium-scale problems and thus incorporated into the AEO framework as a base optimizer. Compared with the state-of-the-art algorithms for MEPs, AEO shows extraordinarily high efficiency for these challenging problems, thus opening new directions for various evolutionary algorithms under AEO to tackle MEPs and greatly advancing the field of medium-scale computationally expensive optimization.