Deep fisher autoencoder combined with self-organizing map for visual industrial process monitoring

Feature extraction plays an important role in industrial process monitoring. Autoencoder and its deep framework, deep autoencoder, are used to extract latent features from complex data. However, the latent features extracted by autoencoder through unsupervised learning may not be useful for discriminative tasks. Fisher discriminant analysis (FDA) is another widely used supervised feature extraction technique that take full advantage of the Fisher criterion to enable the extracted discriminative features to maximize inter-class distance while minimizing intra-class distance. Drawing on FDA and autoencoder, this study proposes Fisher autoencoder (FAE) to extract discriminative features. FAE uses the Fisher criterion to guide the autoencoder in minimizing the reconstruction error while enabling the extracted features by the hidden layer to increase the separation between classes. We stack FAE to derive deep FAE (DFAE) for feature extraction, then we combine DFAE with self-organizing map (DFAE-SOM), which is a tool typically used in visualization for visual process monitoring. Tennessee Eastman process and an actual dataset of the blade icing of wind turbine are applied to test the performance of DFAE-SOM. The experiment demonstrates that DFAE increases the separation between classes more than DAE and other standard techniques. Therefore, DFAE is conducive to visualization and improves the accuracy of process monitoring.     

A teaching method using a self-organizing map for reinforcement learning

We described a new preteaching method for re-inforcement learning using a self-organizing map (SOM). The purpose is to increase the learning rate using a small amount of teaching data generated by a human expert. In our proposed method, the SOM is used to generate the initial teaching data for the reinforcement learning agent from a small amount of teaching data. The reinforcement learning function of the agent is initialized by using the teaching data generated by the SOM in order to increase the probability of selecting the optimal actions it estimates. Because the agent can get high rewards from the start of reinforcement learning, it is expected that the learning rate will increase. The results of a mobile robot simulation showed that the learning rate had increased even though the human expert had showed only a small amount of teaching data. This work was presented in part at the 7th International Symposium on Artificial Life and Robotics, Oita, Japan, January 16–18, 2002     

The memetic self-organizing map approach to the vehicle routing problem

The paper presents an extension of the self- organizing map (SOM) by embedding it into an evolutionary algorithm to solve the Vehicle Routing Problem (VRP). We call it the memetic SOM. The approach is based on the standard SOM algorithm used as a main operator in a population based search. This operator is combined with other derived operators specifically dedicated for greedy insertion moves, a fitness evaluation and a selection operator. The main operators have a similar structure based on the closest point findings and local moves performed in the plane. They can be interpreted as performing parallels and massive insertions, simulating the behavior of agents which interact continuously, having localized and limited abilities. This self-organizing process is intended to allow adaptation to noisy data as well as to confer robustness according to demand fluctuation. Selection is intended to guide the population based search toward useful solution compromises. We show that the approach performs better, with respect to solution quality and/or computation time, than other neural network applications to the VRP presented in the literature. As well, it substantially reduces the gap to classical Operations Research heuristics, specifically on the large VRP instances with time duration constraint.     

Predicting corporate bankruptcy using a self-organizing map: An empirical study to improve the forecasting horizon of a financial failure model

The aim of this study is to show how a Kohonen map can be used to increase the forecasting horizon of a financial failure model. Indeed, most prediction models fail to forecast accurately the occurrence of failure beyond 1 year, and their accuracy tends to fall as the prediction horizon recedes. So we propose a new way of using a Kohonen map to improve model reliability. Our results demonstrate that the generalization error achieved with a Kohonen map remains stable over the period studied, unlike that of other methods, such as discriminant analysis, logistic regression, neural networks and survival analysis, traditionally used for this kind of task.     

Clustering of the self-organizing map using a clustering validity index based on inter-cluster and intra-cluster density

The self-organizing map (SOM) has been widely used in many industrial applications. Classical clustering methods based on the SOM often fail to deliver satisfactory results, specially when clusters have arbitrary shapes. In this paper, through some preprocessing techniques for filtering out noises and outliers, we propose a new two-level SOM-based clustering algorithm using a clustering validity index based on inter-cluster and intra-cluster density. Experimental results on synthetic and real data sets demonstrate that the proposed clustering algorithm is able to cluster data better than the classical clustering algorithms based on the SOM, and find an optimal number of clusters.     

Estimating impervious surfaces from medium spatial resolution imagery using the self-organizing map and multi-layer perceptron neural networks

The studies of impervious surfaces are important because they are related to many environmental problems, such as water quality, stream health, and the urban heat island effect. Previous studies have discussed that the self-organizing map (SOM) can provide a promising alternative to the multi-layer perceptron (MLP) neural networks for image classification at both per-pixel and sub-pixel level. However, the performances of SOM and MLP have not been compared in the estimation and mapping of urban impervious surfaces. In mid-latitude areas, plant phenology has a significant influence on remote sensing of the environment. When the neural networks approaches are applied, how satellite images acquired in different seasons impact impervious surface estimation of various urban surfaces (such as commercial, residential, and suburban/rural areas) remains to be answered. In this paper, an SOM and an MLP neural network were applied to three ASTER images acquired on April 5, 2004, June 16, 2001, and October 3, 2000, respectively, which covered Marion County, Indiana, United States. Six impervious surface maps were yielded, and an accuracy assessment was performed. The root mean square error (RMSE), the mean average error (MAE), and the coefficient of determination (R²) were calculated to indicate the accuracy of impervious surface maps. The results indicated that the SOM can generate a slightly better estimation of impervious surfaces than the MLP. Moreover, the results from three test areas showed that, in the residential areas, more accurate results were yielded by the SOM, which indicates that the SOM was more effective in coping with the mixed pixels than the MLP, because the residential area prevailed with mixed pixels. Results obtained from the commercial area possessed very high RMSE values due to the prevalence of shade, which indicates that both algorithms cannot handle the shade problem well. The lowest RMSE value was obtained from the rural area due to containing of less mixed pixels and shade. This research supports previous observations that the SOM can provide a promising alternative to the MLP neural network. This study also found that the impact of different map sizes on the impervious surface estimation is significant.     

A nonlinear observer design for an activated sludge wastewater treatment process

This paper treats the problem of estimating simultaneously the state and the unknown inputs of a class of nonlinear discrete-time systems. An observer design method for nonlinear Lipschitz discrete-time systems is proposed. By assuming that the linear part of this class of systems is time-varying, the state estimation problem of nonlinear system is transformed into a state estimation problem for LPV system. The stability analysis is performed using a Lyapunov function that leads to the solvability of linear matrix inequalities (LMIs). Performances of the proposed observer are shown through the application to an activated sludge process model.     

Data-derived soft-sensors for biological wastewater treatment plants: An overview

This paper surveys and discusses the application of data-derived soft-sensing techniques in biological wastewater treatment plants. Emphasis is given to an extensive overview of the current status and to the specific challenges and potential that allow for an effective application of these soft-sensors in full-scale scenarios. The soft-sensors presented in the case studies have been found to be effective and inexpensive technologies for extracting and modelling relevant process information directly from the process and laboratory data routinely acquired in biological wastewater treatment facilities. The extracted information is in the form of timely analysis of hard-to-measure primary process variables and process diagnostics that characterize the operation of the plants and their instrumentation. The information is invaluable for an effective utilization of advanced control and optimization strategies.     

A self-organizing map for transactional data and the related categorical domain

After projecting high dimensional data into a two-dimension map via the SOM, users can easily view the inner structure of the data on the 2-D map. In the early stage of data mining, it is useful for any kind of data to inspect their inner structure. However, few studies apply the SOM to transactional data and the related categorical domain, which are usually accompanied with concept hierarchies. Concept hierarchies contain information about the data but are almost ignored in such researches. This may cause mistakes in mapping. In this paper, we propose an extended SOM model, the SOMCD, which can map the varied kinds of data in the categorical domain into a 2-D map and visualize the inner structure on the map. By using tree structures to represent the different kinds of data objects and the neurons’ prototypes, a new devised distance measure which takes information embedded in concept hierarchies into consideration can properly find the similarity between the data objects and the neurons. Besides the distance measure, we base the SOMCD on a tree-growing adaptation method and integrate the U-Matrix for visualization. Users can hierarchically separate the trained neurons on the SOMCD's map into different groups and cluster the data objects eventually. From the experiments in synthetic and real datasets, the SOMCD performs better than other SOM variants and clustering algorithms in visualization, mapping and clustering.     

An AI approach for wastewater treatment systems

Most wastewaters consist of several contaminants (compounds) that need to be removed during the treatment process. A treatability database has been developed containing the treatability of various compounds through different types of treatment processes. In most wastewaters several compounds appear together and two or more treatment processes in series may be needed to meet the effluent limits of the contaminants. The proposed AI wastewater treatment system consists of two phases, analysis phase and synthesis phase. In the analysis phase, an inductive learning algorithm with a grammar based knowledge representation is used to extract knowledge rules from the database. These rules are combined with another set of rules obtained from the experts. All these rules are arranged together to identify the effect of an individual treatment process on several compounds at various concentrations. In the synthesis phase, knowledge rules generated from the analysis phase are used to obtain the sequence of technologies that can satisfy the necessary treatment constraints. Two different methodologies are developed to generate the sequence of technologies. In the first approach, the synthesis phase is formulated as a search problem and a heuristic search function is developed. In the second approach, the synthesis phase is formulated as an optimization problem and a Hopfield neural network is used to obtain the sequence of technologies. Both approaches are compared for the optimality of the solution and the processing time required.Artificial Intelligence & Computer Vision Laboratory, University of CincinnatiDept of Civil & Environmental Engineering, University of Cincinnati     

Modeling and simulation of the open-end equity mutual fund market in Taiwan by using self-organizing map

This research applies artificial intelligence (AI) of unsupervised learning self-organizing map neural network (SOM-NN) to establish a model to select the superior funds. This research period is from year 2000 to 2010 and picks 100 domestic equity mutual funds as study object. This research used 30 days prior to the beginning of each month’s prior 30 days, 60 days, 90 days on fund’s net asset value and the Taiwan Weighted Stock Index (TAIEX) return as the fund’s relative performance evaluation indicators classified by month. Finally, based on the superior rate or the average return rate, this research select the superior funds and simulate investment transactions according to this model.The empirical results show that using the mutual fund’s net asset value and the TAIEX’s relative return as SOM-NN input variables not only finds out the superior fund but also has a good predictive ability. Applying this model to simulate investment transactions will be better than the random trading model and market. The experiments also found that the investment simulation of a three-month interval has the highest profitability. The model operation suggests that it is more suitable for short-term and medium-term investment. This research can assist investors in making the right investment decisions while facing rapid financial environment changes.     

A nonlinear observer for activated sludge wastewater treatment process: Invariant observer

An activated sludge wastewater treatment process model is concerned in this paper. In order to estimate the variables that cannot be measured online, an invariant observer for activated sludge wastewater treatment process is presented. The invariant observer can measure biomass concentration, substrate concentration and dissolved oxygen concentration in high accuracy and rapidity. Meanwhile it can be structured by means of typical form, and its robust convergence property is verified by theoretical analysis and numerical simulations (MATLAB ).     

An integrated modelling approach for the assessment of land use change effects on wastewater infrastructures

The simulation of sewage systems and wastewater treatment plants is strategic for assessing the effect of new dwellings on the existing water facilities. This paper introduces an integrated framework made by a land use change model, a sewage system simulator, and a wastewater treatment plant simulator. This is a complex system since each element is characterized by different dynamics. The land use change model simulates the annual expansion of an urban area according to planners’ guidelines; the sewage system simulator investigates the response of the drainage system to the expansion. The wastewater treatment plant is simulated in order to assess the impact of the new outflows on the existing plant. The three models are integrated into a Simulink model. Two components of the developed framework are based on models well established in literature. The proposed framework is tested on a simple case study of a small town located in south west of Scotland.     

An emergent approach for the control of wastewater treatment plants by means of reinforcement learning techniques

One of the main problems in the automation of the control of wastewater treatment plants (WWTPs) appears when the control system does not respond as it should because of changes on influent load or flow. To tackle this difficult task, the application of Artificial Intelligence is not new, and in fact, currently Expert Systems may supervise the plant 24 h/day assisting the plant operators in their daily work. However, the knowledge of the Expert System must be elicited previously from interviews to plant operators and/or extracted from data previously stored in databases. Although this approach still has a place in the control of wastewater treatment plants, it should aim to develop autonomous systems that learn from the direct interaction with the WWTP and that can operate taking into account changing environmental circumstances. In this paper we present an approach based on an agent with learning capabilities. In this approach, the agent’s knowledge emerges from the interaction with the plant. In order to show the validity of our assertions, we have implemented such an emergent approach for the N-Ammonia removal process in a well established simulated WWTP known as Benchmark Simulation Model No.1 (BSM1).     

Advanced decision control system for effluent violations removal in wastewater treatment plants

This paper presents the application of control strategies for wastewater treatment plants with the goal of effluent limits violations removal as well as achieving a simultaneous improvement of effluent quality and reduction of operational costs. The evaluation is carried out with the Benchmark Simulation Model No. 2. The automatic selection of the suitable control strategy is based on risk detection of effluent violations by Artificial Neural Networks. Fuzzy Controllers are implemented to improve the denitrification or nitrification process based on the proposed objectives. Model Predictive Control is applied for the improvement of dissolved oxygen tracking.