Machine learning-based distributed model predictive control of nonlinear processes

This work explores the design of distributed model predictive control (DMPC) systems for nonlinear processes using machine learning models to predict nonlinear dynamic behavior. Specifically, sequential and iterative DMPC systems are designed and analyzed with respect to closed-loop stability and performance properties. Extensive open-loop data within a desired operating region are used to develop long short-term memory (LSTM) recurrent neural network models with a sufficiently small modeling error from the actual nonlinear process model. Subsequently, these LSTM models are utilized in Lyapunov-based DMPC to achieve efficient real-time computation time while ensuring closed-loop state boundedness and convergence to the origin. Using a nonlinear chemical process network example, the simulation results demonstrate the improved computational efficiency when the process is operated under sequential and iterative DMPCs while the closed-loop performance is very close to the one of a centralized MPC system.     

Machine learning-based predictive control of nonlinear processes. Part I: Theory

This article focuses on the design of model predictive control (MPC) systems for nonlinear processes that utilize an ensemble of recurrent neural network (RNN) models to predict nonlinear dynamics. Specifically, RNN models are initially developed based on a data set generated from extensive open-loop simulations within a desired process operation region to capture process dynamics with a sufficiently small modeling error between the RNN model and the actual nonlinear process model. Subsequently, Lyapunov-based MPC (LMPC) that utilizes RNN models as the prediction model is developed to achieve closed-loop state boundedness and convergence to the origin. Additionally, machine learning ensemble regression modeling tools are employed in the formulation of LMPC to improve prediction accuracy of RNN models and overall closed-loop performance while parallel computing is utilized to reduce computation time. Computational implementation of the method and application to a chemical reactor example is discussed in the second article of this series.     

Online learning-based predictive control of crystallization processes under batch-to-batch parametric drift

This work considers a seeded fesoterodine fumarate (FF) cooling crystallization and presents the methodology and implementation of a real-time machine learning modeling-based predictive controller to handle batch-to-batch (B2B) parametric drift. Specifically, an autoencoder recurrent neural network-based model predictive controller (AERNN-MPC) is developed to optimize product yield, crystal size, and energy consumption while accounting for the physical constraints on cooling jacket temperature. Deviations in the kinetic parameters are considered in the closed-loop simulations to account for the B2B parametric drift, and two error-triggered online update mechanisms are proposed to address issues pertaining to the availability of real-time crystal property measurements and are incorporated into the AERNN-MPC to improve the model prediction accuracy. Closed-loop simulation results demonstrate that the proposed AERNN-MPC with online update, irrespective of the accessibility to real-time crystal property data, achieves a desired closed-loop performance in terms of maximizing product yield and minimizing energy consumption.     

Online machine learning modeling and predictive control of nonlinear systems with scheduled mode transitions

This work develops a model predictive control (MPC) scheme using online learning of recurrent neural network (RNN) models for nonlinear systems switched between multiple operating regions following a prescribed switching schedule. Specifically, an RNN model is initially developed offline to model process dynamics using the historical operational data collected in a small region around a certain steady-state. After the system is switched to another operating region under a Lyapunov-based MPC with suitable constraints to ensure satisfaction of the prescribed switching schedule policy, RNN models are updated using real-time process data to improve closed-loop performance. A generalization error bound is derived for the updated RNN models using the notion of regret, and closed-loop stability results are established for the switched nonlinear system under RNN-based MPC. Finally, a chemical process example with the operation schedule that requires switching between two steady-states is used to demonstrate the effectiveness of the proposed RNN-MPC scheme.     

Modeling and predictive control of nonlinear processes using transfer learning method

This work develops a transfer learning (TL) framework for modeling and predictive control of nonlinear systems using recurrent neural networks (RNNs) with the knowledge obtained in modeling one process transferred to another. Specifically, transfer learning uses a pretrained model developed based on a source domain as the starting point, and adapts the model to a target process with similar configurations. The generalization error for TL-based RNN (TL-RNN) is first derived to demonstrate the generalization capability on the target process. The theoretical error bound that depends on model capacity and the discrepancy between source and target domains is then utilized to guide the development of pretrained models for improved model transferability. Subsequently, the TL-RNN model is utilized as the prediction model in model predictive controller (MPC) for the target process. Finally, the simulation study of chemical reactors via Aspen Plus Dynamics is used to demonstrate the benefits of transfer learning.     

Explicit model predictive control of hybrid systems and multiparametric mixed integer polynomial programming

Hybrid systems are dynamical systems characterized by the simultaneous presence of discrete and continuous variables. Model‐based control of such systems is computationally demanding. To this effect, explicit controllers which provide control inputs as a set of functions of the state variables have been derived, using multiparametric programming mainly for the linear systems. Hybrid polynomial systems are considered resulting in a Mixed Integer Polynomial Programming problem. Treating the initial state of the system as a set of bounded parameters, the problem is reformulated as a multiparametric Mixed Integer Polynomial optimization (mp‐MIPOPT) problem. A novel algorithm for mp‐MIPOPT problems is proposed and the exact explicit control law for polynomial hybrid systems is computed. The key idea is the computation of the analytical solution of the optimality conditions while the binary variables are treated as relaxed parameters. Finally, using symbolic calculations exact nonconvex critical regions are computed. © 2016 The Authors AIChE Journal published by Wiley Periodicals, Inc. on behalf of American Institute of Chemical Engineers AIChE J, 62: 3441–3460, 2016     

A dynamic programming based approach for explicit model predictive control of hybrid systems

This work presents an algorithm for explicit model predictive control of hybrid systems based on recent developments in constrained dynamic programming and multi-parametric programming. By using the proposed approach, suitable for problems with linear cost function, the original model predictive control formulation is disassembled into a set of smaller problems, which can be efficiently solved using multi-parametric mixed-integer programming algorithms. It is also shown how the methodology is applied in the context of explicit robust model predictive control of hybrid systems, where model uncertainty is taken into account. The proposed developments are demonstrated through a numerical example where the methodology is applied to the optimal control of a piece-wise affine system with linear cost function.     

An integrated approach for machine-learning-based system identification of dynamical systems under control: application towards the model predictive control of a highly nonlinear reactor system

Advanced model-based control strategies,e.g.,model predictive control,can offer superior control of key process variables for multiple-input multiple-output systems.The quality of the system model is critical to controller performance and should adequately describe the process dynamics across its operating range while remaining amenable to fast optimization.This work articulates an integrated system identification procedure for deriving black-box nonlinear continuous-time multiple-input multiple-output system models for nonlinear model predictive control.To showcase this approach,five candidate models for polynomial and interaction features of both output and manipulated variables were trained on simulated data and integrated into a nonlinear model predictive controller for a highly nonlinear continuous stirred tank reactor system.This procedure successfully identified system models that enabled effective control in both servo and regulator problems across wider operating ranges.These controllers also had reasonable per-iteration times of ca.0.1 s.This demonstration of how such system models could be identified for nonlinear model predictive control without prior knowledge of system dynamics opens further possibilities for direct data-driven methodologies for model-based control which,in the face of process uncertainties or modelling limitations,allow rapid and stable control over wider operating ranges.     

Some manifold learning considerations toward explicit model predictive control

Model predictive control (MPC) is a de facto standard control algorithm across the process industries. There remain, however, applications where MPC is impractical because an optimization problem is solved at each time step. We present a link between explicit MPC formulations and manifold learning to enable facilitated prediction of the MPC policy. Our method uses a similarity measure informed by control policies and system state variables, to “learn” an intrinsic parametrization of the MPC controller using a diffusion maps algorithm, which will also discover a low-dimensional control law when it exists as a smooth, nonlinear combination of the state variables. We use function approximation algorithms to project points from state space to the intrinsic space, and from the intrinsic space to policy space. The approach is illustrated first by “learning” the intrinsic variables for MPC control of constrained linear systems, and then by designing controllers for an unstable nonlinear reactor.     

Machine-learning-based construction of barrier functions and models for safe model predictive control

In this paper, we propose a control Lyapunov-barrier function-based model predictive control method utilizing a feed-forward neural network specified control barrier function (CBF) and a recurrent neural network (RNN) predictive model to stabilize nonlinear processes with input constraints, and to guarantee that safety requirements are met for all times. The nonlinear system is first modeled using RNN techniques, and a CBF is characterized by constructing a feed-forward neural network (FNN) model with unique structures and properties. The FNN model for the CBF is trained based on data samples collected from safe and unsafe operating regions, and the resulting FNN model is verified to demonstrate that the safety properties of the CBF are satisfied. Given sufficiently small bounded modeling errors for both the FNN and the RNN models, the proposed control system is able to guarantee closed-loop stability while preventing the closed-loop states from entering unsafe regions in state-space under sample-and-hold control action implementation. We provide the theoretical analysis for bounded unsafe sets in state-space, and demonstrate the effectiveness of the proposed control strategy using a nonlinear chemical process example with a bounded unsafe region.     

Statistical machine-learning–based predictive control of uncertain nonlinear processes

In this study, we present machine-learning–based predictive control schemes for nonlinear processes subject to disturbances, and establish closed-loop system stability properties using statistical machine learning theory. Specifically, we derive a generalization error bound via Rademacher complexity method for the recurrent neural networks (RNN) that are developed to capture the dynamics of the nominal system. Then, the RNN models are incorporated in Lyapunov-based model predictive controllers, under which we study closed-loop stability properties for the nonlinear systems subject to two types of disturbances: bounded disturbances and stochastic disturbances with unbounded variation. A chemical reactor example is used to demonstrate the implementation and evaluate the performance of the proposed approach.     

Integration of reinforcement learning and model predictive control to optimize semi-batch bioreactor

As the digital transformation of the bioprocess is progressing, several studies propose to apply data-based methods to obtain a substrate feeding strategy that minimizes the operating cost of a semi-batch bioreactor. However, the negligent application of model-free reinforcement learning (RL) has a high chance to fail on improving the existing control policy because the available amount of data is limited. In this article, we propose an integrated algorithm of double-deep Q-network and model predictive control. The proposed method learns the action-value function in an off-policy fashion and solves the model-based optimal control problem where the terminal cost is assigned by the action-value function. For simulation study, the proposed method, model-based method, and model-free methods are applied to the industrial scale penicillin process. The results show that the proposed method outperforms other methods, and it can learn with fewer data than model-free RL algorithms.     

Arma neuron networks for modeling nonlinear dynamical systems

A new neuronal structure, the ARMA neuron, is proposed here. These new neurons are designed for modeling nonlinear dynamics often encountered in chemical engineering processes. They are an extension of standard neurons which are used for static process modeling. These new neurons contain internal input/output dynamic structure and can model dynamic non-linearities in a flexible manner. A nonlinear output transformation is used here as opposed to a linear version used earlier (Krishnapura and Jutan, 1993). New algorithms for training networks comprised of the new ARMA neurons are developed using the backpropagation approach. The ARMA neurons are used to model both simulated and experimental nonlinear dynamic processes, including an industrial fluidized bed reactor.     

Iterative learning model predictive control for constrained multivariable control of batch processes

In this paper, we propose a model predictive control (MPC) technique combined with iterative learning control (ILC), called the iterative learning model predictive control (ILMPC), for constrained multivariable control of batch processes. Although the general ILC makes the outputs converge to reference trajectories under model uncertainty, it uses open-loop control within a batch; thus, it cannot reject real-time disturbances. The MPC algorithm shows identical performance for all batches, and it highly depends on model quality because it does not use previous batch information. We integrate the advantages of the two algorithms. The proposed ILMPC formulation is based on general MPC and incorporates an iterative learning function into MPC. Thus, it is easy to handle various issues for which the general MPC is suitable, such as constraints, time-varying systems, disturbances, and stochastic characteristics. Simulation examples are provided to show the effectiveness of the proposed ILMPC.     

从区间模型预测控制到双层结构模型预测控制

模型预测控制算法（MPC）存在设定点控制与区间控制两种策略,区间预测控制较之设定点控制在技术上具有先进性。目前,主流的预测控制软件技术均采用双层结构,即上层稳态优化计算最优设定点,下层动态控制负责动态跟踪最优设定点。从过程稳态的角度出发,分别对区间预测控制和双层结构预测控制进行了机理分析,从定性与定量两个方面比较了这两者的异同点,提出并证明了两者的一致性条件。论述了双层结构预测控制较之单层结构下的区间控制更具先进性。     

Sequential and iterative architectures for distributed model predictive control of nonlinear process systems

In this work, we focus on distributed model predictive control of large scale nonlinear process systems in which several distinct sets of manipulated inputs are used to regulate the process. For each set of manipulated inputs, a different model predictive controller is used to compute the control actions, which is able to communicate with the rest of the controllers in making its decisions. Under the assumption that feedback of the state of the process is available to all the distributed controllers at each sampling time and a model of the plant is available, we propose two different distributed model predictive control architectures. In the first architecture, the distributed controllers use a one‐directional communication strategy, are evaluated in sequence and each controller is evaluated only once at each sampling time; in the second architecture, the distributed controllers utilize a bi‐directional communication strategy, are evaluated in parallel and iterate to improve closed‐loop performance. In the design of the distributed model predictive controllers, Lyapunov‐based model predictive control techniques are used. To ensure the stability of the closed‐loop system, each model predictive controller in both architectures incorporates a stability constraint which is based on a suitable Lyapunov‐based controller. We prove that the proposed distributed model predictive control architectures enforce practical stability in the closed‐loop system and optimal performance. The theoretical results are illustrated through a catalytic alkylation of benzene process example. © 2010 American Institute of Chemical Engineers AIChE J, 2010     

Real‐time economic model predictive control of nonlinear process systems

Closed‐loop stability of nonlinear systems under real‐time Lyapunov‐based economic model predictive control (LEMPC) with potentially unknown and time‐varying computational delay is considered. To address guaranteed closed‐loop stability (in the sense of boundedness of the closed‐loop state in a compact state‐space set), an implementation strategy is proposed which features a triggered evaluation of the LEMPC optimization problem to compute an input trajectory over a finite‐time prediction horizon in advance. At each sampling period, stability conditions must be satisfied for the precomputed LEMPC control action to be applied to the closed‐loop system. If the stability conditions are not satisfied, a backup explicit stabilizing controller is applied over the sampling period. Closed‐loop stability under the real‐time LEMPC strategy is analyzed and specific stability conditions are derived. The real‐time LEMPC scheme is applied to a chemical process network example to demonstrate closed‐loop stability and closed‐loop economic performance improvement over that achieved for operation at the economically optimal steady state. © 2014 American Institute of Chemical Engineers AIChE J, 61: 555–571, 2015