Machine learning to assist filtered two-fluid model development for dense gas–particle flows

Machine learning (ML) is experiencing an immensely fascinating resurgence in a wide variety of fields. However, applying such powerful ML to construct subgrid interphase closures has been rarely reported. To this end, we develop two data-driven ML strategies (i.e., artificial neural networks and eXtreme gradient boosting) to accurately predict filtered subgrid drag corrections using big data from highly resolved simulations of gas-particle fluidization. Quantitative assessments of effects of various subgrid input markers on training prediction outputs are performed and three-marker choice is demonstrated to be the optimal one for predicting the unseen test set. We then develop a parallel data loader to integrate this predictive ML model into a computational fluid dynamic (CFD) framework. Subsequent coarse-grid simulations agree fairly well with experiments regarding the underlying hydrodynamics in bubbling and turbulent fluidized beds. The present ML approach provides easily extended ways to facilitate the development of predictive models for multiphase flows.     

Soft Computing of Biochemical Oxygen Demand Using an Improved T–S Fuzzy Neural Network

It is difficult to measure the online values of biochemical oxygen demand (BOD) due to the characteristics of nonlinear dynamics, large lag and uncertainty in wastewater treatment process. In this paper, based on the knowledge representation ability and learning capability, an improved T–S fuzzy neural network (TSFNN) is introduced to predict BOD values by the soft computing method. In this improved TSFNN, a K-means clustering is used to initialize the structure of TSFNN, including the number of fuzzy rules and parameters of membership function. For training TSFNN, a gradient descent method with the momentum item is used to adjust antecedent parameters and consequent parameters. This improved TSFNN is applied to predict the BOD values in effluent of the wastewater treatment process. The simulation results show that the TSFNN with K-means clustering algorithm can measure the BOD values accurately. The algorithm presents better approximation performance than some other methods.     

A hybrid science-guided machine learning approach for modeling chemical processes: A review

This study presents a broad perspective of hybrid process modeling combining the scientific knowledge and data analytics in bioprocessing and chemical engineering with a science-guided machine learning (SGML) approach. We divide the approach into two major categories: ML complements science, and science complements ML. We review the literature relating to the hybrid SGML approach, and propose a systematic classification of hybrid SGML models. For applying ML to improve science-based models, we present expositions of direct serial and parallel hybrid modeling and their combinations, inverse modeling, reduced-order modeling, quantifying uncertainty in the process and even discovering governing equations of the process model. For applying scientific principles to improve ML models, we discuss the science-guided design, learning and refinement. For each subcategory, we identify its requirements, strengths, and limitations, together with their published and potential applications. We also present several examples to illustrate different hybrid SGML methodologies for modeling chemical processes.     

An approach to multicriteria optimization under uncertainty

Inherent in chemical process models are parameters that have uncertainty associated with them. This paper addresses multicriteria optimization that accounts for model and process uncertainty at the design stage. Specifically the authors have developed extensions of the average criterion method, the worst-case strategy and the ε-constraint method under the following conditions: (a) at the design stage the only information available about the uncertain parameters is that they are bounded by a known uncertainty region T, and (b) at the operation stage, process data is rich enough to allow the determination of exact values of all the uncertain parameters. The suggested formulation assumes that at the operation stage, certain process variables (called control variables) can be tuned or manipulated in order to offset the effects of uncertainty. Three illustrative examples (two benchmark and one direct methanol fuel cell) have been employed.     

Improving prediction capabilities of complex dynamic models via parameter selection and estimation

Many mathematical models describing complex (bio-)chemical reaction networks with a high level of detail are available in the literature. While such detailed models are desirable for investigating the dynamics of a particular component, it is often questionable if it is possible to verify these models due to the limited amount of quantitative data that can be generated. Even if it is not possible to estimate all parameters of these models then one would still be interested in determining which parameters should be estimated.This paper addresses this point and introduces a technique for determining the parameters of a model that should be estimated from experimental data. The focus of this work is on ensuring that the model has good prediction capability as over-fitting the model to noisy experimental data is avoided. Towards this goal, a forward selection approach for selecting a subset of parameters for estimation while taking uncertainty into account is developed to minimize the mean squared prediction error of the model. It is shown that the developed technique is closely related to the often-used orthogonalization method. The technique is applied to a model of the NF-κB signal transduction pathway. The presented method is able to generate a smaller mean squared error than estimation of all parameters and also outperforms the orthogonalization method.     

Finite element-based machine learning approach for optimization of process parameters to produce silicon carbide ceramic complex parts

Design and fabrication of silicon carbide ceramic complex parts introduce considerable difficulties during injection molding. Due to the great importance in processing optimization, an accurate prediction on the stress and displacement is required to obtain the desired final product. In this paper, a conceptual framework on combination of finite element method (FEM) and machine learning (ML) method was developed to optimize the injection molding process, which can be used to manufacture large-aperture silicon carbide mirror. The distribution characteristics of temperature field and stress field were extracted from FEM simulation to understand the injection molding process and construct database for ML modeling. To select the most appropriate model, the predictive performance of three ML models were estimated, including generalized regression neural network (GRNN), back propagation neural network (BPNN) and extreme learning machine (ELM). The results show that the developed ELM model exhibits exceptional predictive performance and can be utilized to predict the stress and displacement of the green body. This work allows us to obtain reasonable technique parameters with particular attention to the loading speed and provides some fundamental guidance for the fabrication of lightweight SiC ceramic optical mirror.     

Parameter optimization for the Gaussian model of protein folding

Computational models of protein folding and ligand docking are large and complex. Few systematic methods have yet been developed to optimize the parameters in such models. We describe here an iterative parameter optimization strategy that is based on minimizing a structural error measure by descent in parameter space. At the start, we know the ‘correct’ native structure that we want the model to produce, and an initial set of parameters representing the relative strengths of interactions between the amino acids. The parameters are changed systematically until the model native structure converges as closely as possible to the correct native structure. As a test, we apply this parameter optimization method to the recently developed Gaussian model of protein folding: each amino acid is represented as a bead and all bonds, covalent and noncovalent, are represented by Hooke's law springs. We show that even though the Gaussian model has continuous degrees of freedom, parameters can be chosen to cause its ground state to be identical to that of Go-type lattice models, for which the global ground states are known. Parameters for a more realistic protein model can also be obtained to produce structures close to the real native structures in the protein database.     

Prediction of bio-oil yield during pyrolysis of lignocellulosic biomass using machine learning algorithms

This work aims to implement and use machine learning algorithms to predict the yield of bio-oil during the pyrolysis of lignocellulosic biomass based on the physicochemical properties and composition of the biomass feed and pyrolysis conditions. The biomass pyrolysis process is influenced by different process parameters, such as pyrolysis temperature, heating rate, composition of biomass, and purge gas flow rate. The inter-relation between the yield of different pyrolysis products and process parameters can be well predicted by using different machine learning algorithms. In this study, different machine learning algorithms, namely, multi-linear regression, gradient boosting, random forest, and decision tree, have been trained on the dataset and the models are compared to identify the optimum method for the determination of bio-oil yield prediction model. Analysis of the results showed the gradient boosting method to possess a regression score of 0.97 and 0.89 for the training and testing sets with root-mean-squared error (RMSE) values of 1.19 and 2.39, respectively, and overcome the problem of overfitting. Therefore, the present study provides an approach to train a generalized machine learning model, which can be employed on large datasets while avoiding the error of overfitting.     

Flexibility index of black-box models with parameter uncertainty through derivative-free optimization

The existing methods of flexibility index are mainly based on mixed-integer linear or nonlinear programming methods, making it difficult to readily deal with complex mathematical models. In this article, a novel solution strategy is proposed for finding a reliable upper bound of the flexibility index where the process model is implemented in a black box that can be directly executed by a commercial simulator, and also avoiding the need for calculating derivatives. Then, the flexibility index problem is formulated as a sequence of univariate derivative-free optimization (DFO) models. An external DFO solver based on trust-region methods can be called to solve this model. Finally, after calculating the critical point of the model parameters, the vertex enumeration method and two gradient approximation methods are proposed to evaluate the impact of process parameters and to evaluate the flexibility index. A reaction model is studied to show the efficiency of the proposed algorithm.     

Tests for conditional heteroscedasticity of functional data

Functional data objects derived from high-frequency financial data often exhibit volatility clustering. Versions of functional generalized autoregressive conditionally heteroscedastic (FGARCH) models have recently been proposed to describe such data, however so far basic diagnostic tests for these models are not available. We propose two portmanteau type tests to measure conditional heteroscedasticity in the squares of asset return curves. A complete asymptotic theory is provided for each test. We also show how such tests can be adapted and applied to model residuals to evaluate adequacy, and inform order selection, of FGARCH models. Simulation results show that both tests have good size and power to detect conditional heteroscedasticity and model mis-specification in finite samples. In an application, the tests show that intra-day asset return curves exhibit conditional heteroscedasticity. This conditional heteroscedasticity cannot be explained by the magnitude of inter-daily returns alone, but it can be adequately modeled by an FGARCH(1,1) model.     

Maximum likelihood estimation in space time bilinear models

The space time bilinear (STBL) model is a special form of a multiple bilinear time series that can be used to model time series which exhibit bilinear behaviour on a spatial neighbourhood structure. The STBL model and its identification have been proposed and discussed by Dai and Billard (1998 ). The present work considers the problem of parameter estimation for the STBL model. A conditional maximum likelihood estimation procedure is provided through the use of a Newton–Raphson numerical optimization algorithm. The gradient vector and Hessian matrix are derived together with recursive equations for computation implementation. The methodology is illustrated with two simulated data sets, and one real-life data set.     

动态RBF神经网络在浮选过程模型失配中的应用

铝土矿泡沫浮选过程中,因矿浆的快速沉淀等原因工艺参数在线检测困难,且入矿性质变化频繁,造成浮选过程参数随入矿的变化而不断改变。而通常建立的静态软测量模型利用固定样本集训练得到,当矿源变化时容易发生模型失配现象,使模型不能跟踪当前对象。针对变矿源下的模型失配问题,本文提出基于隐层节点动态分配和模型参数动态修正策略的RBF神经网络建模方法,用于铝土矿浮选过程酸碱度的在线检测建模。实际生产数据仿真结果表明该方法能够有效解决模型失配的问题。     

Comparing the bias and misspecification in ARFIMA models

We investigate the bias in both the short-term and long-term parameters for a range of autoregressive fractional integrated moving-average (ARFIMA) models using both semi-parametric and maximum likelihood (ML) estimation methods. The results suggest that, provided the correct model is estimated, the ML method outperforms the semi-parametric methods in terms of the bias and smaller mean square errors in both the long-term and short-term parameter estimates. These biases often cause model selection criteria to select an incorrect ARFIMA specification. Taking account of the potential misspecification the biases associated with the ML procedure tend to increase, although it continues to have a smaller worst-case bias than either of the semi-parametric procedures.     

生态平衡施肥：Ⅱ．施肥参数指标体系

介绍如何确定通用施肥模型的特征参数.研究表明,任何有效的田间肥料试验结果和测土结果都可以作为确定通用施肥模型特征参数的原始数据,在没有上述数据情况下,也可以采取专家经验、访问和调查数据的方法确定施肥特征参数.在此基础上,进一步评述了常见几种施肥模型的优缺点及通用施肥模型与其关系.     

Time Series Models in Non-Normal Situations: Symmetric Innovations

We consider AR( q ) models in time series with non-normal innovations represented by a member of a wide family of symmetric distributions (Student's t ). Since the ML (maximum likelihood) estimators are intractable, we derive the MML (modified maximum likelihood) estimators of the parameters and show that they are remarkably efficient. We use these estimators for hypothesis testing, and show that the resulting tests are robust and powerful.     

Explaining and Integrating Machine Learning Models with Rigorous Simulation

First-principle flowsheet simulation is a reliable data resource for training machine learning (ML) models for process industry, especially when plant data is not available. Process simulators play an even more important role for evaluation and validation of ML models. In this work, we present a workflow for building and evaluating ML models based on data generated by a commercial flowsheet simulator. The resulting hybrid models, combining data-driven predictions with mass and energy balances, have much lower calculation times than the rigorous models. The implementation of such models shows great potential for solving more complex process engineering problems on the high-dimensional space in the future, while saving the process engineer's time in the present.     

Electrochemical modeling of intercalation processes with phase field models

In this paper, we apply phase field models to move beyond Fick’s law in describing Li diffusion in secondary battery electrodes. Phase field models are potentially more accurate and allow simpler tracking of phase boundaries than Fick’s equation. The phase field models are implemented using the highly accurate but fast Chebyshev-spectral method. Using the phase field we investigate to what extent non-Fickian behavior can affect results from experimental techniques for measuring diffusion coefficients, such as Galvanostatic Intermittent Titration Technique (GITT) and Potentiostatic Intermittent Titration Technique (PITT). We show that GITT and PITT can still accurately measure the diffusion coefficient in systems described by phase field models even when significant gradient energy terms are present.     

Conceptual design and optimization of chemical processes under uncertainty by two-stage programming

This contribution presents a method and a tool for modelling and optimizing process superstructures in the early phase of process design where the models of the processing units and other data are inaccurate. To adequately deal with this uncertainty, we employ a two-stage formulation where the operational parameters can be adapted to the realization of the uncertainty while the design parameters are the first-stage decisions. The uncertainty is represented by a set of discrete scenarios and the optimization problem is solved by stage decomposition. The approach is implemented in the computer tool FSOpt (Flow sheet Superstructure Optimization) FSOpt provides a flexible environment for the modelling of the unit operations and the generation of superstructures and algorithms for the translation of the superstructure into non-linear programming models.The approach is applied to two case studies, the hydroformylation of dodec-1-ene and the separation of an azeotropic mixture of water and formic acid.     

Estimating vibrational and thermodynamic properties of adsorbates with uncertainty using data driven surrogates

In this work, we propose a strategy to develop data driven local surrogate models of ab initio potential energy functions describing the interaction of adsorbates on heterogeneous catalytic materials. We show that these multivariable surrogate models, based on orthogonal polynomial expansion and trained on sampled ab-initio energies/forces, can be used to compute harmonic vibrational frequencies and the entropy of adsorbates. Further, we show that the errors in our surrogate model can be estimated and propagated to calculate the uncertainty in the computed properties. We show proof-of-concept illustrations of our method to calculate the vibrational frequencies of ethene on 1D edges of molybdenum sulfide (MoS₂), (b) 2D surfaces of Pt(111), and (c) 3D micropores of a HZSM-5 zeolite; the entropy of ethane adsorbed on Pt(111); and the associated uncertainties in all the cases.