A hybrid science-guided machine learning approach for modeling chemical processes: A review

This study presents a broad perspective of hybrid process modeling combining the scientific knowledge and data analytics in bioprocessing and chemical engineering with a science-guided machine learning (SGML) approach. We divide the approach into two major categories: ML complements science, and science complements ML. We review the literature relating to the hybrid SGML approach, and propose a systematic classification of hybrid SGML models. For applying ML to improve science-based models, we present expositions of direct serial and parallel hybrid modeling and their combinations, inverse modeling, reduced-order modeling, quantifying uncertainty in the process and even discovering governing equations of the process model. For applying scientific principles to improve ML models, we discuss the science-guided design, learning and refinement. For each subcategory, we identify its requirements, strengths, and limitations, together with their published and potential applications. We also present several examples to illustrate different hybrid SGML methodologies for modeling chemical processes.     

Machine learning modelling and evaluation of jet fires from natural gas processing,storage, and transport

Jet fires and their repercussions play a significant role in catastrophic incidents that typically have a cascading impact in process industries. Several hydrocarbon experiments from 19 papers were incorporated into the current endeavour to develop simulations of jet flames using machine learning (ML) models. Dimensionless characteristics have been used as output and input variables, including mass flow rates, fuel density, jet flame length, and heat release fluxes. When training three layers of the multi-layer feedforward neural network (MLFFNN) method, a Bayesian regularization backpropagation approach was adopted and evaluated with the radial based functions (RBF) algorithm. Through an optimization procedure, the first and second hidden layers of the MLFFNN have been optimized to include 10 and five neurons, respectively. The RBF algorithm with 40 neurons in a single layer has been computed using the same method. The best mean square error (MSE) validation results for RBF and MLFFNN were 0.006 and 0.0002, respectively, for 40 and 100 epochs. The MLFFNN and RBF models' respective regression statistical analysis outputs were 0.9949 and 0.9645. The ML method has been identified as a potentially useful technique for precisely predicting the geometrical and radiative characteristics of jet flames.     

Estimating performance enhancement with alternate control strategies for multiloop control systems

A filter based methodology, studied earlier for SISO systems, is extended to MIMO systems. The presented approach facilitates the calculation of best achievable performance for proportional-integral (PI) controller and the optimal multiloop (ML) PI settings for stochastic disturbance rejection in ML control systems. The filter based approach is further extended to answer some of the key questions for ML control systems such as: (a) performance enhancement possible with the alternate pairing scheme, (b) benefits that will accrue through the employment of decouplers and (c) the performance achievable with the use of multivariable controller (as opposed to an ML controller). Further, the trade-off curve between output variance and control effort is generated for the various control configurations within PI controller domain.     

Reactions between peroxidizing lipids and histidyl residue analogues: Enhancement of lipid oxidation and browning by 4-methylimidazole

As a part of our study on the interactions between peroxidizing lipids and the histidyl imidazole side-chain in simple, low-moisture model systems, 4-methylimidazole (4MI) was reacted with methyl linoleate (ML). This analogue was chosen to avoid interference from other functional groups in histidine (free base) or in proteins. Changes in the concentrations of lipid hydroperoxides, carbonyls, 4MI, and brown pigments were followed over a period of 24 days. The results indicate that 4MI exhibits significant prooxidative activity by reducing the induction period as well as by enhancing the formation of brown pigments. These effects are more pronounced at high 4MI/ML molar ratios and under basic pH’s. Upon interactions with peroxidizing ML, as much as 44% of initially present 4MI was destroyed by the sixth day of incubation. Presented at the AOCS 68th Annual Meeting, New York City, May 1977.     

Explaining and Integrating Machine Learning Models with Rigorous Simulation

First-principle flowsheet simulation is a reliable data resource for training machine learning (ML) models for process industry, especially when plant data is not available. Process simulators play an even more important role for evaluation and validation of ML models. In this work, we present a workflow for building and evaluating ML models based on data generated by a commercial flowsheet simulator. The resulting hybrid models, combining data-driven predictions with mass and energy balances, have much lower calculation times than the rigorous models. The implementation of such models shows great potential for solving more complex process engineering problems on the high-dimensional space in the future, while saving the process engineer's time in the present.     

Exploring the Ability of Cyclic Peptides to Target SAM Domains: A Computational and Experimental Study

Sterile alpha motif (SAM) domains are protein interaction modules with a helical fold. SAM–SAM interactions often adopt the mid-loop (ML)/end-helix (EH) model, in which the C-terminal helix and adjacent loops of one SAM unit (EH site) bind the central regions of another SAM domain (ML site). Herein, an original strategy to attack SAM–SAM associations is reported. It relies on the design of cyclic peptides that target a region of the SAM domain positioned at the bottom side of the EH interface, which is thought to be important for the formation of a SAM–SAM complex. This strategy has been preliminarily tested by using a model system of heterotypic SAM–SAM interactions involving the erythropoietin-producing hepatoma kinase A2 (EphA2) receptor and implementing a multidisciplinary plan made up of computational docking studies, experimental interaction assays (by NMR spectroscopy and surface plasmon resonance techniques) and conformational analysis (by NMR spectroscopy and circular dichroism). This work further highlights how only a specific balance between flexibility and rigidity may be needed to generate modulators of SAM–SAM interactions.     

Unambiguous Identification of β‐Tubulin as the Direct Cellular Target Responsible for the Cytotoxicity of Chalcone by Photoaffinity Labeling

Chalcone is a simple and potentially privileged structure in medicinal chemistry with a diverse repertoire of biological activities, among which cytotoxicity is of particular interest. The sharp structure–activity relationship (SAR) for chalcone's cytotoxicity suggests structure‐specific target interactions. Despite the numerous putative targets proposed, evidence for direct target interactions in cells is unavailable. In this study, guided by the sharp cytotoxic SAR, we developed a cytotoxic chalcone‐based photoaffinity labeling (PAL) probe, (E)‐3‐(3‐azidophenyl)‐1‐[3,5‐dimethoxy‐4‐(prop‐2‐yn‐1‐yloxy)phenyl]‐2‐methylprop‐2‐en‐1‐one (C95; IC₅₀: 0.38±0.01 μm ), along with two structurally similar non‐cytotoxic probes. These probes were used to search for the direct cellular target responsible for chalcone's cytotoxicity through intact cell‐based PAL experiments, in which β‐tubulin was identified to specifically interact with the cytotoxic probe (i.e., C95) but not the non‐cytotoxic probes. A set of phenotypical and biochemical assays further reinforced β‐tubulin as the cytotoxic target of chalcones. Peptide mass quantitation by mass spectrometric analysis revealed one peptide potentially labeled by C95, providing information on chalcone's binding site on β‐tubulin.     

Integration of Multiple Analytical and Computational Tools for the Discovery of High‐Potency Enzyme Inhibitors from Herbal Medicines

Herbal medicines (HMs) are an important source of drugs. In this study, an efficient strategy integrating ultrafiltration LC–MS, microplate bioassays, and molecular docking was proposed to screen high‐potency enzyme inhibitors from HMs. Using this strategy, the structure–activity relationships (SARs) including binding‐affinity‐based SAR, enzymatic‐activity‐based SAR, and structural‐complementarity‐based SAR of compounds in an HM can be analyzed to provide abundant information for drug discovery. A prominent advantage of the approach is that it offers a multidimensional perspective to understand enzyme–ligand interactions, which could help to avoid false‐positive screening results brought by a single method. By using xanthine oxidase (XOD) as an illustrative case, two types of potent XOD inhibitors, including flavones and coumarins, were successfully screened out from an HM of Ginkgo biloba. The study is expected to set a solid foundation for multidisciplinary cooperation in drug discovery.     

Machine learning to assist filtered two-fluid model development for dense gas–particle flows

Machine learning (ML) is experiencing an immensely fascinating resurgence in a wide variety of fields. However, applying such powerful ML to construct subgrid interphase closures has been rarely reported. To this end, we develop two data-driven ML strategies (i.e., artificial neural networks and eXtreme gradient boosting) to accurately predict filtered subgrid drag corrections using big data from highly resolved simulations of gas-particle fluidization. Quantitative assessments of effects of various subgrid input markers on training prediction outputs are performed and three-marker choice is demonstrated to be the optimal one for predicting the unseen test set. We then develop a parallel data loader to integrate this predictive ML model into a computational fluid dynamic (CFD) framework. Subsequent coarse-grid simulations agree fairly well with experiments regarding the underlying hydrodynamics in bubbling and turbulent fluidized beds. The present ML approach provides easily extended ways to facilitate the development of predictive models for multiphase flows.     

A comparative study of short cut procedures for parameter estimation in differential equations

Various short cut procedures for estimation of parameters in mathematical models given in the form of differential equations are compared with each other as well as with the maximum likelihood (ML) approach. Points covered are: computing time; programming effort; deviation of short cut estimates from ML estimates; and convergence of a ML approach following-up the short cut technique.     

Machine Learning in Chemical Engineering: A Perspective

The transformation of the chemical industry to renewable energy and feedstock supply requires new paradigms for the design of flexible plants, (bio-)catalysts, and functional materials. Recent breakthroughs in machine learning (ML) provide unique opportunities, but only joint interdisciplinary research between the ML and chemical engineering (CE) communities will unfold the full potential. We identify six challenges that will open new methods for CE and formulate new types of problems for ML: (1) optimal decision making, (2) introducing and enforcing physics in ML, (3) information and knowledge representation, (4) heterogeneity of data, (5) safety and trust in ML applications, and (6) creativity. Under the umbrella of these challenges, we discuss perspectives for future interdisciplinary research that will enable the transformation of CE.     

Reaction of histidine with methyl linoleate: Characterization of the histidine degradation products

Efforts have been made to characterize the products that result from interactions between L-histidine (free base) and peroxidizing methyl linoleate (ML) in a model system consisting of reactants dispersed on a filter paper. Imidazole lactic acid and imidazole acetic acid are identified as breakdown products when histidine is incubated with ML, methyl linoleate hydroperoxide (MLHPO), or n-hexanal over a period of 3 weeks. Two other reaction products are found to give back histidine upon acid hydrolysis. These products are though to be Schiff's base compounds which result from the condensation of the histidyl α-amino group and carbonyl groups of reactive aldehydes formed during ML peroxidation. Most of the detectable reaction products have the imidazole moiety intact indicating the high relative reactivity of the functional groups, especially the amino group, associated with the α-carbon. Such high reactivity provides an explanation for the low concentrations of ninhydrin-positive free amino compounds that are, at best, barely detectable on thin layer chromatography. Presented at the AOCS Meeting, Chicago, September 1976.     

Inhibition of Amyloid Protein Aggregation Using Selected Peptidomimetics

Aberrant protein aggregation leads to the formation of amyloid fibrils. This phenomenon is linked to the development of more than 40 irremediable diseases such as Alzheimer's disease, Parkinson's disease, type 2 diabetes, and cancer. Plenty of research efforts have been given to understanding the underlying mechanism of protein aggregation, associated toxicity, and the development of amyloid inhibitors. Recently, the peptidomimetic approach has emerged as a potential tool to modulate several protein-protein interactions (PPIs). In this review, we discussed selected peptidomimetic-based approaches for the modulation of important amyloid proteins (Islet Amyloid Polypeptide, Amyloid Beta, α-synuclein, mutant p53, and insulin) aggregation. This approach holds a powerful platform for creating an essential stepping stone for the vital development of anti-amyloid therapeutic agents.     

Ligand-Receptor Interactions and Machine Learning in GCGR and GLP-1R Drug Discovery

The large amount of data that has been collected so far for G protein-coupled receptors requires machine learning (ML) approaches to fully exploit its potential. Our previous ML model based on gradient boosting used for prediction of drug affinity and selectivity for a receptor subtype was compared with explicit information on ligand-receptor interactions from induced-fit docking. Both methods have proved their usefulness in drug response predictions. Yet, their successful combination still requires allosteric/orthosteric assignment of ligands from datasets. Our ligand datasets included activities of two members of the secretin receptor family: GCGR and GLP-1R. Simultaneous activation of two or three receptors of this family by dual or triple agonists is not a typical kind of information included in compound databases. A precise allosteric/orthosteric ligand assignment requires a continuous update based on new structural and biological data. This data incompleteness remains the main obstacle for current ML methods applied to class B GPCR drug discovery. Even so, for these two class B receptors, our ligand-based ML model demonstrated high accuracy (5-fold cross-validation Q² > 0.63 and Q² > 0.67 for GLP-1R and GCGR, respectively). In addition, we performed a ligand annotation using recent cryogenic-electron microscopy (cryo-EM) and X-ray crystallographic data on small-molecule complexes of GCGR and GLP-1R. As a result, we assigned GLP-1R and GCGR actives deposited in ChEMBL to four small-molecule binding sites occupied by positive and negative allosteric modulators and a full agonist. Annotated compounds were added to our recently released repository of GPCR data.     

Application of machine learning to sporadic experimental data for understanding epitaxial strain relaxation

Understanding epitaxial strain relaxation is one of the key challenges in functional thin films with strong structure–property relations. Herein, we employ an emerging data analytics approach to quantitatively evaluate the underlying relationships between critical thickness (h_c) of strain relaxation and various physical and chemical features, despite the sporadic experimental data points available. First, we have collected and refined the reported h_c of the perovskite oxide thin film/substrate system to construct a consistent sub-dataset which captures a common trend among the varying experimental details. Then, we employ correlation analyses and feature engineering to find the most relevant feature set which includes Poisson's ratio and lattice mismatch. With the insight offered by correlation analyses and feature engineering, machine learning (ML) models have been trained to deduce a decent accuracy, which has been further validated experimentally. The demonstrated framework is expected to be efficiently extended to the other classes of thin films in understanding h_c.     

Friction and adsorption properties of normal and high-oleic soybean oils

The steel/steel boundary friction properties of soybean oil (SBO) and high-oleic soybean oil (HOSBO) are compared. HOSBO is significantly more saturated than SBO and more oxidatively stable. Changes in degree of unsaturation affect lateral interactions of adsorbate molecules, which in turn affects their adsorption and, hence, their boundary lubrication properties. To investigate this possibility, the free energies of adsorption (ΔG _ads) of SBO, HOSBO, and methyl laurate (ML) were determined from the analysis of friction-derived adsorption isotherms using the Langmuir and Temkin adsorption models. The results showed a stronger adsorption for the vegetable oils than for ML, an indication of multiple interactions between the ester groups of the triglycerides and the steel surface. The result also showed no difference in the ΔG _ads values of SBO and HOSBO obtained using either the Langmuir or Temkin models. This was interpreted as an indication of the lack of appreciable net lateral interaction between triglyceride adsorbates.     

Ascorbic acid and ascorbyl palmitate have only minor effect on the formation and decomposition of methyl linoleate hydroperoxides

Effects of ascorbic acid (AA) and ascorbyl palmitate (AP) on lipid hydroperoxides were evaluated during the formation and decomposition of methyl linoleate hydroperoxides (ML‐OOH). AA and AP at 1 and 10 mM levels had no effect on the formation of ML‐OOH during the autoxidation of methyl linoleate at 40 °C. However, depending on the reaction medium, AA and AP at 0.2 and 2 mM either slightly inhibited or accelerated the decomposition of 40 mM cis, trans ML‐OOH in hexadecane or in hexadecane‐inwater emulsion. The increased decomposition rate of ML‐OOH, when compared to a control sample, was apparently due to the reductive activity of AA and AP on metal ions present in the system, as the addition of EDTA improved the stability of ML‐OOH. The more detailed analysis of the decomposition reactions of ML‐OOH suggests that under favorable reaction conditions AA and AP were, to some extent, capable of acting as hydrogen atom donors to peroxyl radicals and reducers of hydroperoxides to more stable hydroxy compounds. However, since all these effects of AA and AP on lipid hydroperoxides were relatively small, it is assumed that the antioxidative activity of AA and AP as well as their effect on the stability and reactions of lipid hydroperoxides in biological systems and in foods is mainly related to their synergistic interactions with other antioxidative compounds such as tocopherols.