XAI-MEG: Combining symbolic AI and machine learning to generate first-principles models and causal explanations

Current machine learning methods generally do not reveal any mechanistic insights or provide causal explanations for their decisions. While this may not be a big concern in typical computer vision, game playing, and recommendation systems, this is important for many problems in chemical engineering such as fault diagnosis, process control, and process safety analysis. To address these drawbacks, one needs to go beyond purely data-driven machine learning techniques and incorporate the lessons learned from the expert systems era of artificial intelligence (AI), in the 1970s and 1980s. In this article, we present such a hybrid-AI framework that demonstrates how symbolic AI techniques can be integrated with numeric AI-based machine learning methods.     

Deep Learning in Virtual Screening: Recent Applications and Developments

Drug discovery is a cost and time-intensive process that is often assisted by computational methods, such as virtual screening, to speed up and guide the design of new compounds. For many years, machine learning methods have been successfully applied in the context of computer-aided drug discovery. Recently, thanks to the rise of novel technologies as well as the increasing amount of available chemical and bioactivity data, deep learning has gained a tremendous impact in rational active compound discovery. Herein, recent applications and developments of machine learning, with a focus on deep learning, in virtual screening for active compound design are reviewed. This includes introducing different compound and protein encodings, deep learning techniques as well as frequently used bioactivity and benchmark data sets for model training and testing. Finally, the present state-of-the-art, including the current challenges and emerging problems, are examined and discussed.     

SecProCT: In Silico Prediction of Human Secretory Proteins Based on Capsule Network and Transformer

Identifying secretory proteins from blood, saliva or other body fluids has become an effective method of diagnosing diseases. Existing secretory protein prediction methods are mainly based on conventional machine learning algorithms and are highly dependent on the feature set from the protein. In this article, we propose a deep learning model based on the capsule network and transformer architecture, SecProCT, to predict secretory proteins using only amino acid sequences. The proposed model was validated using cross-validation and achieved 0.921 and 0.892 accuracy for predicting blood-secretory proteins and saliva-secretory proteins, respectively. Meanwhile, the proposed model was validated on an independent test set and achieved 0.917 and 0.905 accuracy for predicting blood-secretory proteins and saliva-secretory proteins, respectively, which are better than conventional machine learning methods and other deep learning methods for biological sequence analysis. The main contributions of this article are as follows: (1) a deep learning model based on a capsule network and transformer architecture is proposed for predicting secretory proteins. The results of this model are better than the those of existing conventional machine learning methods and deep learning methods for biological sequence analysis; (2) only amino acid sequences are used in the proposed model, which overcomes the high dependence of existing methods on the annotated protein features; (3) the proposed model can accurately predict most experimentally verified secretory proteins and cancer protein biomarkers in blood and saliva.     

机器学习在多相反应器中的应用进展

准确理解并精确预测多相反应器内复杂的流体力学特性、传递现象及反应特征,是过程工程领域的热点方向之一。随着试验测量技术及高性能计算机的快速发展,研究者可以获取高精度的多维瞬态流场数据集。近十年来,机器学习作为一门新兴学科,越来越广泛地应用于数据挖掘、图像识别、智能控制等领域。本文概述了几种常用的机器学习方法（包括神经网络模型、支持向量机模型、决策树模型、聚类算法模型等）,总结了机器学习模型的构建过程（包括数据集的建立、特征变量的选择、算法框架的选取、模型参数的调优、模型验证与测试等）,综述了机器学习辅助多相反应器中流场本构模型构建、流场图像重构、流型识别、流场关键参数预测及优化、不确定度分析、数字孪生技术平台等方面的应用进展,剖析了机器学习结合多相反应器领域所面临的挑战,展望了机器学习在多相反应器中可能有待拓展的方向。     

Machine learning for molecular thermodynamics

Thermodynamic properties of complex systems play an essential role in developing chemical engineering processes. It remains a challenge to predict the thermodynamic properties of complex systems in a wide range and describe the behavior of ions and molecules in complex systems. Machine learning emerges as a powerful tool to resolve this issue because it can describe complex relationships beyond the capacity of traditional mathematical functions. This minireview will summarize some fundamental concepts of machine learning methods and their applications in three aspects of the molecular thermodynamics using several examples. The first aspect is to apply machine learning methods to predict the thermodynamic properties of a broad spectrum of systems based on known data. The second aspect is to integer machine learning and molecular simulations to accelerate the discovery of materials. The third aspect is to develop machine learning force field that can eliminate the barrier between quantum mechanics and all-atom molecular dynamics simulations. The applications in these three aspects illustrate the potential of machine learning in molecular thermodynamics of chemical engineering. We will also discuss the perspective of the broad applications of machine learning in chemical engineering.     

Reconciling deep learning and first-principle modelling for the investigation of transport phenomena in chemical engineering

The use of machine learning in chemical engineering has the potential to greatly improve the design and analysis of complex systems. However, there are also risks associated with its adoption, such as the potential for bias in algorithms and the need for careful oversight to ensure the safety and reliability of machine learning-powered systems. This paper explores the opportunities and risks of using machine learning in chemical engineering and provides a perspective on how it may be integrated into engineering practices in a responsible and effective manner. We generated the text of this abstract with GPT-3, OpenAI's large-scale language-generation model. Upon generating the draft, we ensured that the language was to our liking, and we take ultimate responsibility for the content of this publication.     

Diagnostic Utility of Genome-Wide DNA Methylation Analysis in Mendelian Neurodevelopmental Disorders

Mendelian neurodevelopmental disorders customarily present with complex and overlapping symptoms, complicating the clinical diagnosis. Individuals with a growing number of the so-called rare disorders exhibit unique, disorder-specific DNA methylation patterns, consequent to the underlying gene defects. Besides providing insights to the pathophysiology and molecular biology of these disorders, we can use these epigenetic patterns as functional biomarkers for the screening and diagnosis of these conditions. This review summarizes our current understanding of DNA methylation episignatures in rare disorders and describes the underlying technology and analytical approaches. We discuss the computational parameters, including statistical and machine learning methods, used for the screening and classification of genetic variants of uncertain clinical significance. Describing the rationale and principles applied to the specific computational models that are used to develop and adapt the DNA methylation episignatures for the diagnosis of rare disorders, we highlight the opportunities and challenges in this emerging branch of diagnostic medicine.     

Recent Applications of Deep Learning Methods on Evolution- and Contact-Based Protein Structure Prediction

The new advances in deep learning methods have influenced many aspects of scientific research, including the study of the protein system. The prediction of proteins’ 3D structural components is now heavily dependent on machine learning techniques that interpret how protein sequences and their homology govern the inter-residue contacts and structural organization. Especially, methods employing deep neural networks have had a significant impact on recent CASP13 and CASP14 competition. Here, we explore the recent applications of deep learning methods in the protein structure prediction area. We also look at the potential opportunities for deep learning methods to identify unknown protein structures and functions to be discovered and help guide drug–target interactions. Although significant problems still need to be addressed, we expect these techniques in the near future to play crucial roles in protein structural bioinformatics as well as in drug discovery.     

Machine Learning in Chemical Engineering: A Perspective

The transformation of the chemical industry to renewable energy and feedstock supply requires new paradigms for the design of flexible plants, (bio-)catalysts, and functional materials. Recent breakthroughs in machine learning (ML) provide unique opportunities, but only joint interdisciplinary research between the ML and chemical engineering (CE) communities will unfold the full potential. We identify six challenges that will open new methods for CE and formulate new types of problems for ML: (1) optimal decision making, (2) introducing and enforcing physics in ML, (3) information and knowledge representation, (4) heterogeneity of data, (5) safety and trust in ML applications, and (6) creativity. Under the umbrella of these challenges, we discuss perspectives for future interdisciplinary research that will enable the transformation of CE.     

Dissecting Response to Cancer Immunotherapy by Applying Bayesian Network Analysis to Flow Cytometry Data

Cancer immunotherapy, specifically immune checkpoint blockade, has been found to be effective in the treatment of metastatic cancers. However, only a subset of patients achieve clinical responses. Elucidating pretreatment biomarkers predictive of sustained clinical response is a major research priority. Another research priority is evaluating changes in the immune system before and after treatment in responders vs. nonresponders. Our group has been studying immune networks as an accurate reflection of the global immune state. Flow cytometry (FACS, fluorescence-activated cell sorting) data characterizing immune cell panels in peripheral blood mononuclear cells (PBMC) from gastroesophageal adenocarcinoma (GEA) patients were used to analyze changes in immune networks in this setting. Here, we describe a novel computational pipeline to perform secondary analyses of FACS data using systems biology/machine learning techniques and concepts. The pipeline is centered around comparative Bayesian network analyses of immune networks and is capable of detecting strong signals that conventional methods (such as FlowJo manual gating) might miss. Future studies are planned to validate and follow up the immune biomarkers (and combinations/interactions thereof) associated with clinical responses identified with this computational pipeline.     

High-accuracy reliability evaluation for the WC–Co-based cemented carbides assisted by machine learning

Materials reliability analysis is one of the most important substances in industrial manufacturing and practical application. Weibull statistics is a common-used approach to evaluate reliability, especially for brittle materials. However, such a process is limited by the insufficient number of samples and complex analysis steps. Herein, a machine learning-assisted strategy to analyze the reliability of the materials was proposed. The WC–Co-based cemented carbides were taken as the target materials. The machine learning models coupled feature engineering methods with advanced machine learning algorithms. Through an evaluation by designed experiments, the artificial neural network algorithm is determined to be the best machine learning algorithm to accurately capture the variation of property data to identify their distribution and automatically predict the Weibull modulus for reliability evaluation. This study provided a novel approach to evaluate the reliability accurately and shows the application potential to design the process parameters of other materials.     

Machine Learning Approaches for Phase Identification Using Process Variables in Batch Processes

Specialty and fine chemicals are often manufactured in multipurpose batch production plants. Compared to continuous production, these plants offer increased flexibility at the cost of operational complexity. A recipe defines the sequence and process parameters of different batch phases that are needed to transform raw materials into the desired product. In some plants detailed information about the executed recipe is not always captured by data acquisition systems. Knowledge of these phases is essential for optimizing quality and throughput. State-of-the-art data-driven machine learning techniques can recognize recurrent patterns in noisy time series data, enabling automatic labeling of batch phases based on widely available sensor data. In this review paper, we provide an overview of several machine learning approaches that can be used in an industrial setting.     

Review of adaptation mechanisms for data-driven soft sensors

In this article, we review and discuss algorithms for adaptive data-driven soft sensing. In order to be able to provide a comprehensive overview of the adaptation techniques, adaptive soft sensing methods are reviewed from the perspective of machine learning theory for adaptive learning systems. In particular, the concept drift theory is exploited to classify the algorithms into three different types, which are: (i) moving windows techniques; (ii) recursive adaptation techniques; and (iii) ensemble-based methods. The most significant algorithms are described in some detail and critically reviewed in this work. We also provide a comprehensive list of publications where adaptive soft sensors were proposed and applied to practical problems. Furthermore in order to enable the comparison of different methods to standard soft sensor applications, a list of publicly available data sets for the development of data-driven soft sensors is presented.     

基于深度学习的化工过程故障检测与诊断研究综述

化工过程的故障检测与诊断对于现代化工系统的可靠性和安全性具有重要意义.深度学习作为一项新兴的技术,引起了学术界和工业界的广泛关注.从方法的角度出发,将基于深度学习的化工过程故障检测与诊断技术分为:基于自动编码器的方法、基于深度置信网络的方法、基于卷积神经网络的方法和基于循环神经网络的方法,并分别对4种方法的最新研究进展...     

Philosophical Basis and Some Historical Aspects of Systems Biology: From Hegel to Noble - Applications for Bioenergetic Research

We live in times of paradigmatic changes for the biological sciences. Reductionism, that for the last six decades has been the philosophical basis of biochemistry and molecular biology, is being displaced by Systems Biology, which favors the study of integrated systems. Historically, Systems Biology - defined as the higher level analysis of complex biological systems - was pioneered by Claude Bernard in physiology, Norbert Wiener with the development of cybernetics, and Erwin Schrödinger in his thermodynamic approach to the living. Systems Biology applies methods inspired by cybernetics, network analysis, and non-equilibrium dynamics of open systems. These developments follow very precisely the dialectical principles of development from thesis to antithesis to synthesis discovered by Hegel. Systems Biology opens new perspectives for studies of the integrated processes of energy metabolism in different cells. These integrated systems acquire new, system-level properties due to interaction of cellular components, such as metabolic compartmentation, channeling and functional coupling mechanisms, which are central for regulation of the energy fluxes. State of the art of these studies in the new area of Molecular System Bioenergetics is analyzed.     

A Computational Framework Based on Ensemble Deep Neural Networks for Essential Genes Identification

Essential genes contain key information of genomes that could be the key to a comprehensive understanding of life and evolution. Because of their importance, studies of essential genes have been considered a crucial problem in computational biology. Computational methods for identifying essential genes have become increasingly popular to reduce the cost and time-consumption of traditional experiments. A few models have addressed this problem, but performance is still not satisfactory because of high dimensional features and the use of traditional machine learning algorithms. Thus, there is a need to create a novel model to improve the predictive performance of this problem from DNA sequence features. This study took advantage of a natural language processing (NLP) model in learning biological sequences by treating them as natural language words. To learn the NLP features, a supervised learning model was consequentially employed by an ensemble deep neural network. Our proposed method could identify essential genes with sensitivity, specificity, accuracy, Matthews correlation coefficient (MCC), and area under the receiver operating characteristic curve (AUC) values of 60.2%, 84.6%, 76.3%, 0.449, and 0.814, respectively. The overall performance outperformed the single models without ensemble, as well as the state-of-the-art predictors on the same benchmark dataset. This indicated the effectiveness of the proposed method in determining essential genes, in particular, and other sequencing problems, in general.     

Deep Learning with Neuroimaging and Genomics in Alzheimer’s Disease

A growing body of evidence currently proposes that deep learning approaches can serve as an essential cornerstone for the diagnosis and prediction of Alzheimer’s disease (AD). In light of the latest advancements in neuroimaging and genomics, numerous deep learning models are being exploited to distinguish AD from normal controls and/or to distinguish AD from mild cognitive impairment in recent research studies. In this review, we focus on the latest developments for AD prediction using deep learning techniques in cooperation with the principles of neuroimaging and genomics. First, we narrate various investigations that make use of deep learning algorithms to establish AD prediction using genomics or neuroimaging data. Particularly, we delineate relevant integrative neuroimaging genomics investigations that leverage deep learning methods to forecast AD on the basis of incorporating both neuroimaging and genomics data. Moreover, we outline the limitations as regards to the recent AD investigations of deep learning with neuroimaging and genomics. Finally, we depict a discussion of challenges and directions for future research. The main novelty of this work is that we summarize the major points of these investigations and scrutinize the similarities and differences among these investigations.     

基于流形正则化半监督学习的污水处理操作工况识别方法

污水处理过程容易受外界冲激扰动影响,引发污泥上浮、老化、中毒、膨胀等故障工况,导致出水水质质量差,能源消耗高等问题,如何快速准确识别污水操作工况故障至关重要。针对污水工况识别过程中现有监督学习方法未利用大量未标记数据蕴含的丰富操作工况信息,采用基于流形正则化极限学习机的半监督学习方法,监视生化污水处理过程操作运行工况。该方法在学习过程中,在标记和未标记数据输入空间构建图拉普拉斯算子,通过随机特征映射建立隐含层,在流形正则化框架下,求解隐含层和输出层之间的权重,保留随机神经网络的计算效率和泛化性能。仿真实验结果表明,基于半监督极限学习机的污水处理工况识别在准确率与可靠性方面相对优于基本极限学习机方法。     

基于数据特征提取的AANN-ELM研究及化工应用

针对极限学习机不能有效解决化工过程中高维数据建模的问题,本文将其与自联想神经网络结合,通过自联想神经网络过滤输入数据中存在的冗余信息、提取特征分量,并对所提取的特征分量采用极限学习机进行训练,由此形成了一种基于数据特征提取的AANN-ELM(auto-associative neural network-extreme learning machine)神经网络。同时,以UCI标准数据集进行测试,以精对苯二甲酸(PTA)溶剂系统进行验证,结果表明,AANN-ELM在处理高维数据时具有学习速度快、网络稳定性强、建模精度高的特点,为神经网络在复杂化工生产中的应用提供了新思路。