Integrating Deep Learning and Machine Translation for Understanding Unrefined Languages

In the field of natural language processing (NLP), the advancement of neural machine translation has paved the way for cross-lingual research. Yet, most studies in NLP have evaluated the proposed language models on well-refined datasets. We investigate whether a machine translation approach is suitable for multilingual analysis of unrefined datasets, particularly, chat messages in Twitch. In order to address it, we collected the dataset, which included 7,066,854 and 3,365,569 chat messages from English and Korean streams, respectively. We employed several machine learning classifiers and neural networks with two different types of embedding: word-sequence embedding and the final layer of a pre-trained language model. The results of the employed models indicate that the accuracy difference between English, and English to Korean was relatively high, ranging from 3% to 12%. For Korean data (Korean, and Korean to English), it ranged from 0% to 2%. Therefore, the results imply that translation from a low-resource language (e.g., Korean) into a high-resource language (e.g., English) shows higher performance, in contrast to vice versa. Several implications and limitations of the presented results are also discussed. For instance, we suggest the feasibility of translation from resource-poor languages for using the tools of resource-rich languages in further analysis.     

Arabic Named Entity Recognition: A BERT-BGRU Approach

Named Entity Recognition (NER) is one of the fundamental tasks in Natural Language Processing (NLP), which aims to locate, extract, and classify named entities into a predefined category such as person, organization and location. Most of the earlier research for identifying named entities relied on using handcrafted features and very large knowledge resources, which is time consuming and not adequate for resource-scarce languages such as Arabic. Recently, deep learning achieved state-of-the-art performance on many NLP tasks including NER without requiring hand-crafted features. In addition, transfer learning has also proven its efficiency in several NLP tasks by exploiting pretrained language models that are used to transfer knowledge learned from large-scale datasets to domain-specific tasks. Bidirectional Encoder Representation from Transformer (BERT) is a contextual language model that generates the semantic vectors dynamically according to the context of the words. BERT architecture relay on multi-head attention that allows it to capture global dependencies between words. In this paper, we propose a deep learning-based model by fine-tuning BERT model to recognize and classify Arabic named entities. The pre-trained BERT context embeddings were used as input features to a Bidirectional Gated Recurrent Unit (BGRU) and were fine-tuned using two annotated Arabic Named Entity Recognition (ANER) datasets. Experimental results demonstrate that the proposed model outperformed state-of-the-art ANER models achieving 92.28% and 90.68% F-measure values on the ANERCorp dataset and the merged ANERCorp and AQMAR dataset, respectively.     

Machine Learning in Chemical Engineering: Strengths,Weaknesses, Opportunities,and Threats

Chemical engineers rely on models for design, research, and daily decision-making, often with potentially large financial and safety implications. Previous efforts a few decades ago to combine artificial intelligence and chemical engineering for modeling were unable to fulfill the expectations. In the last five years, the increasing availability of data and computational resources has led to a resurgence in machine learning-based research. Many recent efforts have facilitated the roll-out of machine learning techniques in the research field by developing large databases, benchmarks, and representations for chemical applications and new machine learning frameworks. Machine learning has significant advantages over traditional modeling techniques, including flexibility, accuracy, and execution speed. These strengths also come with weaknesses, such as the lack of interpretability of these black-box models. The greatest opportunities involve using machine learning in time-limited applications such as real-time optimization and planning that require high accuracy and that can build on models with a self-learning ability to recognize patterns, learn from data, and become more intelligent over time. The greatest threat in artificial intelligence research today is inappropriate use because most chemical engineers have had limited training in computer science and data analysis. Nevertheless, machine learning will definitely become a trustworthy element in the modeling toolbox of chemical engineers.     

Causal Inference

Causal inference is a powerful modeling tool for explanatory analysis, which might enable current machine learning to become explainable. How to marry causal inference with machine learning to develop explainable artificial intelligence (XAI) algorithms is one of key steps toward to the artificial intelligence 2.0. With the aim of bringing knowledge of causal inference to scholars of machine learning and artificial intelligence, we invited researchers working on causal inference to write this survey from different aspects of causal inference. This survey includes the following sections: “Estimating average treatment effect: A brief review and beyond” from Dr. Kun Kuang, “Attribution problems in counterfactual inference” from Prof. Lian Li, “The Yule–Simpson paradox and the surrogate paradox” from Prof. Zhi Geng, “Causal potential theory” from Prof. Lei Xu, “Discovering causal information from observational data” from Prof. Kun Zhang, “Formal argumentation in causal reasoning and explanation” from Profs. Beishui Liao and Huaxin Huang, “Causal inference with complex experiments” from Prof. Peng Ding, “Instrumental variables and negative controls for observational studies” from Prof. Wang Miao, and “Causal inference with interference” from Dr. Zhichao Jiang.     

A Neural Network-Based Approach for Statistical Probability Distribution Recognition

The statistical probability distribution of data should be known in advance, so that we can make some statistical inference based on the data and realize what information the data provides. Until now, a nonparametric goodness of fit test has been widely used in probability distribution recognition. However, such a procedure cannot guarantee a precise distribution recognition when only small data samples are available. In addition, the number of the divided groups will influence the results. This study proposes a neural network-based approach for probability distribution recognition. Two types of neural networks, backpropagation and learning vector quantization, are used in classifying normal, exponential, Weibull, Uniform, Chi-square, t, F, and Lognormal distributions. Implementation results demonstrate that the proposed approach outperforms the traditional statistical approach.     

Convolutional and generative adversarial neural networks in manufacturing

Manufacturing is undergoing transformation driven by the developments in process technology, information technology, and data science. A future manufacturing enterprise will be highly digital. This will create opportunities for machine learning algorithms to generate predictive models across the enterprise in the spirit of the digital twin concept. Convolutional and generative adversarial neural networks have received some attention of the manufacturing research community. Representative research and applications of the two machine learning concepts in manufacturing are presented. Advantages and limitations of each neural network are discussed. The paper might be helpful in identifying research gaps, inspire machine learning research in new manufacturing domains, contribute to the development of successful neural network architectures, and getting deeper insights into the manufacturing data.     

An Improved End-to-End Memory Network for QA Tasks

At present, End-to-End trainable Memory Networks (MemN2N) has proven to be promising in many deep learning fields, especially on simple natural language-based reasoning question and answer (QA) tasks. However, when solving some subtasks such as basic induction, path finding or time reasoning tasks, it remains challenging because of limited ability to learn useful information between memory and query. In this paper, we propose a novel gated linear units (GLU) and local-attention based end-to-end memory networks (MemN2N-GL) motivated by the success of attention mechanism theory in the field of neural machine translation, it shows an improved possibility to develop the ability of capturing complex memory-query relations and works better on some subtasks. It is an improved end-to-end memory network for QA tasks. We demonstrate the effectiveness of these approaches on the 20 bAbI dataset which includes 20 challenging tasks, without the use of any domain knowledge. Our project is open source on github⁴.     

Quasi-Phase Equilibrium Prediction of Multi-Element Alloys Based on Machine Learning and Deep Learning

In this study, a phase field model is established to simulate the microstructure formation during the solidification of dendrites by taking the Al-Cu-Mg ternary alloy as an example, and machine learning and deep learning methods are combined with the Kim-Kim-Suzuki (KKS) phase field model to predict the quasi-phase equilibrium. The paper first uses the least squares method to obtain the required data and then applies eight machine learning methods and five deep learning methods to train the quasi-phase equilibrium prediction models. After obtaining different models, this paper compares the reliability of the established models by using the test data and uses two evaluation criteria to analyze the performance of these models. This work find that the performance of the established deep learning models is generally better than that of the machine learning models, and the Multilayer Perceptron (MLP) based quasi-phase equilibrium prediction model achieves the best performance. Meanwhile the Convolutional Neural Network (CNN) based model also achieves competitive results. The experimental results show that the model proposed in this paper can predict the quasi-phase equilibrium of the KKS phase-field model accurately, which proves that it is feasible to combine machine learning and deep learning methods with phase-field model simulation.     

A Neural Network-Based Trust Management System for Edge Devices in Peer-to-Peer Networks

Edge devices in Internet of Things (IoT) applications can form peers to communicate in peer-to-peer (P2P) networks over P2P protocols. Using P2P networks ensures scalability and removes the need for centralized management. However, due to the open nature of P2P networks, they often suffer from the existence of malicious peers, especially malicious peers that unite in groups to raise each other's ratings. This compromises users' safety and makes them lose their confidence about the files or services they are receiving. To address these challenges, we propose a neural network-based algorithm, which uses the advantages of a machine learning algorithm to identify whether or not a peer is malicious. In this paper, a neural network (NN) was chosen as the machine learning algorithm due to its efficiency in classification. The experiments showed that the NNTrust algorithm is more effective and has a higher potential of reducing the number of invalid files and increasing success rates than other well-known trust management systems.     

Image Super-Resolution Based on Generative Adversarial Networks: A Brief Review

Single image super resolution (SISR) is an important research content in the field of computer vision and image processing. With the rapid development of deep neural networks, different image super-resolution models have emerged. Compared to some traditional SISR methods, deep learning-based methods can complete the superresolution tasks through a single image. In addition, compared with the SISR methods using traditional convolutional neural networks, SISR based on generative adversarial networks (GAN) has achieved the most advanced visual performance. In this review, we first explore the challenges faced by SISR and introduce some common datasets and evaluation metrics. Then, we review the improved network structures and loss functions of GAN-based perceptual SISR. Subsequently, the advantages and disadvantages of different networks are analyzed by multiple comparative experiments. Finally, we summarize the paper and look forward to the future development trends of GAN-based perceptual SISR.     

Deep Feature Fusion Model for Sentence Semantic Matching

Sentence semantic matching (SSM) is a fundamental research in solving natural language processing tasks such as question answering and machine translation. The latest SSM research benefits from deep learning techniques by incorporating attention mechanism to semantically match given sentences. However, how to fully capture the semantic context without losing significant features for sentence encoding is still a challenge. To address this challenge, we propose a deep feature fusion model and integrate it into the most popular deep learning architecture for sentence matching task. The integrated architecture mainly consists of embedding layer, deep feature fusion layer, matching layer and prediction layer. In addition, we also compare the commonly used loss function, and propose a novel hybrid loss function integrating MSE and cross entropy together, considering confidence interval and threshold setting to preserve the indistinguishable instances in training process. To evaluate our model performance, we experiment on two real world public data sets: LCQMC and Quora. The experiment results demonstrate that our model outperforms the most existing advanced deep learning models for sentence matching, benefited from our enhanced loss function and deep feature fusion model for capturing semantic context.     

A Framework of Deep Optimal Features Selection for Apple Leaf Diseases Recognition

Identifying fruit disease manually is time-consuming, expert-required, and expensive; thus, a computer-based automated system is widely required. Fruit diseases affect not only the quality but also the quantity. As a result, it is possible to detect the disease early on and cure the fruits using computer-based techniques. However, computer-based methods face several challenges, including low contrast, a lack of dataset for training a model, and inappropriate feature extraction for final classification. In this paper, we proposed an automated framework for detecting apple fruit leaf diseases using CNN and a hybrid optimization algorithm. Data augmentation is performed initially to balance the selected apple dataset. After that, two pre-trained deep models are fine-tuning and trained using transfer learning. Then, a fusion technique is proposed named Parallel Correlation Threshold (PCT). The fused feature vector is optimized in the next step using a hybrid optimization algorithm. The selected features are finally classified using machine learning algorithms. Four different experiments have been carried out on the augmented Plant Village dataset and yielded the best accuracy of 99.8%. The accuracy of the proposed framework is also compared to that of several neural nets, and it outperforms them all.     

Statistical transfer learning: A review and some extensions to statistical process control

ABSTRACT

The rapid development of information technology, together with advances in sensory and data acquisition techniques, has led to the increasing necessity of handling datasets from multiple domains. In recent years, transfer learning has emerged as an effective framework for tackling related tasks in target domains by transferring previously-acquired knowledge from source domains. Statistical models and methodologies are widely involved in transfer learning and play a critical role, which, however, has not been emphasized in most surveys of transfer learning. In this article, we conduct a comprehensive literature review on statistical transfer learning, i.e., transfer learning techniques with a focus on statistical models and statistical methodologies, demonstrating how statistics can be used in transfer learning. In addition, we highlight opportunities for the use of statistical transfer learning to improve statistical process control and quality control. Several potential future issues in statistical transfer learning are discussed.     

Applying ANN,ANFIS and LSSVM Models for Estimation of Acid Solvent Solubility in Supercritical CO₂

In the present work, a novel machine learning computational investigation is carried out to accurately predict the solubility of different acids in supercritical carbon dioxide. Four different machine learning algorithms of radial basis function, multi-layer perceptron (MLP), artificial neural networks (ANN), least squares support vector machine (LSSVM) and adaptive neuro-fuzzy inference system (ANFIS) are used to model the solubility of different acids in carbon dioxide based on the temperature, pressure, hydrogen number, carbon number, molecular weight, and the dissociation constant of acid. To evaluate the proposed models, different graphical and statistical analyses, along with novel sensitivity analysis, are carried out. The present study proposes an efficient tool for acid solubility estimation in supercritical carbon dioxide, which can be highly beneficial for engineers and chemists to predict operational conditions in industries.     

Dark,Beyond Deep: A Paradigm Shift to Cognitive AI with Humanlike Common Sense

Recent progress in deep learning is essentially based on a “big data for small tasks” paradigm, under which massive amounts of data are used to train a classifier for a single narrow task. In this paper, we call for a shift that flips this paradigm upside down. Specifically, we propose a “small data for big tasks” paradigm, wherein a single artificial intelligence (AI) system is challenged to develop “common sense,” enabling it to solve a wide range of tasks with little training data. We illustrate the potential power of this new paradigm by reviewing models of common sense that synthesize recent breakthroughs in both machine and human vision. We identify functionality, physics, intent, causality, and utility (FPICU) as the five core domains of cognitive AI with humanlike common sense. When taken as a unified concept, FPICU is concerned with the questions of “why” and “how,” beyond the dominant “what” and “where” framework for understanding vision. They are invisible in terms of pixels but nevertheless drive the creation, maintenance, and development of visual scenes. We therefore coin them the “dark matter” of vision. Just as our universe cannot be understood by merely studying observable matter, we argue that vision cannot be understood without studying FPICU. We demonstrate the power of this perspective to develop cognitive AI systems with humanlike common sense by showing how to observe and apply FPICU with little training data to solve a wide range of challenging tasks, including tool use, planning, utility inference, and social learning. In summary, we argue that the next generation of AI must embrace “dark” humanlike common sense for solving novel tasks.     

Decoding the glass genome

Glasses have played a critical role in the development of modern civilization and will continue to bring new solutions to global challenges from energy and the environment to healthcare and information/communication technology. To meet the accelerated pace of modern technology delivery, a more sophisticated approach to the design of advanced glass chemistries must be developed to enable faster, cheaper, and better research and development of new glass compositions for future applications. In the spirit of the U.S. Materials Genome Initiative, here we describe an approach for designing new glasses based on a mathematical optimization of composition-dependent glass property models. The models combine known physical insights regarding glass composition-property relationships together with data-driven approaches including machine learning techniques. Using such a combination of physical and empirical modeling approaches, we seek to decode the “glass genome,” enabling the improved and accelerated design of new glassy materials.     

On the Detection of COVID-19 from Chest X-Ray Images Using CNN-Based Transfer Learning

Coronavirus disease (COVID-19) is an extremely infectious disease and possibly causes acute respiratory distress or in severe cases may lead to death. There has already been some research in dealing with coronavirus using machine learning algorithms, but few have presented a truly comprehensive view. In this research, we show how convolutional neural network (CNN) can be useful to detect COVID-19 using chest X-ray images. We leverage the CNN-based pre-trained models as feature extractors to substantiate transfer learning and add our own classifier in detecting COVID-19. In this regard, we evaluate performance of five different pre-trained models with fine-tuning the weights from some of the top layers. We also develop an ensemble model where the predictions from all chosen pre-trained models are combined to generate a single output. The models are evaluated through 5-fold cross validation using two publicly available data repositories containing healthy and infected (both COVID-19 and other pneumonia) chest X-ray images. We also leverage two different visualization techniques to observe how efficiently the models extract important features related to the detection of COVID- 19 patients. The models show high degree of accuracy, precision, and sensitivity. We believe that the models will aid medical professionals with improved and faster patient screening and pave a way to further COVID-19 research.     

Clustering scientific documents with topic modeling

Topic modeling is a type of statistical model for discovering the latent “topics” that occur in a collection of documents through machine learning. Currently, latent Dirichlet allocation (LDA) is a popular and common modeling approach. In this paper, we investigate methods, including LDA and its extensions, for separating a set of scientific publications into several clusters. To evaluate the results, we generate a collection of documents that contain academic papers from several different fields and see whether papers in the same field will be clustered together. We explore potential scientometric applications of such text analysis capabilities.     

MoTransFrame: Model Transfer Framework for CNNs on Low-Resource Edge Computing Node

Deep learning technology has been widely used in computer vision, speech recognition, natural language processing, and other related fields. The deep learning algorithm has high precision and high reliability. However, the lack of resources in the edge terminal equipment makes it difficult to run deep learning algorithms that require more memory and computing power. In this paper, we propose MoTransFrame, a general model processing framework for deep learning models. Instead of designing a model compression algorithm with a high compression ratio, MoTransFrame can transplant popular convolutional neural networks models to resources-starved edge devices promptly and accurately. By the integration method, Deep learning models can be converted into portable projects for Arduino, a typical edge device with limited resources. Our experiments show that MoTransFrame has good adaptability in edge devices with limited memories. It is more flexible than other model transplantation methods. It can keep a small loss of model accuracy when the number of parameters is compressed by tens of times. At the same time, the computational resources needed in the reasoning process are less than what the edge node could handle.