How to find similar companies using websites?

The selection of industry partners for Research and Development (R&D) is a challenging task for many organizations. Present methods for partner-selection, based on patents, publications or company databases, do often fail for highly specialized SMEs. Our approach aims at calculating the technological similarity for partner discovery. We apply methods from Natural Language Processing (NLP) on companies’ website texts. We show that the deep-learning language model BERT outperforms other methods at this task. Tested against expert-proven ground truth, it achieves an F1-score up to 0.90. Our results imply that website texts are useful for the purpose of estimating the similarity between companies. We see great potential in the scalability of the semantic analysis of company website texts.     

A Semantic Supervision Method for Abstractive Summarization

In recent years, many text summarization models based on pre-training methods have achieved very good results. However, in these text summarization models, semantic deviations are easy to occur between the original input representation and the representation that passed multi-layer encoder, which may result in inconsistencies between the generated summary and the source text content. The Bidirectional Encoder Representations from Transformers (BERT) improves the performance of many tasks in Natural Language Processing (NLP). Although BERT has a strong capability to encode context, it lacks the fine-grained semantic representation. To solve these two problems, we proposed a semantic supervision method based on Capsule Network. Firstly, we extracted the fine-grained semantic representation of the input and encoded result in BERT by Capsule Network. Secondly, we used the fine-grained semantic representation of the input to supervise the fine-grained semantic representation of the encoded result. Then we evaluated our model on a popular Chinese social media dataset (LCSTS), and the result showed that our model achieved higher ROUGE scores (including R-1, R-2), and our model outperformed baseline systems. Finally, we conducted a comparative study on the stability of the model, and the experimental results showed that our model was more stable.     

Arabic Named Entity Recognition: A BERT-BGRU Approach

Named Entity Recognition (NER) is one of the fundamental tasks in Natural Language Processing (NLP), which aims to locate, extract, and classify named entities into a predefined category such as person, organization and location. Most of the earlier research for identifying named entities relied on using handcrafted features and very large knowledge resources, which is time consuming and not adequate for resource-scarce languages such as Arabic. Recently, deep learning achieved state-of-the-art performance on many NLP tasks including NER without requiring hand-crafted features. In addition, transfer learning has also proven its efficiency in several NLP tasks by exploiting pretrained language models that are used to transfer knowledge learned from large-scale datasets to domain-specific tasks. Bidirectional Encoder Representation from Transformer (BERT) is a contextual language model that generates the semantic vectors dynamically according to the context of the words. BERT architecture relay on multi-head attention that allows it to capture global dependencies between words. In this paper, we propose a deep learning-based model by fine-tuning BERT model to recognize and classify Arabic named entities. The pre-trained BERT context embeddings were used as input features to a Bidirectional Gated Recurrent Unit (BGRU) and were fine-tuned using two annotated Arabic Named Entity Recognition (ANER) datasets. Experimental results demonstrate that the proposed model outperformed state-of-the-art ANER models achieving 92.28% and 90.68% F-measure values on the ANERCorp dataset and the merged ANERCorp and AQMAR dataset, respectively.     

Patent classification by fine-tuning BERT language model

In this work we focus on fine-tuning a pre-trained BERT model and applying it to patent classification. When applied to large datasets of over two million patents, our approach outperforms the state of the art by an approach using CNN with word embeddings. Besides, we focus on patent claims without other parts in patent documents. Our contributions include: (1) a new state-of-the-art result based on pre-trained BERT model and fine-tuning for patent classification, (2) a large dataset USPTO-3M at the CPC subclass level with SQL statements that can be used by future researchers, (3) showing that patent claims alone are sufficient to achieve state-of-the-art results for classification task, in contrast to conventional wisdom.     

Breast Cancer Detection in Saudi Arabian Women Using Hybrid Machine Learning on Mammographic Images

Breast cancer (BC) is the most common cause of women’s deaths worldwide. The mammography technique is the most important modality for the detection of BC. To detect abnormalities in mammographic images, the Breast Imaging Reporting and Data System (BI-RADs) is used as a baseline. The correct allocation of BI-RADs categories for mammographic images is always an interesting task, even for specialists. In this work, to detect and classify the mammogram images in BI-RADs, a novel hybrid model is presented using a convolutional neural network (CNN) with the integration of a support vector machine (SVM). The dataset used in this research was collected from different hospitals in the Qassim health cluster of Saudi Arabia. The collection of all categories of BI-RADs is one of the major contributions of this paper. Another significant contribution is the development of a hybrid approach through the integration of CNN and SVM. The proposed hybrid approach uses three CNN models to obtain ensemble CNN model results. This ensemble model saves the values to integrate them with SVM. The proposed system achieved a classification accuracy, sensitivity, specificity, precision, and F1-score of 93.6%, 94.8%, 96.9%, 96.6%, and 95.7%, respectively. The proposed model achieved better performance compared to previously available methods.     

Improving Chinese Word Representation with Conceptual Semantics

The meaning of a word includes a conceptual meaning and a distributive meaning. Word embedding based on distribution suffers from insufficient conceptual semantic representation caused by data sparsity, especially for low-frequency words. In knowledge bases, manually annotated semantic knowledge is stable and the essential attributes of words are accurately denoted. In this paper, we propose a Conceptual Semantics Enhanced Word Representation (CEWR) model, computing the synset embedding and hypernym embedding of Chinese words based on the Tongyici Cilin thesaurus, and aggregating it with distributed word representation to have both distributed information and the conceptual meaning encoded in the representation of words. We evaluate the CEWR model on two tasks: word similarity computation and short text classification. The Spearman correlation between model results and human judgement are improved to 64.71%, 81.84%, and 85.16% on Wordsim297, MC30, and RG65, respectively. Moreover, CEWR improves the F1 score by 3% in the short text classification task. The experimental results show that CEWR can represent words in a more informative approach than distributed word embedding. This proves that conceptual semantics, especially hypernymous information, is a good complement to distributed word representation.     

Sentiment Analysis and Classification Using Deep Semantic Information and Contextual Knowledge

Sentiment analysis (AS) is one of the basic research directions in natural language processing (NLP), it is widely adopted for news, product review, and politics. Aspect-based sentiment analysis (ABSA) aims at identifying the sentiment polarity of a given target context, previous existing model of sentiment analysis possesses the issue of the insufficient exaction of features which results in low accuracy. Hence this research work develops a deep-semantic and contextual knowledge networks (DSCNet). DSCNet tends to exploit the semantic and contextual knowledge to understand the context and enhance the accuracy based on given aspects. At first temporal relationships are established then deep semantic knowledge and contextual knowledge are introduced. Further, a deep integration layer is introduced to measure the importance of features for efficient extraction of different dimensions. Novelty of DSCNet model lies in introducing the deep contextual. DSCNet is evaluated on three datasets i.e., Restaurant, Laptop, and Twitter dataset considering different deep learning (DL) metrics like precision, recall, accuracy, and Macro-F1 score. Also, comparative analysis is carried out with different baseline methods in terms of accuracy and Macro-F1 score. DSCNet achieves 92.59% of accuracy on restaurant dataset, 86.99% of accuracy on laptop dataset and 78.76% of accuracy on Twitter dataset.     

Automatic text classification using BPLion-neural network and semantic word processing

Text mining has become a major research topic in which text classification is the important task for finding the relevant information from the new document. Accordingly, this paper presents a semantic word processing technique for text categorization that utilizes semantic keywords, instead of using independent features of the keywords in the documents. Hence, the dimensionality of the search space can be reduced. Here, the Back Propagation Lion algorithm (BP Lion algorithm) is also proposed to overcome the problem in updating the neuron weight. The proposed text classification methodology is experimented over two data sets, namely, 20 Newsgroup and Reuter. The performance of the proposed BPLion is analysed, in terms of sensitivity, specificity, and accuracy, and compared with the performance of the existing works. The result shows that the proposed BPLion algorithm and semantic processing methodology classifies the documents with less training time and more classification accuracy of 90.9%.     

Deep Learning and Machine Learning-Based Model for Conversational Sentiment Classification

In the current era of the internet, people use online media for conversation, discussion, chatting, and other similar purposes. Analysis of such material where more than one person is involved has a spate challenge as compared to other text analysis tasks. There are several approaches to identify users’ emotions from the conversational text for the English language, however regional or low resource languages have been neglected. The Urdu language is one of them and despite being used by millions of users across the globe, with the best of our knowledge there exists no work on dialogue analysis in the Urdu language. Therefore, in this paper, we have proposed a model which utilizes deep learning and machine learning approaches for the classification of users’ emotions from the text. To accomplish this task, we have first created a dataset for the Urdu language with the help of existing English language datasets for dialogue analysis. After that, we have preprocessed the data and selected dialogues with common emotions. Once the dataset is prepared, we have used different deep learning and machine learning techniques for the classification of emotion. We have tuned the algorithms according to the Urdu language datasets. The experimental evaluation has shown encouraging results with 67% accuracy for the Urdu dialogue datasets, more than 10, 000 dialogues are classified into five emotions i.e., joy, fear, anger, sadness, and neutral. We believe that this is the first effort for emotion detection from the conversational text in the Urdu language domain.     

AI设计下的智能驾驶场景文本识别技术

目的 交通标志识别作为智能驾驶、交通系统研究中的一项重要内容,具有较大的理论价值和应用前景.尤其是文本型交通标志,其含有丰富的高层语义信息,能够提供极其丰富的道路信息.因此通过设计并实现一套新的端到端交通标志文本识别系统,达到有效缓解交通拥堵、提高道路安全的目的.方法 系统主要包括文本区域检测和文字识别两个视觉任务,并基于卷积神经网络的深度学习技术实现.首先以ResNet-50为骨干网络提取特征,并采用类FPN结构进行多层特征融合,将融合后的特征作为文本检测和识别的共享特征.文本检测定位文本区域并输出候选文本框的坐标,文字识别输出词条对应的文本字符串.结果 通过实验验证,系统在Traffic Guide Panel Dataset上取得了令人满意的结果,行识别准确率为71.08％.结论 端到端交通标志文本识别非常具有现实意义.通过卷积神经网络的深度学习技术,提出了一套端到端交通标志文本识别系统,并在开源的Traffic Guide Panel Dataset上证明了该系统的优越性.     

Analyzing Arabic Twitter-Based Patient Experience Sentiments Using Multi-Dialect Arabic Bidirectional Encoder Representations from Transformers

Healthcare organizations rely on patients’ feedback and experiences to evaluate their performance and services, thereby allowing such organizations to improve inadequate services and address any shortcomings. According to the literature, social networks and particularly Twitter are effective platforms for gathering public opinions. Moreover, recent studies have used natural language processing to measure sentiments in text segments collected from Twitter to capture public opinions about various sectors, including healthcare. The present study aimed to analyze Arabic Twitter-based patient experience sentiments and to introduce an Arabic patient experience corpus. The authors collected 12,400 tweets from Arabic patients discussing patient experiences related to healthcare organizations in Saudi Arabia from 1 January 2008 to 29 January 2022. The tweets were labeled according to sentiment (positive or negative) and sector (public or private), and thereby the Hospital Patient Experiences in Saudi Arabia (HoPE-SA) dataset was produced. A simple statistical analysis was conducted to examine differences in patient views of healthcare sectors. The authors trained five models to distinguish sentiments in tweets automatically with the following schemes: a transformer-based model fine-tuned with deep learning architecture and a transformer-based model fine-tuned with simple architecture, using two different transformer-based embeddings based on Bidirectional Encoder Representations from Transformers (BERT), Multi-dialect Arabic BERT (MARBERT), and multilingual BERT (mBERT), as well as a pre-trained word2vec model with a support vector machine classifier. This is the first study to investigate the use of a bidirectional long short-term memory layer followed by a feedforward neural network for the fine-tuning of MARBERT. The deep-learning fine-tuned MARBERT-based model—the authors’ best-performing model—achieved accuracy, micro-F1, and macro-F1 scores of 98.71%, 98.73%, and 98.63%, respectively.     

RBEBT: A ResNet-Based BA-ELM for Brain Tumor Classification

Brain tumor refers to the formation of abnormal cells in the brain. It can be divided into benign and malignant. The main diagnostic methods for brain tumors are plain X-ray film, Magnetic resonance imaging (MRI), and so on. However, these artificial diagnosis methods are easily affected by external factors. Scholars have made such impressive progress in brain tumors classification by using convolutional neural network (CNN). However, there are still some problems: (i) There are many parameters in CNN, which require much calculation. (ii) The brain tumor data sets are relatively small, which may lead to the overfitting problem in CNN. In this paper, our team proposes a novel model (RBEBT) for the automatic classification of brain tumors. We use fine-tuned ResNet18 to extract the features of brain tumor images. The RBEBT is different from the traditional CNN models in that the randomized neural network (RNN) is selected as the classifier. Meanwhile, our team selects the bat algorithm (BA) to optimize the parameters of RNN. We use five-fold cross-validation to verify the superiority of the RBEBT. The accuracy (ACC), specificity (SPE), precision (PRE), sensitivity (SEN), and F1-score (F1) are 99.00%, 95.00%, 99.00%, 100.00%, and 100.00%. The classification performance of the RBEBT is greater than 95%, which can prove that the RBEBT is an effective model to classify brain tumors.     

Optimized Convolutional Neural Network Models for Skin Lesion Classification

Skin cancer is one of the most severe diseases, and medical imaging is among the main tools for cancer diagnosis. The images provide information on the evolutionary stage, size, and location of tumor lesions. This paper focuses on the classification of skin lesion images considering a framework of four experiments to analyze the classification performance of Convolutional Neural Networks (CNNs) in distinguishing different skin lesions. The CNNs are based on transfer learning, taking advantage of ImageNet weights. Accordingly, in each experiment, different workflow stages are tested, including data augmentation and fine-tuning optimization. Three CNN models based on DenseNet-201, Inception-ResNet-V2, and Inception-V3 are proposed and compared using the HAM10000 dataset. The results obtained by the three models demonstrate accuracies of 98%, 97%, and 96%, respectively. Finally, the best model is tested on the ISIC 2019 dataset showing an accuracy of 93%. The proposed methodology using CNN represents a helpful tool to accurately diagnose skin cancer disease.     

A Novel Hybrid Machine Learning Approach for Classification of Brain Tumor Images

Abnormal growth of brain tissues is the real cause of brain tumor. Strategy for the diagnosis of brain tumor at initial stages is one of the key step for saving the life of a patient. The manual segmentation of brain tumor magnetic resonance images (MRIs) takes time and results vary significantly in low-level features. To address this issue, we have proposed a ResNet-50 feature extractor depended on multilevel deep convolutional neural network (CNN) for reliable images segmentation by considering the low-level features of MRI. In this model, we have extracted features through ResNet-50 architecture and fed these feature maps to multi-level CNN model. To handle the classification process, we have collected a total number of 2043 MRI patients of normal, benign, and malignant tumor. Three model CNN, multi-level CNN, and ResNet-50 based multi-level CNN have been used for detection and classification of brain tumors. All the model results are calculated in terms of various numerical values identified as precision (P), recall (R), accuracy (Acc) and f1-score (F1-S). The obtained average results are much better as compared to already existing methods. This modified transfer learning architecture might help the radiologists and doctors as a better significant system for tumor diagnosis.     

Image Generation of Tomato Leaf Disease Identification Based on Small-ACGAN

Plant diseases have become a challenging threat in the agricultural field. Various learning approaches for plant disease detection and classification have been adopted to detect and diagnose these diseases early. However, deep learning entails extensive data for training, and it may be challenging to collect plant datasets. Even though plant datasets can be collected, they may be uneven in quantity. As a result, the problem of classification model overfitting arises. This study targets this issue and proposes an auxiliary classifier GAN (small-ACGAN) model based on a small number of datasets to extend the available data. First, after comparing various attention mechanisms, this paper chose to add the lightweight Coordinate Attention (CA) to the generator module of Auxiliary Classifier GANs (ACGAN) to improve the image quality. Then, a gradient penalty mechanism was added to the loss function to improve the training stability of the model. Experiments show that the proposed method can best improve the recognition accuracy of the classifier with the doubled dataset. On AlexNet, the accuracy was increased by 11.2%. In addition, small-ACGAN outperformed the other three GANs used in the experiment. Moreover, the experimental accuracy, precision, recall, and F1 scores of the five convolutional neural network (CNN) classifiers on the enhanced dataset improved by an average of 3.74%, 3.48%, 3.74%, and 3.80% compared to the original dataset. Furthermore, the accuracy of MobileNetV3 reached 97.9%, which fully demonstrated the feasibility of this approach. The general experimental results indicate that the method proposed in this paper provides a new dataset expansion method for effectively improving the identification accuracy and can play an essential role in expanding the dataset of the sparse number of plant diseases.     

LPNet: A lightweight CNN with discrete wavelet pooling strategies for colon polyps classification

The traditional process of disease diagnosis from medical images follows a manual process, which is tedious and arduous. A computer-aided diagnosis (CADs) system can work as an assistive tool to improve the diagnosis process. In this pursuit, this article introduces a unique architecture LPNet for classifying colon polyps from the colonoscopy video frames. Colon polyps are abnormal growth of cells in the colon wall. Over time, untreated colon polyps may cause colorectal cancer. Different convolutional neural networks (CNNs) based systems have been developed in recent years. However, CNN uses pooling to reduce the number of parameters and expand the receptive field. On the other hand, pooling results in data loss and is deleterious to subsequent processes. Pooling strategies based on discrete wavelet operations have been proposed in our architecture as a solution to this problem, with the promise of achieving a better trade-off between receptive field size and computing efficiency. The overall performance of this model is superior to the others, according to experimental results on a colonoscopy dataset. LPNet with bio-orthogonal wavelet achieved the highest performance with an accuracy of 93.55%. It outperforms the other state-of-the-art (SOTA) CNN models for the polyps classification task, and it is lightweight in terms of the number of learnable parameters compared with them, making the model easily deployable in edge devices.     

Anomaly Based Camera Prioritization in Large Scale Surveillance Networks

Digital surveillance systems are ubiquitous and continuously generate massive amounts of data, and manual monitoring is required in order to recognise human activities in public areas. Intelligent surveillance systems that can automatically ide.pngy normal and abnormal activities are highly desirable, as these would allow for efficient monitoring by selecting only those camera feeds in which abnormal activities are occurring. This paper proposes an energy-efficient camera prioritisation framework that intelligently adjusts the priority of cameras in a vast surveillance network using feedback from the activity recognition system. The proposed system addresses the limitations of existing manual monitoring surveillance systems using a three-step framework. In the first step, the salient frames are selected from the online video stream using a frame differencing method. A lightweight 3D convolutional neural network (3DCNN) architecture is applied to extract spatio-temporal features from the salient frames in the second step. Finally, the probabilities predicted by the 3DCNN network and the metadata of the cameras are processed using a linear threshold gate sigmoid mechanism to control the priority of the camera. The proposed system performs well compared to state-of-the-art violent activity recognition methods in terms of efficient camera prioritisation in large-scale surveillance networks. Comprehensive experiments and an evaluation of activity recognition and camera prioritisation showed that our approach achieved an accuracy of 98% with an F1-score of 0.97 on the Hockey Fight dataset, and an accuracy of 99% with an F1-score of 0.98 on the Violent Crowd dataset.     

Convolutional Neural Network for Histopathological Osteosarcoma Image Classification

Osteosarcoma is one of the most widespread causes of bone cancer globally and has a high mortality rate. Early diagnosis may increase the chances of treatment and survival however the process is time-consuming (reliability and complexity involved to extract the hand-crafted features) and largely depends on pathologists’ experience. Convolutional Neural Network (CNN—an end-to-end model) is known to be an alternative to overcome the aforesaid problems. Therefore, this work proposes a compact CNN architecture that has been rigorously explored on a Small Osteosarcoma histology Image Dataaseet (a high-class imbalanced dataset). Though, during training, class-imbalanced data can negatively affect the performance of CNN. Therefore, an oversampling technique has been proposed to overcome the aforesaid issue and improve generalization performance. In this process, a hierarchical CNN model is designed, in which the former model is non-regularized (due to dense architecture) and the later one is regularized, specifically designed for small histopathology images. Moreover, the regularized model is integrated with CNN’s basic architecture to reduce overfitting. Experimental results demonstrate that oversampling might be an effective way to address the imbalanced class problem during training. The training and testing accuracies of the non-regularized CNN model are 98% & 78% with an imbalanced dataset and 96% & 81% with a balanced dataset, respectively. The regularized CNN model training and testing accuracies are 84% & 75% for an imbalanced dataset and 87% & 86% for a balanced dataset.     

Type 2 Diabetes Risk Prediction Using Deep Convolutional Neural Network Based-Bayesian Optimization

Diabetes mellitus is a long-term condition characterized by hyperglycemia. It could lead to plenty of difficulties. According to rising morbidity in recent years, the world’s diabetic patients will exceed 642 million by 2040, implying that one out of every ten persons will be diabetic. There is no doubt that this startling figure requires immediate attention from industry and academia to promote innovation and growth in diabetes risk prediction to save individuals’ lives. Due to its rapid development, deep learning (DL) was used to predict numerous diseases. However, DL methods still suffer from their limited prediction performance due to the hyperparameters selection and parameters optimization. Therefore, the selection of hyper-parameters is critical in improving classification performance. This study presents Convolutional Neural Network (CNN) that has achieved remarkable results in many medical domains where the Bayesian optimization algorithm (BOA) has been employed for hyperparameters selection and parameters optimization. Two issues have been investigated and solved during the experiment to enhance the results. The first is the dataset class imbalance, which is solved using Synthetic Minority Oversampling Technique (SMOTE) technique. The second issue is the model's poor performance, which has been solved using the Bayesian optimization algorithm. The findings indicate that the Bayesian based-CNN model superbases all the state-of-the-art models in the literature with an accuracy of 89.36%, F1-score of 0.88.6, and Matthews Correlation Coefficient (MCC) of 0.88.6.