Detection of COVID-19 from chest X-ray images: Boosting the performance with convolutional neural network and transfer learning

Coronavirus disease (COVID-19) is a pandemic that has caused thousands of casualties and impacts all over the world. Most countries are facing a shortage of COVID-19 test kits in hospitals due to the daily increase in the number of cases. Early detection of COVID-19 can protect people from severe infection. Unfortunately, COVID-19 can be misdiagnosed as pneumonia or other illness and can lead to patient death. Therefore, in order to avoid the spread of COVID-19 among the population, it is necessary to implement an automated early diagnostic system as a rapid alternative diagnostic system. Several researchers have done very well in detecting COVID-19; however, most of them have lower accuracy and overfitting issues that make early screening of COVID-19 difficult. Transfer learning is the most successful technique to solve this problem with higher accuracy. In this paper, we studied the feasibility of applying transfer learning and added our own classifier to automatically classify COVID-19 because transfer learning is very suitable for medical imaging due to the limited availability of data. In this work, we proposed a CNN model based on deep transfer learning technique using six different pre-trained architectures, including VGG16, DenseNet201, MobileNetV2, ResNet50, Xception, and EfficientNetB0. A total of 3886 chest X-rays (1200 cases of COVID-19, 1341 healthy and 1345 cases of viral pneumonia) were used to study the effectiveness of the proposed CNN model. A comparative analysis of the proposed CNN models using three classes of chest X-ray datasets was carried out in order to find the most suitable model. Experimental results show that the proposed CNN model based on VGG16 was able to accurately diagnose COVID-19 patients with 97.84% accuracy, 97.90% precision, 97.89% sensitivity, and 97.89% of F1-score. Evaluation of the test data shows that the proposed model produces the highest accuracy among CNNs and seems to be the most suitable choice for COVID-19 classification. We believe that in this pandemic situation, this model will support healthcare professionals in improving patient screening.     

Leveraging Multimodal Ensemble Fusion-Based Deep Learning for COVID-19 on Chest Radiographs

Recently, COVID-19 has posed a challenging threat to researchers, scientists, healthcare professionals, and administrations over the globe, from its diagnosis to its treatment. The researchers are making persistent efforts to derive probable solutions for managing the pandemic in their areas. One of the widespread and effective ways to detect COVID-19 is to utilize radiological images comprising X-rays and computed tomography (CT) scans. At the same time, the recent advances in machine learning (ML) and deep learning (DL) models show promising results in medical imaging. Particularly, the convolutional neural network (CNN) model can be applied to identifying abnormalities on chest radiographs. While the epidemic of COVID-19, much research is led on processing the data compared with DL techniques, particularly CNN. This study develops an improved fruit fly optimization with a deep learning-enabled fusion (IFFO-DLEF) model for COVID-19 detection and classification. The major intention of the IFFO-DLEF model is to investigate the presence or absence of COVID-19. To do so, the presented IFFO-DLEF model applies image pre-processing at the initial stage. In addition, the ensemble of three DL models such as DenseNet169, EfficientNet, and ResNet50, are used for feature extraction. Moreover, the IFFO algorithm with a multilayer perceptron (MLP) classification model is utilized to identify and classify COVID-19. The parameter optimization of the MLP approach utilizing the IFFO technique helps in accomplishing enhanced classification performance. The experimental result analysis of the IFFO-DLEF model carried out on the CXR image database portrayed the better performance of the presented IFFO-DLEF model over recent approaches.     

Automatic Detection of COVID-19 Infection Using Chest X-Ray Images Through Transfer Learning

The new coronavirus(COVID-19),declared by the World Health Organization as a pandemic,has infected more than 1 million people and killed more than 50 thousand.An infection caused by COVID-19 can develop into pneumonia,which can be detected by a chest X-ray exam and should be treated appropriately.In this work,we propose an automatic detection method for COVID-19 infection based on chest X-ray images.The datasets constructed for this study are composed of194 X-ray images of patients diagnosed with coronavirus and 194 X-ray images of healthy patients.Since few images of patients with COVID-19 are publicly available,we apply the concept of transfer learning for this task.We use different architectures of convolutional neural networks(CNNs)trained on Image Net,and adapt them to behave as feature extractors for the X-ray images.Then,the CNNs are combined with consolidated machine learning methods,such as k-Nearest Neighbor,Bayes,Random Forest,multilayer perceptron(MLP),and support vector machine(SVM).The results show that,for one of the datasets,the extractor-classifier pair with the best performance is the Mobile Net architecture with the SVM classifier using a linear kernel,which achieves an accuracy and an F1-score of 98.5%.For the other dataset,the best pair is Dense Net201 with MLP,achieving an accuracy and an F1-score of 95.6%.Thus,the proposed approach demonstrates efficiency in detecting COVID-19 in X-ray images.     

深度特征融合方法及其在叶片病害识别中的应用

农作物叶片病害的自动识别是计算机视觉技术在农业领域的一个重要应用. 近年来, 深度学习在农作物叶片病害识别上取得了一些进展, 但这些方法都是采用基于单一深度卷积神经网络模型的深度特征表示. 而不同的深度卷积神经网络模型对图像的表征能力的互补性这一有用的特性, 还没有得到关注和研究. 本文提出一种用于融合不同深度特征的网络模型MDFF-Net. MDFF-Net将两个预训练的深度卷积神经网络模型进行并联, 再为各个模型分别设置一个具有相同神经元个数的全连接层, 以将不同模型输出的深度特征变换成相同维度的特征, 再通过2个全连接层的非线性变换, 进一步提升特征融合的效果. 我们选取VGG-16和ResNet-50作为MDFF-Net网络的并联骨干网络, 在一个包含5种苹果叶片病害的公开数据集上进行实验. 实验结果显示, MDFF-Net网络的识别精度为96.59%, 取得了比VGG-16和ResNet-50单一网络更好的识别效果, 证明了该深度特征融合方法的有效性.     

Corona-Nidaan: lightweight deep convolutional neural network for chest X-Ray based COVID-19 infection detection

The coronavirus COVID-19 pandemic is today’s major public health crisis, we have faced since the Second World War. The pandemic is spreading around the globe like a wave, and according to the World Health Organization’s recent report, the number of confirmed cases and deaths are rising rapidly. COVID-19 pandemic has created severe social, economic, and political crises, which in turn will leave long-lasting scars. One of the countermeasures against controlling coronavirus outbreak is specific, accurate, reliable, and rapid detection technique to identify infected patients. The availability and affordability of RT-PCR kits remains a major bottleneck in many countries, while handling COVID-19 outbreak effectively. Recent findings indicate that chest radiography anomalies can characterize patients with COVID-19 infection. In this study, Corona-Nidaan, a lightweight deep convolutional neural network (DCNN), is proposed to detect COVID-19, Pneumonia, and Normal cases from chest X-ray image analysis; without any human intervention. We introduce a simple minority class oversampling method for dealing with imbalanced dataset problem. The impact of transfer learning with pre-trained CNNs on chest X-ray based COVID-19 infection detection is also investigated. Experimental analysis shows that Corona-Nidaan model outperforms prior works and other pre-trained CNN based models. The model achieved 95% accuracy for three-class classification with 94% precision and recall for COVID-19 cases. While studying the performance of various pre-trained models, it is also found that VGG19 outperforms other pre-trained CNN models by achieving 93% accuracy with 87% recall and 93% precision for COVID-19 infection detection. The model is evaluated by screening the COVID-19 infected Indian Patient chest X-ray dataset with good accuracy.

     

多感知兴趣区域特征融合的图像识别方法

针对自然图像识别过程中不同深度学习模型关注兴趣区域不同的现象,本文引入深度卷积神经网络融合机制,结合深度迁移学习方法,给出了一种基于多感知兴趣区域特征融合的图像识别方法.本文将迁移学习方法引入牛津大学视觉组网络模型(visual geometry group network,VGGNet)和残差网络模型(residua...     

Real-time COVID-19 detection over chest x-ray images in edge computing

Severe Coronavirus Disease 2019 (COVID-19) has been a global pandemic which provokes massive devastation to the society, economy, and culture since January 2020. The pandemic demonstrates the inefficiency of superannuated manual detection approaches and inspires novel approaches that detect COVID-19 by classifying chest x-ray (CXR) images with deep learning technology. Although a wide range of researches about bran-new COVID-19 detection methods that classify CXR images with centralized convolutional neural network (CNN) models have been proposed, the latency, privacy, and cost of information transmission between the data resources and the centralized data center will make the detection inefficient. Hence, in this article, a COVID-19 detection scheme via CXR images classification with a lightweight CNN model called MobileNet in edge computing is proposed to alleviate the computing pressure of centralized data center and ameliorate detection efficiency. Specifically, the general framework is introduced first to manifest the overall arrangement of the computing and information services ecosystem. Then, an unsupervised model DCGAN is employed to make up for the small scale of data set. Moreover, the implementation of the MobileNet for CXR images classification is presented at great length. The specific distribution strategy of MobileNet models is followed. The extensive evaluations of the experiments demonstrate the efficiency and accuracy of the proposed scheme for detecting COVID-19 over CXR images in edge computing.     

X-Ray Covid-19 Detection Based on Scatter Wavelet Transform and Dense Deep Neural Network

Notwithstanding the discovery of vaccines for Covid-19, the virus's rapid spread continues due to the limited availability of vaccines, especially in poor and emerging countries. Therefore, the key issues in the present COVID-19 pandemic are the early identification of COVID-19, the cautious separation of infected cases at the lowest cost and curing the disease in the early stages. For that reason, the methodology adopted for this study is imaging tools, particularly computed tomography, which have been critical in diagnosing and treating the disease. A new method for detecting Covid-19 in X-rays and CT images has been presented based on the Scatter Wavelet Transform and Dense Deep Neural Network. The Scatter Wavelet Transform has been employed as a feature extractor, while the Dense Deep Neural Network is utilized as a binary classifier. An extensive experiment was carried out to evaluate the accuracy of the proposed method over three datasets: IEEE 80200, Kaggle, and Covid-19 X-ray image data Sets. The dataset used in the experimental part consists of 14142. The numbers of training and testing images are 8290 and 2810, respectively. The analysis of the result refers that the proposed methods achieved high accuracy of 98%. The proposed model results show an excellent outcome compared to other methods in the same domain, such as (DeTraC) CNN, which achieved only 93.1%, CNN, which achieved 94%, and stacked Multi-Resolution CovXNet, which achieved 97.4%. The accuracy of CapsNet reached 97.24%.     

Ground object information extraction from hyperspectral remote sensing images using deep learning algorithm

Mining and utilizing coal resources play an influential role in economic development. In this regard, the feature information extraction in the area is researched to accurately and efficiently assist the production arrangement and deployment in the mining area. First, the detection ability of Hyperspectral Remote Sensing Image (HRSI) technology is analyzed. It has high spectral resolution and many bands. Specific bands can be extracted as needed to highlight target features. According to the characteristics of HRSIs, the data spectrum information and spatial information are comprehensively utilized, and the Convolutional Neural Network (CNN) based on deep learning is employed for feature extraction. CNN allows the machine to automatically obtain data features by learning and guide the classification of features. Taking the Liuyuan research area in Gansu as an example, three CNN models are used to extract and classify the ground features in the area. The VGG-19 model can provide the highest classification accuracy rate, reaching 87.3%; the VGG-16 model has the highest classification accuracy rate of the ground in the mining area, reaching 95.2%. ResNet model has the best effect on road classification. Then, the lithology classification is applied based on Thermal Airborne Hyperspectral Imager (TASI) data. The noise level of the first 20 bands is comparatively stable; afterward, it increases exponentially, showing a higher noise level, and the spectrum curve of the data after denoising becomes smoother. The end-member extraction method is employed to extract 25 end-member spectra of almost all lithology in the research area from the image. The similarity coefficient clustering analysis is employed to group the curves, which are divided into six categories in total. The separability of similar categories can be constrained by the objective function using the dictionary learning method, and the accuracy of the sparse representation of the category spectrum can be improved. The spectral matching method is used to subdivide each group of mapping results, suggesting that in the research area, granite is the most widely distributed, followed by diorite, andesite, and quartzite. Deep learning algorithms are applied to extract ground feature information, which is of great significance to the safety production in the mining area. The hyperspectral remote sensing rock and mineral thematic information extraction module is developed, which preliminarily realizes the quantitative acquisition and high-precision identification of typical mineral information, and provides technical support for the research of remote sensing geological evaluation technology of resource exploration in the new era.     

基于VGG-NET的特征融合面部表情识别

为了解决在面部表情特征提取过程中卷积神经网络CNN和局部二值模式LBP只能提取面部表情图像的单一特征,难以提取与面部变化高度相关的精确特征的问题,提出了一种基于深度学习的特征融合的表情识别方法。该方法将LBP特征和CNN卷积层提取的特征通过加权的方式结合在改进的VGG-16网络连接层中,最后将融合特征送入Softmax分类器获取各类特征的概率,完成基本的6种表情分类。实验结果表明,所提方法在CK+和JAFFE数据集上的平均识别准确率分别达到了97.5%和97.62%,利用融合特征得到的识别结果明显优于利用单一特征识别的效果。与其他方法相比较,该方法能有效提高表情识别准确率,对光照变化更加鲁棒。     

融合频率域特征的双路网络模型诊断新冠肺炎

CT检查在新冠肺炎诊断中起着重要作用,为了能够在有限的CT胸部图像集中获得更多有关新冠肺炎的特征信息、建立更加敏感通用的诊断模型,提出了融合CT图像频域特征的双路网络模型（Dp-Net）,该模型主干部分采用ResNet网络模型,并将卷积神经网络的训练过程分为两个部分,一部分提取CT图像空间域的特征,另一部分通过傅里叶变换提取频率域上的特征,将两者训练的结果按照一定的权重进行融合,融合后再由Layer4模块进行一次特征提取。在公开的COVID-CT数据集上与ResNet、VGG等传统的CNN模型进行了比较,也与Self-Trans和LA-DNN等一些改进的CNN模型进行了比较,并对不同权重的融合方案进行了比较,实验结果表明提出的Dp-Net模型在各种评价指标上取得了更好的结果。     

融合多头注意力机制的新冠肺炎联合诊断与分割

目的 新冠肺炎疫情席卷全球,为快速诊断肺炎患者,确认患者肺部感染区域,大量检测网络相继提出,但现有网络大多只能处理一种任务,即诊断或分割。本文提出了一种融合多头注意力机制的联合诊断与分割网络,能同时完成X线胸片的肺炎诊断分类和新冠感染区分割。方法 整个网络由3部分组成,双路嵌入层通过两种不同的图像嵌入方式分别提取X线胸片的浅层直观特征和深层抽象特征;Transformer模块综合考虑提取到的浅层直观与深层抽象特征;分割解码器扩大特征图以输出分割区域。为响应联合训练,本文使用了一种混合损失函数以动态平衡分类与分割的训练。分类损失定义为分类对比损失与交叉熵损失的和;分割损失是二分类的交叉熵损失。结果 基于6个公开数据集的合并数据实验结果表明,所提网络取得了95.37%的精度、96.28%的召回率、95.95%的F1指标和93.88%的kappa系数,诊断分类性能超过了主流的ResNet50、VGG16(Visual Geometry Group)和Inception＿v3等网络;在新冠病灶分割表现上,相比流行的U-Net及其改进网络,取得最高的精度(95.96%),优异的敏感度(78.89...     

基于卷积词袋网络的视觉识别

近年来,卷积神经网络（CNN）凭借其强大的特征学习能力在视觉识别领域取得重要进展。针对CNN全连接层对图像平移、旋转、缩放等变换比较敏感的问题,提出了一种混合模型--卷积词袋网络（BoCW-Net）。它将BoW模型嵌入CNN结构中并代替全连接层,通过端到端的方式学习特征、字典和分类器。为实现BoCW-Net整个网络的有监督学习,提出基于方向相似度的BoCW编码。同时,为充分利用中层特征和高层特征的鉴别性,将中层辅助分类器与高层分类器集成,形成主-辅集成分类器。实验结果表明：相比全连接层,BoCW表示对各种变换具有更强的不变性;主-辅集成分类器能有效融合中层、高层特征,提高BoCW-Net的识别性能;相比新近发展的CNN模型,BoCW-Net在CIFAR-10、CIFAR-100和MNIST数据库上均取得了改进的识别性能,最终分别获得4.88%、22.48%和0.21%的测试错误率。     

COVID TCL: A Joint Metric Loss Function for Diagnosing COVID-19 Patient in the Early and Incubation Period

Convolution Neural Networks (CNN) can quickly diagnose COVID-19 patients by analyzing computed tomography (CT) images of the lung, thereby effectively preventing the spread of COVID-19. However, the existing CNN-based COVID-19 diagnosis models do consider the problem that the lung images of COVID-19 patients in the early stage and incubation period are extremely similar to those of the non-COVID-19 population. Which reduces the model’s classification sensitivity, resulting in a higher probability of the model misdiagnosing COVID-19 patients as non-COVID-19 people. To solve the problem, this paper first attempts to apply triplet loss and center loss to the field of COVID-19 image classification, combining softmax loss to design a jointly supervised metric loss function COVID Triplet-Center Loss (COVID-TCL). Triplet loss can increase inter-class discreteness, and center loss can improve intra-class compactness. Therefore, COVID-TCL can help the CNN-based model to extract more discriminative features and strengthen the diagnostic capacity of COVID-19 patients in the early stage and incubation period. Meanwhile, we use the extreme gradient boosting (XGBoost) as a classifier to design a COVID-19 images classification model of CNN-XGBoost architecture, to further improve the CNN-based model’s classification effect and operation efficiency. The experiment shows that the classification accuracy of the model proposed in this paper is 97.41%, and the sensitivity is 97.61%, which is higher than the other 7 reference models. The COVID-TCL can effectively improve the classification sensitivity of the CNN-based model, the CNN-XGBoost architecture can further improve the CNN-based model’s classification effect.     

基于卷积长短时记忆网络的人体行为识别研究

人体行为识别利用深度学习网络模型自动提取数据的深层特征,但传统机器学习算法存在依赖手工特征提取、模型泛化能力差等问题。提出基于空时特征融合的深度学习模型（CLT-net）用于人体行为识别。采用卷积神经网络（CNN）自动提取人体行为数据的深层次隐含特征,利用长短时记忆（LSTM）网络构建时间序列模型,学习人体行为特征在时间序列上的长期依赖关系。在此基础上,通过softmax分类器实现对不同人体行为分类。在DaLiAc数据集的实验结果表明,相比CNN、LSTM、BP模型,CLT-net模型对13种人体行为的总体识别率达到了97.6%,具有较优的人体行为识别分类性能。     

Histogram Matched Chest X-Rays Based Tuberculosis Detection Using CNN

Tuberculosis (TB) is a severe infection that mostly affects the lungs and kills millions of people’s lives every year. Tuberculosis can be diagnosed using chest X-rays (CXR) and data-driven deep learning (DL) approaches. Because of its better automated feature extraction capability, convolutional neural networks (CNNs) trained on natural images are particularly effective in image categorization. A combination of 3001 normal and 3001 TB CXR images was gathered for this study from different accessible public datasets. Ten different deep CNNs (Resnet50, Resnet101, Resnet152, InceptionV3, VGG16, VGG19, DenseNet121, DenseNet169, DenseNet201, MobileNet) are trained and tested for identifying TB and normal cases. This study presents a deep CNN approach based on histogram matched CXR images that does not require object segmentation of interest, and this coupled methodology of histogram matching with the CXRs improves the accuracy and detection performance of CNN models for TB detection. Furthermore, this research contains two separate experiments that used CXR images with and without histogram matching to classify TB and non-TB CXRs using deep CNNs. It was able to accurately detect TB from CXR images using pre-processing, data augmentation, and deep CNN models. Without histogram matching the best accuracy, sensitivity, specificity, precision and F1-score in the detection of TB using CXR images among ten models are 99.25%, 99.48%, 99.52%, 99.48% and 99.22% respectively. With histogram matching the best accuracy, sensitivity, specificity, precision and F1-score are 99.58%, 99.82%, 99.67%, 99.65% and 99.56% respectively. The proposed methodology, which has cutting-edge performance, will be useful in computer-assisted TB diagnosis and aids in minimizing irregularities in TB detection in developing countries.     

基于连续小波和支持向量机分类音乐类型

音乐类型分类主要包括两个阶段：特征提取和分类。文中在研究小波变换理论基础上,采用连续小波分析方法提取音乐特征参数。支持向量机是专门针对有限样本情况下的一种分类方法。它是建立在统计学习理论的VC维理论和结构风险最小原理基础上,根据有限的样本信息在模型的复杂性和学习能力之间寻求最佳折衷,以期获得最好的推广能力。采用指数径向基函数（脚）内核,分类正确率可达85％,比传统的混合高斯模型和K近邻分类器,分类性能分别提高了21％和23％。实验结果表明,采用小波和支持向量机方法是一种相当有效的音乐类型分类方法。     

Sentiment analysis of tweets using a unified convolutional neural network‐long short‐term memory network model

Sentiment analysis focuses on identifying and classifying the sentiments expressed in text messages and reviews. Social networks like Twitter, Facebook, and Instagram generate heaps of data filled with sentiments, and the analysis of such data is very fruitful when trying to improve the quality of both products and services alike. Classic machine learning techniques have a limited capability to efficiently analyze such large amounts of data and produce precise results; they are thus supported by deep learning models to achieve higher accuracy. This study proposes a combination of convolutional neural network and long short‐term memory (CNN‐LSTM) deep network for performing sentiment analysis on Twitter datasets. The performance of the proposed model is analyzed with machine learning classifiers, including the support vector classifier, random forest (RF), stochastic gradient descent (SGD), logistic regression, a voting classifier (VC) of RF and SGD, and state‐of‐the‐art classifier models. Furthermore, two feature extraction methods (term frequency‐inverse document frequency and word2vec) are also investigated to determine their impact on prediction accuracy. Three datasets (US airline sentiments, women's e‐commerce clothing reviews, and hate speech) are utilized to evaluate the performance of the proposed model. Experiment results demonstrate that the CNN‐LSTM achieves higher accuracy than those of other classifiers.