Software Defect Prediction Based on Stacked Contractive Autoencoder and Multi-Objective Optimization

Software defect prediction plays an important role in software quality assurance. However, the performance of the prediction model is susceptible to the irrelevant and redundant features. In addition, previous studies mostly regard software defect prediction as a single objective optimization problem, and multi-objective software defect prediction has not been thoroughly investigated. For the above two reasons, we propose the following solutions in this paper: (1) we leverage an advanced deep neural network—Stacked Contractive AutoEncoder (SCAE) to extract the robust deep semantic features from the original defect features, which has stronger discrimination capacity for different classes (defective or non-defective). (2) we propose a novel multi-objective defect prediction model named SMONGE that utilizes the Multi-Objective NSGAII algorithm to optimize the advanced neural network—Extreme learning machine (ELM) based on state-of-the-art Pareto optimal solutions according to the features extracted by SCAE. We mainly consider two objectives. One objective is to maximize the performance of ELM, which refers to the benefit of the SMONGE model. Another objective is to minimize the output weight norm of ELM, which is related to the cost of the SMONGE model. We compare the SCAE with six state-of-the-art feature extraction methods and compare the SMONGE model with multiple baseline models that contain four classic defect predictors and the MONGE model without SCAE across 20 open source software projects. The experimental results verify that the superiority of SCAE and SMONGE on seven evaluation metrics.     

Deep Feature Fusion Model for Sentence Semantic Matching

Sentence semantic matching (SSM) is a fundamental research in solving natural language processing tasks such as question answering and machine translation. The latest SSM research benefits from deep learning techniques by incorporating attention mechanism to semantically match given sentences. However, how to fully capture the semantic context without losing significant features for sentence encoding is still a challenge. To address this challenge, we propose a deep feature fusion model and integrate it into the most popular deep learning architecture for sentence matching task. The integrated architecture mainly consists of embedding layer, deep feature fusion layer, matching layer and prediction layer. In addition, we also compare the commonly used loss function, and propose a novel hybrid loss function integrating MSE and cross entropy together, considering confidence interval and threshold setting to preserve the indistinguishable instances in training process. To evaluate our model performance, we experiment on two real world public data sets: LCQMC and Quora. The experiment results demonstrate that our model outperforms the most existing advanced deep learning models for sentence matching, benefited from our enhanced loss function and deep feature fusion model for capturing semantic context.     

Stacked denoising autoencoder‐based feature learning for out‐of‐control source recognition in multivariate manufacturing process

In multivariate statistical process control (MSPC), regular multivariate control charts (eg, T²) are shown to be effective in detecting out‐of‐control signals based upon an overall statistic. But these charts do not relieve the need for multivariate process pattern recognition (MPPR). MPPR would be very useful for quality operators to locate the assignable causes that give rise to out‐of‐control situation in multivariate manufacturing process. Deep learning has been widely applied and obtained many successes in image and visual analysis. This paper presents an effective and reliable deep learning method known as stacked denoising autoencoder (SDAE) for MPPR in manufacturing processes. This study will concentrate on developing a SDAE model to learn effective discriminative features from the process signals through deep network architectures. Feature visualization is performed to explicitly present feature representations of the proposed SDAE model. The experimental results illustrate that the proposed SDAE model is capable of implementing detection and recognition of various process patterns in complicated multivariate processes. Analysis from this study provides the guideline in developing deep learning‐based MSPC systems.     

Deep Learning Enabled Autoencoder Architecture for Collaborative Filtering Recommendation in IoT Environment

The era of the Internet of things (IoT) has marked a continued exploration of applications and services that can make people’s lives more convenient than ever before. However, the exploration of IoT services also means that people face unprecedented difficulties in spontaneously selecting the most appropriate services. Thus, there is a paramount need for a recommendation system that can help improve the experience of the users of IoT services to ensure the best quality of service. Most of the existing techniques—including collaborative filtering (CF), which is most widely adopted when building recommendation systems—suffer from rating sparsity and cold-start problems, preventing them from providing high quality recommendations. Inspired by the great success of deep learning in a wide range of fields, this work introduces a deep-learning-enabled autoencoder architecture to overcome the setbacks of CF recommendations. The proposed deep learning model is designed as a hybrid architecture with three key networks, namely autoencoder (AE), multilayered perceptron (MLP), and generalized matrix factorization (GMF). The model employs two AE networks to learn deep latent feature representations of users and items respectively and in parallel. Next, MLP and GMF networks are employed to model the linear and non-linear user-item interactions respectively with the extracted latent user and item features. Finally, the rating prediction is performed based on the idea of ensemble learning by fusing the output of the GMF and MLP networks. We conducted extensive experiments on two benchmark datasets, MoiveLens100K and MovieLens1M, using four standard evaluation metrics. Ablation experiments were conducted to confirm the validity of the proposed model and the contribution of each of its components in achieving better recommendation performance. Comparative analyses were also carried out to demonstrate the potential of the proposed model in gaining better accuracy than the existing CF methods with resistance to rating sparsity and cold-start problems.     

Enhanced Deep Autoencoder Based Feature Representation Learning for Intelligent Intrusion Detection System

In the era of Big data, learning discriminant feature representation from network traffic is identified has as an invariably essential task for improving the detection ability of an intrusion detection system (IDS). Owing to the lack of accurately labeled network traffic data, many unsupervised feature representation learning models have been proposed with state-of-the-art performance. Yet, these models fail to consider the classification error while learning the feature representation. Intuitively, the learnt feature representation may degrade the performance of the classification task. For the first time in the field of intrusion detection, this paper proposes an unsupervised IDS model leveraging the benefits of deep autoencoder (DAE) for learning the robust feature representation and one-class support vector machine (OCSVM) for finding the more compact decision hyperplane for intrusion detection. Specially, the proposed model defines a new unified objective function to minimize the reconstruction and classification error simultaneously. This unique contribution not only enables the model to support joint learning for feature representation and classifier training but also guides to learn the robust feature representation which can improve the discrimination ability of the classifier for intrusion detection. Three set of evaluation experiments are conducted to demonstrate the potential of the proposed model. First, the ablation evaluation on benchmark dataset, NSL-KDD validates the design decision of the proposed model. Next, the performance evaluation on recent intrusion dataset, UNSW-NB15 signifies the stable performance of the proposed model. Finally, the comparative evaluation verifies the efficacy of the proposed model against recently published state-of-the-art methods.     

An Optimized Approach to Vehicle-Type Classification Using a Convolutional Neural Network

Vehicle type classification is considered a central part of an intelligent traffic system. In recent years, deep learning had a vital role in object detection in many computer vision tasks. To learn high-level deep features and semantics, deep learning offers powerful tools to address problems in traditional architectures of handcrafted feature-extraction techniques. Unlike other algorithms using handcrated visual features, convolutional neural network is able to automatically learn good features of vehicle type classification. This study develops an optimized automatic surveillance and auditing system to detect and classify vehicles of different categories. Transfer learning is used to quickly learn the features by recording a small number of training images from vehicle frontal view images. The proposed system employs extensive data-augmentation techniques for effective training while avoiding the problem of data shortage. In order to capture rich and discriminative information of vehicles, the convolutional neural network is fine-tuned for the classification of vehicle types using the augmented data. The network extracts the feature maps from the entire dataset and generates a label for each object (vehicle) in an image, which can help in vehicle-type detection and classification. Experimental results on a public dataset and our own dataset demonstrated that the proposed method is quite effective in detection and classification of different types of vehicles. The experimental results show that the proposed model achieves 96.04% accuracy on vehicle type classification.     

基于深度学习的易拉罐缺陷检测技术

目的 现有的易拉罐缺陷检测系统在高速生产线中存在错检率和漏检率高,检测精度相对较低等问题,为了提高易拉罐缺陷识别的准确性,使易拉罐生产线实现进一步自动化、智能化,基于深度学习技术和迁移学习技术,提出一种适用于易拉罐制造的在线检测的算法。方法 利用深度卷积网络提取易拉罐缺陷特征,通过优化卷积核,减短易拉罐缺陷检测的时间。针对国内外数据集缺乏食品包装制造的缺陷图像,构建易拉罐缺陷数据集,结合预训练网络,通过调整VGG16提升对易拉罐缺陷的识别准确率。结果 对易拉罐数据集在卷积神经网络、迁移学习和调整后的预训练网络进行了易拉罐缺陷检测的性能对比,验证了基于深度学习的易拉罐缺陷检测技术在学习率为0.0005,训练10个迭代后可达到较好的识别效果,最终二分类缺陷识别率为99.7%,算法耗时119 ms。结论 相较于现有的易拉罐检测算法,文中提出的基于深度学习的易拉罐检测算法的识别性能更优,智能化程度更高。同时,该研究有助于制罐企业利用深度学习等AI技术促进智能化生产,减少人力成本,符合国家制造业产业升级的策略,具有一定的实际意义。     

Deer Hunting Optimization with Deep Learning Model for Lung Cancer Classification

Lung cancer is the main cause of cancer related death owing to its destructive nature and postponed detection at advanced stages. Early recognition of lung cancer is essential to increase the survival rate of persons and it remains a crucial problem in the healthcare sector. Computer aided diagnosis (CAD) models can be designed to effectually identify and classify the existence of lung cancer using medical images. The recently developed deep learning (DL) models find a way for accurate lung nodule classification process. Therefore, this article presents a deer hunting optimization with deep convolutional neural network for lung cancer detection and classification (DHODCNN-LCC) model. The proposed DHODCNN-LCC technique initially undergoes pre-processing in two stages namely contrast enhancement and noise removal. Besides, the features extraction process on the pre-processed images takes place using the Nadam optimizer with RefineDet model. In addition, denoising stacked autoencoder (DSAE) model is employed for lung nodule classification. Finally, the deer hunting optimization algorithm (DHOA) is utilized for optimal hyper parameter tuning of the DSAE model and thereby results in improved classification performance. The experimental validation of the DHODCNN-LCC technique was implemented against benchmark dataset and the outcomes are assessed under various aspects. The experimental outcomes reported the superior outcomes of the DHODCNN-LCC technique over the recent approaches with respect to distinct measures.     

Data Mining with Comprehensive Oppositional Based Learning for Rainfall Prediction

Data mining process involves a number of steps from data collection to visualization to identify useful data from massive data set. the same time, the recent advances of machine learning (ML) and deep learning (DL) models can be utilized for effectual rainfall prediction. With this motivation, this article develops a novel comprehensive oppositional moth flame optimization with deep learning for rainfall prediction (COMFO-DLRP) Technique. The proposed CMFO-DLRP model mainly intends to predict the rainfall and thereby determine the environmental changes. Primarily, data pre-processing and correlation matrix (CM) based feature selection processes are carried out. In addition, deep belief network (DBN) model is applied for the effective prediction of rainfall data. Moreover, COMFO algorithm was derived by integrating the concepts of comprehensive oppositional based learning (COBL) with traditional MFO algorithm. Finally, the COMFO algorithm is employed for the optimal hyperparameter selection of the DBN model. For demonstrating the improved outcomes of the COMFO-DLRP approach, a sequence of simulations were carried out and the outcomes are assessed under distinct measures. The simulation outcome highlighted the enhanced outcomes of the COMFO-DLRP method on the other techniques.     

基于深度学习的舰船辐射噪声多特征融合分类

为了改善分类系统的性能,进一步提高舰船辐射噪声分类的正确率,该文提出了一种基于深度神经网络的多特征融合分类方法。该方法首先提取舰船辐射噪声几种不同的特征,将提取的特征同时用于训练具有多个输入分支的深度神经网络,使网络直接在多种特征参数上进行联合学习,通过神经网络的输入分支和连接层实现特征融合,再对舰船辐射噪声进行分类。为了特征深度学习提取了舰船辐射噪声的频谱特征、梅尔倒谱系数和功率谱特征,并将多特征融合分类方法与在一种特征上进行深度学习分类方法的正确率进行对比。实验结果表明,基于深度学习的多特征融合分类方法可以有效地提高舰船辐射噪声分类的正确率,是一种可行的分类方法。     

Big data driven jobs remaining time prediction in discrete manufacturing system: a deep learning-based approach

Implementing advanced big data (BD) analytic is significant for successful incorporation of artificial intelligence in manufacturing. With the widespread deployment of smart sensors and internet of things (IOT) in the job shop, there is an increasing need for handling manufacturing BD for predictive manufacturing. In this study, we conceive the jobs remaining time (JRT) prediction during manufacturing execution based on deep learning (DL) with production BD. We developed a procedure for JRT prediction that includes three parts: raw data collection, candidate dataset design and predictive modelling. First, the historical production data are collected by the widely deployed IOT in the job shop. Then, the candidate dataset is formalised to capture various contributory factors for JRT prediction. Further, a DL model named stacked sparse autoencoder (S-SAE) is constructed to learn representative features from high dimensional manufacturing BD to make robust and accurate JRT prediction. Our work represents the first DL model for the JRT prediction at run time during production. The proposed methods are applied in a large-scale job shop that is equipped with 44 machine tools and produces 13 types of parts. Lastly, the experimental results show the S-SAE model has higher accuracy than previous linear regression, back-propagation network, multi-layer network and deep belief network in JRT prediction.     

A Novel Combinational Convolutional Neural Network for Automatic Food-Ingredient Classification

With the development of deep learning and Convolutional Neural Networks (CNNs), the accuracy of automatic food recognition based on visual data have significantly improved. Some research studies have shown that the deeper the model is, the higher the accuracy is. However, very deep neural networks would be affected by the overfitting problem and also consume huge computing resources. In this paper, a new classification scheme is proposed for automatic food-ingredient recognition based on deep learning. We construct an up-to-date combinational convolutional neural network (CBNet) with a subnet merging technique. Firstly, two different neural networks are utilized for learning interested features. Then, a well-designed feature fusion component aggregates the features from subnetworks, further extracting richer and more precise features for image classification. In order to learn more complementary features, the corresponding fusion strategies are also proposed, including auxiliary classifiers and hyperparameters setting. Finally, CBNet based on the well-known VGGNet, ResNet and DenseNet is evaluated on a dataset including 41 major categories of food ingredients and 100 images for each category. Theoretical analysis and experimental results demonstrate that CBNet achieves promising accuracy for multi-class classification and improves the performance of convolutional neural networks.     

Centralized embedding hypersphere feature learning for person re-identification

ABSTRACT

Deep metric learning has become a general method for person re-identification (ReID) recently. Existing methods train ReID model with various loss functions to learn feature representation and identify pedestrian. However, the interaction between person features and classification vectors in the training process is rarely concerned. Distribution of pedestrian features will greatly affect convergence of the model and the pedestrian similarity computing in the test phase. In this paper, we formulate improved softmax function to learn pedestrian features and classification vectors. Our method applies pedestrian feature representation to be scattered across the coordinate space and embedding hypersphere to solve the classification problem. Then, we propose an end-to-end convolutional neural network (CNN) framework with improved softmax function to improve the performance of pedestrian features. Finally, experiments are performed on four challenging datasets. The results demonstrate that our work is competitive compared to the state-of-the-art.     

Image Recognition of Citrus Diseases Based on Deep Learning

In recent years, with the development of machine learning and deep learning, it is possible to identify and even control crop diseases by using electronic devices instead of manual observation. In this paper, an image recognition method of citrus diseases based on deep learning is proposed. We built a citrus image dataset including six common citrus diseases. The deep learning network is used to train and learn these images, which can effectively identify and classify crop diseases. In the experiment, we use MobileNetV2 model as the primary network and compare it with other network models in the aspect of speed, model size, accuracy. Results show that our method reduces the prediction time consumption and model size while keeping a good classification accuracy. Finally, we discuss the significance of using MobileNetV2 to identify and classify agricultural diseases in mobile terminal, and put forward relevant suggestions.     

Deep Semisupervised Learning-Based Network Anomaly Detection in Heterogeneous Information Systems

The extensive proliferation of modern information services and ubiquitous digitization of society have raised cybersecurity challenges to new levels. With the massive number of connected devices, opportunities for potential network attacks are nearly unlimited. An additional problem is that many low-cost devices are not equipped with effective security protection so that they are easily hacked and applied within a network of bots (botnet) to perform distributed denial of service (DDoS) attacks. In this paper, we propose a novel intrusion detection system (IDS) based on deep learning that aims to identify suspicious behavior in modern heterogeneous information systems. The proposed approach is based on a deep recurrent autoencoder that learns time series of normal network behavior and detects notable network anomalies. An additional feature of the proposed IDS is that it is trained with an optimized dataset, where the number of features is reduced by 94% without classification accuracy loss. Thus, the proposed IDS remains stable in response to slight system perturbations, which do not represent network anomalies. The proposed approach is evaluated under different simulation scenarios and provides a 99% detection accuracy over known datasets while reducing the training time by an order of magnitude.     

Deep learning and machine learning-based early survival predictions of glioblastoma patients using pre-operative three-dimensional brain magnetic resonance imaging modalities

To propose and implement an automated machine learning (ML) based methodology to predict the overall survival of glioblastoma multiforme (GBM) patients. In the proposed methodology, we used deep learning (DL) based 3D U-shaped Convolutional Neural Network inspired encoder-decoder architecture to segment the brain tumor. Further, feature extraction was performed on these segmented and raw magnetic resonance imaging (MRI) scans using a pre-trained 2D residual neural network. The dimension-reduced principal components were integrated with clinical data and the handcrafted features of tumor subregions to compare the performance of regression-based automated ML techniques. Through the proposed methodology, we achieved the mean squared error (MSE) of 87 067.328, median squared error of 30 915.66, and a SpearmanR correlation of 0.326 for survival prediction (SP) with the validation set of Multimodal Brain Tumor Segmentation 2020 dataset. These results made the MSE far better than the existing automated techniques for the same patients. Automated SP of GBM patients is a crucial topic with its relevance in clinical use. The results proved that DL-based feature extraction using 2D pre-trained networks is better than many heavily trained 3D and 2D prediction models from scratch. The ensembled approach has produced better results than single models. The most crucial feature affecting GBM patients' survival is the patient's age, as per the feature importance plots presented in this work. The most critical MRI modality for SP of GBM patients is the T2 fluid attenuated inversion recovery, as evident from the feature importance plots.     

AI设计下的智能驾驶场景文本识别技术

目的 交通标志识别作为智能驾驶、交通系统研究中的一项重要内容,具有较大的理论价值和应用前景.尤其是文本型交通标志,其含有丰富的高层语义信息,能够提供极其丰富的道路信息.因此通过设计并实现一套新的端到端交通标志文本识别系统,达到有效缓解交通拥堵、提高道路安全的目的.方法 系统主要包括文本区域检测和文字识别两个视觉任务,并基于卷积神经网络的深度学习技术实现.首先以ResNet-50为骨干网络提取特征,并采用类FPN结构进行多层特征融合,将融合后的特征作为文本检测和识别的共享特征.文本检测定位文本区域并输出候选文本框的坐标,文字识别输出词条对应的文本字符串.结果 通过实验验证,系统在Traffic Guide Panel Dataset上取得了令人满意的结果,行识别准确率为71.08％.结论 端到端交通标志文本识别非常具有现实意义.通过卷积神经网络的深度学习技术,提出了一套端到端交通标志文本识别系统,并在开源的Traffic Guide Panel Dataset上证明了该系统的优越性.     

基于改进YOLOv5的试剂卡印刷缺陷检测算法

目的 针对试剂卡生产企业采用人工分选印刷缺陷的试剂卡存在效率低、成本高、易漏检的问题，提出一种基于深度神经网络YOLOv5s的改进试剂卡印刷缺陷检测算法YOLOv5s-EF。方法 通过图像预处理算法获得高质量的缺陷图像数据集，在YOLOv5s的主干特征提取网络中添加高效通道注意力（Efficient Channel Attention, ECA）机制，增强特征图中重要特征的表示能力；引入焦点损失函数（Focal Loss）来缓解正负样本不均衡的影响；结合印刷区域的定位结果，二次精确定位并构建方位特征向量，提出一种特征向量相似度匹配方法。结果 实验结果表明，本文提出的试剂卡印刷缺陷检测算法在测试集上的检测平均准确度可以达到97.3%，速度为22.6帧/s。结论 相较于其他网络模型，本文提出的方法可以实现对多种印刷缺陷的识别与定位，模型具有较好的检测速度和鲁棒性，有利于提高企业生产的智能化水平。     

Design of Intelligent Mosquito Nets Based on Deep Learning Algorithms

An intelligent mosquito net employing deep learning has been one of the hotspots in the field of Internet of Things as it can reduce significantly the spread of pathogens carried by mosquitoes, and help people live well in mosquito-infested areas. In this study, we propose an intelligent mosquito net that can produce and transmit data through the Internet of Medical Things. In our method, decision-making is controlled by a deep learning model, and the proposed method uses infrared sensors and an array of pressure sensors to collect data. Moreover the ZigBee protocol is used to transmit the pressure map which is formed by pressure sensors with the deep learning perception model, determining automatically the intention of the user to open or close the mosquito net. We used optical flow to extract pressure map features, and they were fed to a 3-dimensional convolutional neural network (3D-CNN) classification model subsequently. We achieved the expected results using a nested cross-validation method to evaluate our model. Deep learning has better adaptability than the traditional methods and also has better anti-interference by the different bodies of users. This research has the potential to be used in intelligent medical protection and large-scale sensor array perception of the environment.     

基于深度强化学习的图像去模糊方法

目的为了有效地去除多种图像模糊,提高图像质量,提出基于深度强化学习的图像去模糊方法。方法选用GoPro与DIV2K这2个数据集进行实验,以峰值信噪比(PSNR)和结构相似性(SSIM)为客观评价指标。通过卷积神经网络获得模糊图像的高维特征,利用深度强化学习结合多种CNN去模糊工具建立去模糊框架,将峰值信噪比(PSNR)作为训练奖励评价函数,来选择最优修复策略,逐步对模糊图像进行修复。结果通过训练与测试,与现有的主流算法相比,文中方法有着更好的主观视觉效果,且PSNR值与SSIM值都有更好的表现。结论实验结果表明,文中方法能有效地解决图像的高斯模糊和运动模糊等问题,并取得了良好的视觉效果,在图像去模糊领域具有一定的参考价值。