A novel enhanced region proposal network and modified loss function: threat object detection in secure screening using deep learning

The Journal of Supercomputing - Detection of threat objects concealed in passenger clothing and baggage poses a challenge to aviation security. At present, the detection technology is capable of...     

A dyadic multi-resolution deep convolutional neural wavelet network for image classification

For almost the past four decades, image classification has gained a lot of attention in the field of pattern recognition due to its application in various fields. Given its importance, several approaches have been proposed up to now. In this paper, we will present a dyadic multi-resolution deep convolutional neural wavelets’ network approach for image classification. This approach consists of performing the classification of one class versus all the other classes of the dataset by the reconstruction of a Deep Convolutional Neural Wavelet Network (DCNWN). This network is based on the Neural Network (NN) architecture, the Fast Wavelet Transform (FWT) and the Adaboost algorithm. It consists, first, of extracting features using the FWT based on the Multi-Resolution Analysis (MRA). These features are used to calculate the inputs of the hidden layer. Second, those inputs are filtered by using the Adaboost algorithm to select the best ones corresponding to each image. Third, we create an AutoEncoder (AE) using wavelet networks of all images. Finally, we apply a pooling for each hidden layer of the wavelet network to obtain a DCNWN that permits the classification of one class and rejects all other classes of the dataset. Classification rates given by our approach show a clear improvement compared to those cited in this article.

     

Wind power prediction using deep neural network based meta regression and transfer learning

An innovative short term wind power prediction system is proposed which exploits the learning ability of deep neural network based ensemble technique and the concept of transfer learning. In the proposed DNN-MRT scheme, deep auto-encoders act as base-regressors, whereas Deep Belief Network is used as a meta-regressor. Employing the concept of ensemble learning facilitates robust and collective decision on test data, whereas deep base and meta-regressors ultimately enhance the performance of the proposed DNN-MRT approach. The concept of transfer learning not only saves time required during training of a base-regressor on each individual wind farm dataset from scratch but also stipulates good weight initialization points for each of the wind farm for training. The effectiveness of the proposed, DNN-MRT technique is expressed by comparing statistical performance measures in terms of root mean squared error (RMSE), mean absolute error (MAE), and standard deviation error (SDE) with other existing techniques.     

Multi spectral image fusion by deep convolutional neural network and new spectral loss function

In this paper we propose a new multispectral image fusion architecture. The proposed method includes two steps related to two neural networks. First the extracted spatial information, from panchromatic (Pan) image, is injected to upsampled multi-spectral (MS) image. In this step, the method employed a deep convolution neural network (DCNN) to estimate the spatial information of the MS image, according to multi-resolution analysis (MRA) scheme. This DCNN is trained by the low-spatial resolution version of Pan as an input, and by the spatial information as the target. This trained DCNN is called ‘Fusion network (FN)’. The FN, adaptively, estimates the spatial information of the MS images, and operates as an injection gain in the MRA scheme. In the second step, the spectral compensation is performed on the fused MS image. For this purpose, we used a novel loss function for this DCNN, to reduce the spectral distortion in the fused images, and simultaneously maintain the spatial information. This network is called ‘Spectral compensation network (SCN)’. Finally, the proposed method is compared to the several state-of-the-art methods on three datasets, using both full-reference and reduced reference criterion. The experimental results show that the proposed method can achieve competitive performance in both spatial and spectral information.     

Attention-based convolutional neural network deep learning approach for robust malware classification

Recently, transforming windows files into images and its analysis using machine learning and deep learning have been considered as a state-of-the art works for malware detection and classification. This is mainly due to the fact that image-based malware detection and classification is platform independent, and the recent surge of success of deep learning model performance in image classification. Literature survey shows that convolutional neural network (CNN) deep learning methods are successfully employed for image-based windows malware classification. However, the malwares were embedded in a tiny portion in the overall image representation. Identifying and locating these affected tiny portions is important to achieve a good malware classification accuracy. In this work, a multi-headed attention based approach is integrated to a CNN to locate and identify the tiny infected regions in the overall image. A detailed investigation and analysis of the proposed method was done on a malware image dataset. The performance of the proposed multi-headed attention-based CNN approach was compared with various non-attention-CNN-based approaches on various data splits of training and testing malware image benchmark dataset. In all the data-splits, the attention-based CNN method outperformed non-attention-based CNN methods while ensuring computational efficiency. Most importantly, most of the methods show consistent performance on all the data splits of training and testing and that illuminates multi-headed attention with CNN model's generalizability to perform on the diverse datasets. With less number of trainable parameters, the proposed method has achieved an accuracy of 99% to classify the 25 malware families and performed better than the existing non-attention based methods. The proposed method can be applied on any operating system and it has the capability to detect packed malware, metamorphic malware, obfuscated malware, malware family variants, and polymorphic malware. In addition, the proposed method is malware file agnostic and avoids usual methods such as disassembly, de-compiling, de-obfuscation, or execution of the malware binary in a virtual environment in detecting malware and classifying malware into their malware family.     

Shallow and deep learning for image classification

The paper is focused on the idea to demonstrate the advantages of deep learning approaches over ordinary shallow neural network on their comparative applications to image classifying from such popular benchmark databases as FERET and MNIST. An autoassociative neural network is used as a standalone program realized the nonlinear principal component analysis for prior extracting the most informative features of input data for neural networks to be compared further as classifiers. A special study of the optimal choice of activation function and the normalization transformation of input data allows to improve efficiency of the autoassociative program. One more study devoted to denoising properties of this program demonstrates its high efficiency even on noisy data. Three types of neural networks are compared: feed-forward neural net with one hidden layer, deep network with several hidden layers and deep belief network with several pretraining layers realized restricted Boltzmann machine. The number of hidden layer and the number of hidden neurons in them were chosen by cross-validation procedure to keep balance between number of layers and hidden neurons and classification efficiency. Results of our comparative study demonstrate the undoubted advantage of deep networks, as well as denoising power of autoencoders. In our work we use both multiprocessor graphic card and cloud services to speed up our calculations. The paper is oriented to specialists in concrete fields of scientific or experimental applications, who have already some knowledge about artificial neural networks, probability theory and numerical methods.     

糖尿病性视网膜图像的深度神经网络分类方法

针对传统的视网膜图像处理步骤复杂、泛化性差、缺少完整的自动识别系统等问题,提出了一套完整的基于深度神经网络的视网膜图像自动识别系统。首先,对图像进行去噪、归一化、数据扩增等预处理;然后,设计了紧凑的神经网络模型——CompactNet,CompactNet继承了AlexNet的浅层结构参数,深层网络参数则根据训练数据进行自适应调整;最后,针对不同的训练方法和不同的网络结构进行了性能测试。实验结果表明,CompactNet网络的微调方法要优于传统的网络训练方法,其分类指标可以达到0.87,与传统直接训练相比高出0.27;对于LeNet,AlexNet和CompactNet三种网络模型,CompactNet网络模型的分类准确率最高;并且通过实验证实了数据扩增等预处理方法的必要性。     

基于深度卷积神经网络的快速图像分类算法

为了应对大量图像的分类问题,提出一种基于深度卷积神经网络和CUDA-cuDNN并行运算的快速图像分类方法。该方法利用深度卷积神经网络自动学习特征的优势来解决手工设计特征普适性差等问题,同时结合基于CUDA架构的cuDNN并行运算策略来提高训练速度和加快分类速度,并且针对深度卷积神经网络易受参数扰动等缺点,引入批量正则化（Batch Normalization）以提高算法的鲁棒性。实验结果表明,该方法不仅大幅缩短了训练时间同时加快了图像的分类速度,而且进一步降低了图像分类的错误率。     

A real-time hourly ozone prediction system using deep convolutional neural network

Neural Computing and Applications - This study uses a deep learning approach to forecast ozone concentrations over Seoul, South Korea for 2017. We use a deep convolutional neural network (CNN). We...     

RETRACTED ARTICLE: A novel large-scale multimedia image data classification algorithm based on mapping assisted deep neural network

Multimedia Tools and Applications - With the increasing number of the images, how to effectively manage and use these images becomes an urgent problem to be solved. The classification of the images...     

A novel solution of enhanced loss function using deep learning in sleep stage classification: predict and diagnose patients with sleep disorders

Multimedia Tools and Applications - Sleep stage classification is important to accurately predict and diagnose patients with sleep disorders. Though various deep learning approaches have been...     

The skin cancer classification using deep convolutional neural network

Multimedia Tools and Applications - This paper addresses the demand for an intelligent and rapid classification system of skin cancer using contemporary highly-efficient deep convolutional neural...     

Quantum convolutional neural network for image classification

Pattern Analysis and Applications - In this paper we propose two scale-inspired local feature extraction methods based on Quantum Convolutional Neural Network (QCNN) in the Tensorflow quantum...     

A neural learning approach for adaptive image restoration using afuzzy model-based network architecture

We address the problem of adaptive regularization in image restoration by adopting a neural-network learning approach. Instead of explicitly specifying the local regularization parameter values, they are regarded as network weights which are then modified through the supply of appropriate training examples. The desired response of the network is in the form of a gray level value estimate of the current pixel using weighted order statistic (WOS) filter. However, instead of replacing the previous value with this estimate, this is used to modify the network weights, or equivalently, the regularization parameters such that the restored gray level value produced by the network is closer to this desired response. In this way, the single WOS estimation scheme can allow appropriate parameter values to emerge under different noise conditions, rather than requiring their explicit selection in each occasion. In addition, we also consider the separate regularization of edges and textures due to their different noise masking capabilities. This in turn requires discriminating between these two feature types. Due to the inability of conventional local variance measures to distinguish these two high variance features, we propose the new edge-texture characterization (ETC) measure which performs this discrimination based on a scalar value only. This is then incorporated into a fuzzified form of the previous neural network which determines the degree of membership of each high variance pixel in two fuzzy sets, the EDGE and TEXTURE fuzzy sets, from the local ETC value, and then evaluates the appropriate regularization parameter by appropriately combining these two membership function values.     

IIRNet: A lightweight deep neural network using intensely inverted residuals for image recognition

Deep neural networks have achieved great success in many tasks of pattern recognition. However, large model size and high cost in computation limit their applications in resource-limited systems. In this paper, our focus is to design a lightweight and efficient convolutional neural network architecture by directly training the compact network for image recognition. To achieve a good balance among classification accuracy, model size, and computation complexity, we propose a lightweight convolutional neural network architecture named IIRNet for resource-limited systems. The new architecture is built based on Intensely Inverted Residual block (IIR block) to decrease the redundancy of the convolutional blocks. By utilizing two new operations, intensely inverted residual and multi-scale low-redundancy convolutions, IIR block greatly reduces its model size and computational costs while matches the classification accuracy of the state-of-the-art networks. Experiments on CIFAR-10, CIFAR-100, and ImageNet datasets demonstrate the superior performance of IIRNet on the trade-offs among classification accuracy, computation complexity, and model size, compared to the mainstream compact network architectures.     

A central-point-enhanced convolutional neural network for high-resolution remote-sensing image classification

As one of the most important algorithms in the field of deep learning technology, the convolutional neural network (CNN) has been successfully applied in many fields. CNNs can recognize objects in an image by considering morphology and structure rather than simply individual pixels. One advantage of CNNs is that they exhibit translational invariance; when an image contains a certain degree of distortion or shift, a CNN can still recognize the object in the image. However, this advantage becomes a disadvantage when CNNs are applied to pixel-based classification of remote-sensing images, because their translational invariance characteristics causes distortions in land-cover boundaries and outlines in the classification result image. This problem severely limits the application of CNNs in remote-sensing classification. To solve this problem, we propose a central-point-enhanced convolutional neural network (CE-CNN) to classify high-resolution remote-sensing images. By introducing the central-point-enhanced layer when classifying a sample, the CE-CNN increases the weight of the central point in feather maps while preserving the original textures and characteristics. In our experiment, we selected four representative positions on a high-resolution remote-sensing image to test the classification ability of the proposed method and compared the CE-CNN with the traditional multi-layer perceptron (MLP) and a traditional CNN. The results show that the proposed method can not only achieves a higher classification accuracy but also less distortion and fewer incorrect results at the boundaries of land covers. We further compared the CE-CNN with six state-of-the-art methods: k-NN, maximum likelihood, classification and regression tree (CART), MLP, support vector machine, and CNN. The results show that the CE-CNN’s classification accuracy is better than the other methods.     

An evaluation of deep neural network models for music classification using spectrograms

Multimedia Tools and Applications - Deep Neural Network (DNN) models have lately received considerable attention for that the network structure can extract deep features to improve classification...     

基于稀疏自编码深度神经网络的林火图像分类

针对林火与相似目标很难区分的问题,提出一种基于稀疏自编码深度神经网络的林火图像分类新方法。采用无监督的特征学习算法稀疏自编码从无标签图像小块中学习特征参数,完成深度神经网络的训练;利用学习到的特征从原始大小分类图像中提取特征并卷积和均值池化特征;对卷积和池化后的特征采用softmax回归来训练最终softmax分类器。实验结果表明,跟传统的BP神经网络相比,新方法能够更有效区分林火与红旗、红叶等类似物体。     

全卷积网络结合改进的条件随机场循环神经网络用于SAR图像场景分类

传统合成孔径雷达（SAR）图像基于粗分割像素块提取相关特征,后接支持向量机（SVM）和马尔可夫随机场（MRF）或条件随机场（CRF）进行分类,该方法存在同一像素块内部不同类别像素的误差,而且只考虑邻近区域未充分用到全局信息和结构信息。故考虑基于像素点引入全卷积网络（FCN）,以ESAR卫星图像为样本,基于像素点级别构建卷积网络进行训练,得到各像素的初始类别分类概率。为了考虑全局像素类别的影响后接CRF-循环神经网络（CRF-RNN）,利用FCN得到的初始概率,结合CRF结构得到全局像素类别转移结果,之后进行RNN的迭代进一步优化实验结果。由于基于像素点和考虑了全局信息与结构信息,克服了传统分类的部分缺点,使正确率较传统SVM或CRF方法平均提高了约6.5个百分点。由于CRF-RNN的距离权重是用高斯核人为拟合的,不能随实际训练样本来改变和确定,故存在一定误差,针对该问题提出可训练的全图距离权重卷积网络来改进CRF-RNN,最终实验结果表明改进后方法的正确率较未改进的CRF-RNN又提高了1.04个百分点。