Convolutional neural network based early fire detection

The detection of manmade disasters particularly fire is valuable because it causes many damages in terms of human lives. Research on fire detection using wireless sensor network and video-based methods is a very hot research topic. However, the WSN based detection model need fire happens and a lot of smoke and fire for detection. Similarly, video-based models also have some drawbacks because conventional algorithms need feature vectors and high rule-based models for detection. In this paper, we proposed a fire detection method which is based on powerful machine learning and deep learning algorithms. We used both sensors data as well as images data for fire prevention. Our proposed model has three main deep neural networks i.e. a hybrid model which consists of Adaboost and many MLP neural networks, Adaboost-LBP model and finally convolutional neural network. We used Adaboost-MLP model to predict the fire. After the prediction, we proposed two neural networks i.e. Adaboost-LBP model and convolutional neural network for detection of fire using the videos and images taken from the cameras installed for the surveillance. Adaboost-LBP model is to generate the ROIs from the image where emergencies exist Our proposed model results are quite good, and the accuracy is almost 99%. The false alarming rate is very low and can be reduced more using further training.

     

A bus passenger re-identification dataset and a deep learning baseline using triplet embedding

Bus passenger re-identification is a special case of person re-identification, which aims to establish identity correspondence between the front door camera and the back door camera. In bus environment,it is hard to capture the full body of the passengers. So this paper proposes a bus passenger re-identification dataset,which contains 97,136 head images of 1,720 passengers obtained from hundreds of thousands of video frames with different lighting and perspectives. We also provide a evaluation applied to the dataset based on deep learning and triplet loss. After data augmentation,using ResNet with trihard loss as benchmark network and pre-training on pedestrian re-identification dataset Market-1501, we achieve mAP accuracy of 55.79% and Rank-1 accuracy of 67.91% on passenger re-identification dataset.

     

Using a deep learning network to recognise low back pain in static standing

Abstract

Low back pain (LBP) remains one of the most prevalent musculoskeletal disorders, while algorithms that able to recognise LBP patients from healthy population using balance performance data are rarely seen. In this study, human balance and body sway performance during standing trials were utilised to recognise chronic LBP populations using deep neural networks. To be specific, 44 chronic LBP and healthy individuals performed static standing tasks, while their spine kinematics and centre of pressure were recorded. A deep learning network with long short-term memory units was used for training, prediction and implementation. The performance of the model was evaluated by: (a) overall accuracy, (b) precision, (c) recall, (d) F1 measure, (e) receiver-operating characteristic and (f) area under the curve. Results indicated that deep neural networks could recognise LBP populations with precision up to 97.2% and recall up to 97.2%. Meanwhile, the results showed that the model with the C7 sensor output performed the best.

Practitioner summary: Low back pain (LBP) remains the most common musculoskeletal disorder. In this study, we investigated the feasibility of applying artificial intelligent deep neural network in detecting LBP population from healthy controls with their kinematics data. Results showed a deep learning network can solve the above classification problem with both promising precision and recall performance.     

Deep learning networks for stock market analysis and prediction: Methodology,data representations,and case studies

We offer a systematic analysis of the use of deep learning networks for stock market analysis and prediction. Its ability to extract features from a large set of raw data without relying on prior knowledge of predictors makes deep learning potentially attractive for stock market prediction at high frequencies. Deep learning algorithms vary considerably in the choice of network structure, activation function, and other model parameters, and their performance is known to depend heavily on the method of data representation. Our study attempts to provides a comprehensive and objective assessment of both the advantages and drawbacks of deep learning algorithms for stock market analysis and prediction. Using high-frequency intraday stock returns as input data, we examine the effects of three unsupervised feature extraction methods—principal component analysis, autoencoder, and the restricted Boltzmann machine—on the network’s overall ability to predict future market behavior. Empirical results suggest that deep neural networks can extract additional information from the residuals of the autoregressive model and improve prediction performance; the same cannot be said when the autoregressive model is applied to the residuals of the network. Covariance estimation is also noticeably improved when the predictive network is applied to covariance-based market structure analysis. Our study offers practical insights and potentially useful directions for further investigation into how deep learning networks can be effectively used for stock market analysis and prediction.     

An efficient deep learning technique for facial emotion recognition

Emotion recognition from facial images is considered as a challenging task due to the varying nature of facial expressions. The prior studies on emotion classification from facial images using deep learning models have focused on emotion recognition from facial images but face the issue of performance degradation due to poor selection of layers in the convolutional neural network model.To address this issue, we propose an efficient deep learning technique using a convolutional neural network model for classifying emotions from facial images and detecting age and gender from the facial expressions efficiently. Experimental results show that the proposed model outperformed baseline works by achieving an accuracy of 95.65% for emotion recognition, 98.5% for age recognition, and 99.14% for gender recognition.

     

Medical image semantic segmentation based on deep learning

The image semantic segmentation has been extensively studying. The modern methods rely on the deep convolutional neural networks, which can be trained to address this problem. A few years ago networks require the huge dataset to be trained. However, the recent advances in deep learning allow training networks on the small datasets, which is a critical issue for medical images, since the hospitals and research organizations usually do not provide the huge amount of data. In this paper, we address medical image semantic segmentation problem by applying the modern CNN model. Moreover, the recent achievements in deep learning allow processing the whole image per time by applying concepts of the fully convolutional neural network. Our qualitative and quantitate experiment results demonstrated that modern CNN can successfully tackle the medical image semantic segmentation problem.

     

11K Hands: Gender recognition and biometric identification using a large dataset of hand images

Human hand not only possesses distinctive feature for gender information, it is also considered one of the primary biometric traits used to identify a person. Unlike face images, which are usually unconstrained, an advantage of hand images is they are usually captured under a controlled position. Most state-of-the-art methods, that rely on hand images for gender recognition or biometric identification, employ handcrafted features to train an off-the-shelf classifier or be used by a similarity metric for biometric identification. In this work, we propose a deep learning-based method to tackle the gender recognition and biometric identification problems. Specifically, we design a two-stream convolutional neural network (CNN) which accepts hand images as input and predicts gender information from these hand images. This trained model is then used as a feature extractor to feed a set of support vector machine classifiers for biometric identification. As part of this effort, we propose a large dataset of human hand images, 11K Hands, which contains dorsal and palmar sides of human hand images with detailed ground-truth information for different problems including gender recognition and biometric identification. By leveraging thousands of hand images, we could effectively train our CNN-based model achieving promising results. One of our findings is that the dorsal side of human hands is found to have effective distinctive features similar to, if not better than, those available in the palmar side of human hand images. To facilitate access to our 11K Hands dataset, the dataset, the trained CNN models, and our Matlab source code are available at (https://goo.gl/rQJndd).

     

Research on Transfer Learning Methods for Classification of Typhoon Cloud Image

Aiming at the complexity of traditional methods for feature extraction about satellite cloud images, and the difficulty of developing deep convolutional neural network from scratch, a parameter-based transfer learning method for classifying typhoon intensity is proposed. Take typhoon satellite cloud images published by Japan Meteorological Agency, which includes 10 000 scenes among nearly 40 years to construct training and test typhoon datasets. Three deep convolutional neural networks, VGG16, InceptionV3 and ResNet50 are trained as source models on the large-scale ImageNet datasets. Considering the discrepancy between low-level features and high-level semantic features of typhoon cloud images, adapt the optimal number of transferable layers in neural networks and freeze weights of low-level network. Meanwhile, fine-tune surplus weights on typhoon dataset adaptively. Finally, a transferred prediction model which is suitable for small sample typhoon datasets, called T-typCNNs is proposed. Experimental results show that the T-typCNNs can achieve training accuracy of 95.081% and testing accuracy of 91.134%, 18.571% higher than using shallow convolutional neural network, 9.819% higher than training with source models from scratch.     

Deep Metallogenic prediction model construction of the Xiongcun no. II orebody based on the DNN algorithm

With the continuous mining and gradual reduction of shallow deposits, deep prospecting has become a new global prospecting trend. In addition, with the development of artificial intelligence, deep learning provides a favorable means for geological big data analysis. This paper, researches the No. II Orebody of the Xiongcun deposit. First, based on previous research results and metallogenic regularity, prospecting information, namely, lithology, Au-Ag-Cu chemical elements and wall rock alteration is extracted, and the block model is established by combining the Kriging interpolation structure. Second, the datasets are divided into dataset I and dataset II according to “randomness” and “depth”. Third, deep prospecting prediction models based on deep neural networks (DNN) and the convolutional neural networks (CNN) is constructed, and the model parameters are optimized. Finally, the models are applied to the deep prediction of the Xiongcun No. II Orebody. The results show that the accuracy rate and recall rate of the prediction model based on the DNN algorithm are 96.15% and 89.23%, respectively, and the AUC is 96.39%, which are higher values than those of the CNN algorithm, indicating that the performance of the prediction model based on the DNN algorithm is better. The accuracy of prediction model based on dataset I is higher than that of dataset II. The accuracy of deep metallogenic prediction based on the DNN algorithm is approximately 89%, that based on the CNN is approximately 87%, and that based on prospecting information method is approximately 61.27%. The prediction results of the DNN algorithm are relatively consistent in the spatial location and scale of the orebody. Therefore, based on the work done in this paper, it is feasible to use a deep learning method to carry out deep mineral prediction.

     

基于迁移学习及气象卫星云图的台风等级分类研究

针对传统卫星云图特征提取方法复杂且深度卷积神经网络（Deep Convolutional Neural Network, DCNN）模型开发困难的问题,提出一种基于参数迁移的台风等级分类方法。利用日本气象厅发布的近40 a 10 000多景台风云图数据,构建了适应于迁移学习的台风云图训练集和测试集。在大规模ImageNet源数据集上训练出3种源模型VGG16,InceptionV3和ResNet50,依据台风云图低层特征与高层语义特征的差异,适配网络最佳迁移层数并冻结低层权重,高层权重采用自适应微调策略,构建出了适用于台风小样本数据集的迁移预报模型T-typCNNs。实验结果表明:T-typCNNs模型在自建台风数据集上的训练精度为95.081%,验证精度可达91.134%,比利用浅层卷积神经网络训练出的精度高18.571%,相比于直接用源模型训练最多提高9.819%。     

Adversarial image detection in deep neural networks

Deep neural networks are more and more pervading many computer vision applications and in particular image classification. Notwithstanding that, recent works have demonstrated that it is quite easy to create adversarial examples, i.e., images malevolently modified to cause deep neural networks to fail. Such images contain changes unnoticeable to the human eye but sufficient to mislead the network. This represents a serious threat for machine learning methods. In this paper, we investigate the robustness of the representations learned by the fooled neural network, analyzing the activations of its hidden layers. Specifically, we tested scoring approaches used for kNN classification, in order to distinguish between correctly classified authentic images and adversarial examples. These scores are obtained searching only between the very same images used for training the network. The results show that hidden layers activations can be used to reveal incorrect classifications caused by adversarial attacks.

     

Wavelet-packets for deepfake image analysis and detection

As neural networks become able to generate realistic artificial images, they have the potential to improve movies, music, video games and make the internet an even more creative and inspiring place. Yet, the latest technology potentially enables new digital ways to lie. In response, the need for a diverse and reliable method toolbox arises to identify artificial images and other content. Previous work primarily relies on pixel-space convolutional neural networks or the Fourier transform. To the best of our knowledge, synthesized fake image analysis and detection methods based on a multi-scale wavelet-packet representation, localized in both space and frequency, have been absent thus far. The wavelet transform conserves spatial information to a degree, allowing us to present a new analysis. Comparing the wavelet coefficients of real and fake images allows interpretation. Significant differences are identified. Additionally, this paper proposes to learn a model for the detection of synthetic images based on the wavelet-packet representation of natural and generated images. Our forensic classifiers exhibit competitive or improved performance at small network sizes, as we demonstrate on the Flickr Faces High Quality, Large-scale Celeb Faces Attributes and Large-scale Scene UNderstanding source identification problems. Furthermore, we study the binary Face Forensics++ (ff++) fake-detection problem.

     

DeepFusion: predicting movie popularity via cross-platform feature fusion

For online video service providers, the accurate prediction of video popularity directly impacts their advertisement revenue, bandwidth provisioning policy and copyright procurement decision. Most of previous approaches only utilize data from a single platform (e.g., view history) for prediction. However, such approaches cannot provide satisfactory prediction accuracy, as video popularity may be affected by many influential features dispersed over multiple platforms. In this paper, we focus on the popularity prediction of online movies and propose a prediction framework called DeepFusion to fuse salient features from multiple platforms so as to boost the accuracy of popularity prediction of online movies. For this purpose, we extract influential factors from Douban, which is a leading movie rating website in China, and Youku, which is one of the largest online video service providers in China. Considering the complexity incurred by numerous parameters, we choose to feed these influential factors into deep neural networks for prediction and thus avoid the limitation of traditional predictive models. Compared with previous approaches, our solution can significantly improve the prediction accuracy over 40%. Moreover, even for movies without any historical views, our approach can also well capture their popular trends and overcome the cold-start problem.

     

Deep feature extraction and classification of breast ultrasound images

Controlled despeckling (structure/edges/feature preservation with smoothing the homogeneous areas) is a desired pre-processing step for the design of computer-aided diagnostic (CAD) systems using ultrasound images as the presence of speckle noise masks diagnostically important information making interpretation difficult even for experienced radiologist. For efficiently classifying the breast tumors, the conventional CAD system designs use hand-crafted features. However, these features are not robust to the variations in size, shape and orientation of the tumors resulting in lower sensitivity. Thus deep feature extraction and classification of breast ultrasound images have recently gained attention from research community. The deep networks come with an advantage of directly learning the representative features from the images. However, these networks are difficult to train from scratch if the representative training data is small in size. Therefore transfer learning approach for deep feature extraction and classification of medical images has been widely used. In the present work the performance of four pre-trained convolutional neural networks VGG-19, SqueezeNet, ResNet-18 and GoogLeNet has been evaluated for differentiating between benign and malignant tumor types. From the results of the experiments, it is noted that CAD system design using GoogLeNet architecture for deep feature extraction followed by correlation based feature selection and fuzzy feature selection using ANFC-LH yields highest accuracy of 98.0% with individual class accuracy value of 100% and 96% for benign and malignant classes respectively. For differentiating between the breast tumors, the proposed CAD system design can be utilized in routine clinical environment.

     

Depth-guided asymmetric CycleGAN for rain synthesis and image deraining

Based on supervised learning, most of the existing single image deraining networks are trained on paired images including one clean image and one rain image. Since it is difficult to obtain a sufficient number of paired images, most of the rain images are manually synthesized from the clean ones. However, it costs huge time and effort, and requires professional experience to mimic the real rain images well. Moreover, the superior performance of these deraining networks trained on manually synthetic rain images is hard to be maintained when tested on real rain images. In this work, to obtain more realistic rain images for training supervised deraining networks, the depth-guided asymmetric CycleGAN (DA-CycleGAN) is proposed to translate clean images to their rainy counterparts automatically. Due to the cycle consistency strategy, DA-CycleGAN can also implement the single image deraining task unsupervised while synthesizing rain on clean images. Since rain streaks and rain mist vary with depth from the camera, DA-CycleGAN adopts depth information as an aid for rain synthesis and deraining. Furthermore, we design generators with different architectures for these two processes due to the information asymmetry in rain synthesis and deraining. Extensive experiments indicate that the DA-CycleGAN can synthesize more lifelike rain images and provide commensurate deraining performance compared with the state-of-the-art deraining methods.

     

A content based image information retrieval and video thumbnail extraction framework using SOM

Searching an image or a video in a huge volume of graphical data is a tedious time-consuming process. If this search is performed using the conventional element matching technique, the complexity of the search will render the system useless. To overcome this problem, the current paper proposes a Content-Based Image Retrieval (CBIR) and a Content-Based Video Retrieval (CBVR) technique using clustering algorithms based on neural networks. Neural networks have proved to be quite powerful for dimensionality reduction due to their parallel computations. Retrieval of images in a large database on the basis of the content of the query image has been proved fast and efficient through practical results. Two images of the same object, but taken from different camera angles or have rotational and scaling transforms is also matched effectively. In medical domain, CBIR has proved to be a boon to the doctors. The tumor, cancer etc can be easily deducted comparing the images with normal to the images with diseases. Java and Weka have been used for implementation. The thumbnails extracted from the video facilitates the video search in a large videos database. The unsupervised nature of Self Organizing Maps (SOM) has made the software all the more robust.

     

自学习规则下的多聚焦图像融合

目的 基于深度学习的多聚焦图像融合方法主要是利用卷积神经网络（convolutional neural network,CNN）将像素分类为聚焦与散焦。监督学习过程常使用人造数据集,标签数据的精确度直接影响了分类精确度,从而影响后续手工设计融合规则的准确度与全聚焦图像的融合效果。为了使融合网络可以自适应地调整融合规则,提出了一种基于自学习融合规则的多聚焦图像融合算法。方法 采用自编码网络架构,提取特征,同时学习融合规则和重构规则,以实现无监督的端到端融合网络;将多聚焦图像的初始决策图作为先验输入,学习图像丰富的细节信息;在损失函数中加入局部策略,包含结构相似度（structural similarity index measure,SSIM）和均方误差（mean squared error,MSE）,以确保更加准确地还原图像。结果 在Lytro等公开数据集上从主观和客观角度对本文模型进行评价,以验证融合算法设计的合理性。从主观评价来看,模型不仅可以较好地融合聚焦区域,有效避免融合图像中出现伪影,而且能够保留足够的细节信息,视觉效果自然清晰;从客观评价来看,通过将模型融合的图像与其他主流多聚焦图像融合算法的融合图像进行量化比较,在熵、Q_w、相关系数和视觉信息保真度上的平均精度均为最优,分别为7.457 4,0.917 7,0.978 8和0.890 8。结论 提出了一种用于多聚焦图像的融合算法,不仅能够对融合规则进行自学习、调整,并且融合图像效果可与现有方法媲美,有助于进一步理解基于深度学习的多聚焦图像融合机制。     

T-GAN: A deep learning framework for prediction of temporal complex networks with adaptive graph convolution and attention mechanism

Complex network is graph network with non-trivial topological features often occurring in real systems, such as video monitoring networks, social networks and sensor networks. While there is growing research study on complex networks, the main focus has been on the analysis and modeling of large networks with static topology. Predicting and control of temporal complex networks with evolving patterns are urgently needed but have been rarely studied. In view of the research gaps we are motivated to propose a novel end-to-end deep learning based network model, which is called temporal graph convolution and attention (T-GAN) for prediction of temporal complex networks. To joint extract both spatial and temporal features of complex networks, we design new adaptive graph convolution and integrate it with Long Short-Term Memory (LSTM) cells. An encoder-decoder framework is applied to achieve the objectives of predicting properties and trends of complex networks. And we proposed a dual attention block to improve the sensitivity of the model to different time slices. Our proposed T-GAN architecture is general and scalable, which can be used for a wide range of real applications. We demonstrate the applications of T-GAN to three prediction tasks for evolving complex networks, namely, node classification, feature forecasting and topology prediction over 6 open datasets. Our T-GAN based approach significantly outperforms the existing models, achieving improvement of more than 4.7% in recall and 25.1% in precision. Additional experiments are also conducted to show the generalization of the proposed model on learning the characteristic of time-series images. Extensive experiments demonstrate the effectiveness of T-GAN in learning spatial and temporal feature and predicting properties for complex networks.     

Effects of the deep learning-based super-resolution method on thermal image classification applications

Thermal imaging can be used in many sectors such as public security, health, and defense in image processing. However, thermal imaging systems are very costly, limiting their use, especially in the medical field. Also, thermal camera systems obtain blurry images with low levels of detail. Therefore, the need to improve their resolution has arisen. Here, super-resolution techniques can be a solution. Developments in deep learning in recent years have increased the success of super-resolution (SR) applications. This study proposes a new deep learning-based approach TSRGAN model for SR applications performed on a new dataset consisting of thermal images of premature babies. This dataset was created by downscaling the thermal images (ground truth) of premature babies as traditional SR studies. Thus, a dataset consisting of high-resolution (HR) and low-resolution (LR) thermal images were obtained. SR images created due to the applications were compared with LR, bicubic interpolation images, and obtained SR images using state-of-the-art models. The success of the results was evaluated using image quality metrics of peak signal to noise ratio (PSNR) and structural similarity index measure (SSIM). The results show that the proposed model achieved the second-best PSNR value and the best SSIM value. Additionally, a CNN-based classifier model was developed to perform task-based evaluation, and classification applications were carried out separately on LR, HR, and reconstructed SR image sets. Here, the success of classifying unhealthy and healthy babies was compared. This study showed that the classification accuracy of SR images increased by approximately 5% compared to the classification accuracy of LR images. In addition, the classification accuracy of SR thermal images approached the classification accuracy of HR thermal images by about 2%. Therefore, with the approach proposed in this study, it has been proven that LR thermal images can be used in classification applications by increasing their resolution. Thus, widespread use of thermal imaging systems with lower costs in the medical field will be achieved.

     

Performance improvement of Deep Learning Models using image augmentation techniques

The major barrier while using deep learning models is lack of large number of images in the training dataset. In fact, there is a need of thousands of images in each image categories based on the complexity of problem. Prior studies have shown that picture augmentation techniques can be used to enhance the number of images in a training dataset artificially. These techniques can aid in improving the overall learning process and performance of a deep learning model. Hence, to address this problem we have proposed three algorithms. Firstly, two image acquisition algorithms have been proposed to systematically obtain real field images for testing and images from public datasets for training a model. Secondly, an algorithm is proposed to describe the procedure how the augmentations can be applied to enhance the datasets. During this study, we have investigated 52 augmentations that can allow enhancing the size of input dataset by improving the quantity of images. To perform the classification process of four maize crop diseases, a new convolutional neural network model is developed and several experiments have been performed to prove its effectiveness. Firstly, two tests were carried out using the original dataset from Kaggle public repository and the augmented dataset. When compared with the original dataset, the model improved by 5.14% with the augmented dataset. Secondly, three experiments carried out to evaluate the performance of proposed augmentation method. Experimental results demonstrated that the proposed approach outperforms the existing three approaches by 27.38%, 3.14%, and 1.34% during the classification process. The proposed IPA augmentation method has been compared with six existing methods: Full Stage Data Augmentation Framework, LeafGAN, Novel Augmentation method based on GAN, Wasserstein Generative Adversarial Network (WGAN), Activation Reconstruction-GAN, and Step-by-Step Data Augmentation Method and experimental results show that performance is better than existing methods by 28.31%, 19.76%, 20.18%, 13.75%, 2.42%, and 12.68% respectively.