Encoder-decoder based convolutional neural networks for image forgery detection

Multimedia Tools and Applications - Today, images editing software has greatly evolved, thanks to them that the semantic manipulation of images has become easier. On the other hand, the...     

图像适应算法中非冗余显著图的计算

近些年为了将大尺寸数字图像的内容自适应地显示在手机、PDA（掌上电脑）等屏幕较小的显示终端上以得到更好的显示效果,图像适应得到了很大的关注。现有的大量方法使用了不同的重要度映射来得到最优的图像缩放结果。图像适应算法就是为了去除图像中的不重要信息,为此提出在图像适应方法中使用非冗余显著图的计算来分析不重要信息的位置。首先,引入了图像分解,提出图像高梯度区域的冗余信息。然后,提出纹理梯度的概念,并进一步针对细节信息建立了纹理冗余度分析机制。纹理梯度非常适合卡通图像分析,从而将现有图像适应方法推广到非真实图像。大量实验结果显示,本文方法并不受图像适应算子的影响,该方法对具有复杂背景和包含多目标物体等的图像适应问题都比较鲁棒。     

DSFMA: deeply supervised fully convolutional neural networks based on multi-level aggregation for saliency detection

Multimedia Tools and Applications - In recent years, the emergence of fully convolutional neural networks (FCNs) has delivered significant success in the field of saliency detection. Although the...     

Anti-steganalysis for image on convolutional neural networks

Multimedia Tools and Applications - Nowadays, convolutional neural network (CNN) based steganalysis methods achieved great performance. While those methods are also facing security problems. In...     

一种基于级联神经网络的飞机检测方法

由于旋转角度多样性、极端的尺度差异的影响,遥感图像中的飞机检测目前仍存在挑战。为了解决旋转和尺度的问题,目前的策略是将现有的自然场景下的目标检测算法(如Faster R-CNN、SSD等)直接迁移到遥感图像中。这些算法的主干网络复杂,模型占用空间大,难以应用到低功耗和嵌入式设备中。为了在准确率不降低的情况下提高检测速度,本文提出了一个仅包含9层的卷积神经网络来解决飞机检测问题。该网络采用了由粗到细的策略,通过级联两个网络的方式减少计算开销。为了评估方法的有效性,我们建立了一个针对飞机检测的遥感数据集。实验结果表明,该方法超越了VGG16等复杂的主干网络,达到了接近主流检测方法的性能表现,同时显著降低了参数量并使检测速度提高了2倍以上。     

Beyond saliency: Understanding convolutional neural networks from saliency prediction on layer-wise relevance propagation

Despite the tremendous achievements of deep convolutional neural networks (CNNs) in many computer vision tasks, understanding how they actually work remains a significant challenge. In this paper, we propose a novel two-step understanding method, namely Salient Relevance (SR) map, which aims to shed light on how deep CNNs recognize images and learn features from areas, referred to as attention areas, therein. Our proposed method starts out with a layer-wise relevance propagation (LRP) step which estimates a pixel-wise relevance map over the input image. Following, we construct a context-aware saliency map, SR map, from the LRP-generated map which predicts areas close to the foci of attention instead of isolated pixels that LRP reveals. In human visual system, information of regions is more important than of pixels in recognition. Consequently, our proposed approach closely simulates human recognition. Experimental results using the ILSVRC2012 validation dataset in conjunction with two well-established deep CNN models, AlexNet and VGG-16, clearly demonstrate that our proposed approach concisely identifies not only key pixels but also attention areas that contribute to the underlying neural network's comprehension of the given images. As such, our proposed SR map constitutes a convenient visual interface which unveils the visual attention of the network and reveals which type of objects the model has learned to recognize after training. The source code is available at https://github.com/Hey1Li/Salient-Relevance-Propagation.     

No-reference synthetic image quality assessment with convolutional neural network and local image saliency

Depth-image-based rendering (DIBR) is widely used in 3DTV, free-viewpoint video, and interactive 3D graphics applications. Typically, synthetic images generated by DIBR-based systems incorporate various distortions, particularly geometric distortions induced by object dis-occlusion. Ensuring the quality of synthetic images is critical to maintaining adequate system service. However, traditional 2D image quality metrics are ineffective for evaluating synthetic images as they are not sensitive to geometric distortion. In this paper, we propose a novel no-reference image quality assessment method for synthetic images based on convolutional neural networks, introducing local image saliency as prediction weights. Due to the lack of existing training data, we construct a new DIBR synthetic image dataset as part of our contribution. Experiments were conducted on both the public benchmark IRCCyN/IVC DIBR image dataset and our own dataset. Results demonstrate that our proposed metric outperforms traditional 2D image quality metrics and state-of-the-art DIBR-related metrics.

     

Surface defect detection of voltage-dependent resistors using convolutional neural networks

Surface defect detection is an important way to improve the production quality of voltage-dependent resistors (VDRs). To improve the accuracy and efficiency of VDR surface quality detection, an end-to-end surface quality detection method based on deep convolutional neural networks (CNNs) was proposed. The method includes four stages: data preparation, convolution neural network design, CNN training, and testing. First, images of VDRs were acquired from three perspectives, i.e., the front, back, and side, and then training, validation and testing sets were obtained. Second, the proposed CNN models for VDR surface defect detection were constructed. Third, during the training stage, the images with class labels from the established training sets were input to the proposed network for training and validation. Finally, in the testing stage, test images from a total of 408 samples of two VDR models were used to test the trained network. The sensitivity, specificity, accuracy, precision and F measure of the proposed algorithm were compared with those of state-of-the-art methods, and the experimental results showed that the proposed method has a high recognition speed and accuracy and meets the requirements of online real-time detection.

     

Obstructive sleep apnea detection using ecg-sensor with convolutional neural networks

In recent years, AI(Artificial Intelligence) has achieved great development in modern society. More and more modern technologies are applied in surveillance and monitoring. Healthcare monitoring is growing ubiquitous in modern wearable devices, such as a smart watch, electrocardiogram (ECG) necklace, smart band. Many sensors are attached to these smart devices to record and monitor physiological signals caused by activities, and then propagated those recorded electrical data to be further processed to give health diagnosis, disease prevention or making a distress call automatically. Obstructive sleep apnea (OSA) is a sleep disorder with a high occurrence in adult people and observed as an autonomous risk factor for circulatory problems such as ischemic heart attacks and stroke. Numerous traditional neural network based methods have been developed to detect OSA, where these methods however could not provide the intended result because they rely on shallow network. In this paper, we propose an effective OSA detection based on Convolutional neural network. Our method first extracts features from Apnea-Electrocardiogram (ECG) recordings using RR-intervals (time interval from one R-wave to the next R-wave in an ECG signal) and then CNN model having three convolution layers and three fully connected layers is trained with extracted features and applied for OSA detection. The first two convolution layers are followed by batch normalization and pooling layer, and softmax is connected to the last fully connected layer to give final decision. Experimental results on extracted feature of Apnea-ECG signal reveal that our model have better results in terms of performance measure sensitivity, specificity and accuracy. It is expected that the related technology can be applied into smart sensors, especially wearable devices.

     

图像分类的深度卷积神经网络模型综述

图像分类是计算机视觉中的一项重要任务,传统的图像分类方法具有一定的局限性。随着人工智能技术的发展,深度学习技术越来越成熟,利用深度卷积神经网络对图像进行分类成为研究热点,图像分类的深度卷积神经网络结构越来越多样,其性能远远好于传统的图像分类方法。本文立足于图像分类的深度卷积神经网络模型结构,根据模型发展和模型优化的历程,将深度卷积神经网络分为经典深度卷积神经网络模型、注意力机制深度卷积神经网络模型、轻量级深度卷积神经网络模型和神经网络架构搜索模型等4类,并对各类深度卷积神经网络模型结构的构造方法和特点进行了全面综述,对各类分类模型的性能进行了对比与分析。虽然深度卷积神经网络模型的结构设计越来越精妙,模型优化的方法越来越强大,图像分类准确率在不断刷新的同时,模型的参数量也在逐渐降低,训练和推理速度不断加快。然而深度卷积神经网络模型仍有一定的局限性,本文给出了存在的问题和未来可能的研究方向,即深度卷积神经网络模型主要以有监督学习方式进行图像分类,受到数据集质量和规模的限制,无监督式学习和半监督学习方式的深度卷积神经网络模型将是未来的重点研究方向之一;深度卷积神经网络模型的速度和资源消耗仍不尽人意,应用于移动式设备具有一定的挑战性;模型的优化方法以及衡量模型优劣的度量方法有待深入研究;人工设计深度卷积神经网络结构耗时耗力,神经架构搜索方法将是未来深度卷积神经网络模型设计的发展方向。     

Feature learning for steganalysis using convolutional neural networks

Multimedia Tools and Applications - Traditional steganalysis methods usually rely on handcrafted features. However, with the rapid development of advanced steganography, manual design of complex...     

Fast and robust multiple ColorChecker detection using deep convolutional neural networks

ColorCheckers are reference standards that professional photographers and filmmakers use to ensure predictable results under every lighting condition. The objective of this work is to propose a new fast and robust method for automatic ColorChecker detection. The process is divided into two steps: (1) ColorCheckers localization and (2) ColorChecker patches recognition. For the ColorChecker localization, we trained a detection convolutional neural network using synthetic images. The synthetic images are created with the 3D models of the ColorChecker and different background images. The output of the neural networks are the bounding box of each possible ColorChecker candidates in the input image. Each bounding box defines a cropped image which is evaluated by a recognition system, and each image is canonized with regards to color and dimensions. Subsequently, all possible color patches are extracted and grouped with respect to the center's distance. Each group is evaluated as a candidate for a ColorChecker part, and its position in the scene is estimated. Finally, a cost function is applied to evaluate the accuracy of the estimation. The method is tested using real and synthetic images. The proposed method is fast, robust to overlaps and invariant to affine projections. The algorithm also performs well in case of multiple ColorCheckers detection.     

Towards resource-frugal deep convolutional neural networks for hyperspectral image segmentation

Hyperspectral image analysis has been gaining research attention thanks to the current advances in sensor design which have made acquiring such imagery much more affordable. Although there exist various approaches for segmenting hyperspectral images, deep learning has become the mainstream. However, such large-capacity learners are characterized by significant memory footprints. This is a serious obstacle in employing deep neural networks on board a satellite for Earth observation. In this paper, we introduce resource-frugal quantized convolutional neural networks, and greatly reduce their size without adversely affecting the classification capability. Our experiments performed over two hyperspectral benchmarks showed that the quantization process can be seamlessly applied during the training, and it leads to much smaller and still well-generalizing deep models.     

Treetop detection using convolutional neural networks trained through automatically generated pseudo labels

ABSTRACT

Using remote sensing techniques to detect trees at the individual level is crucial for forest management while finding the treetop is an initial and important first step. However, due to the large variations of tree size and shape, traditional unsupervised treetop detectors need to be carefully designed with heuristic knowledge making an efficient and versatile treetop detection still challenging. Currently, the deep convolutional neural networks (CNNs) have shown powerful capabilities to classify and segment images, but the required volume of labelled data for the training impedes their applications. Considering the strengths and limitations of the unsupervised and deep learning methods, we propose a framework using the automatically generated pseudo labels from unsupervised treetop detectors to train the CNNs, which saves the manual labelling efforts. In this study, we use multi-view satellite imagery derived digital surface model (DSM) and multispectral orthophoto as research data and train the fully convolutional networks (FCN) with pseudo labels separately generated from two unsupervised treetop detectors: top-hat by reconstruction (THR) operation and local maxima filter with a fixed window (FFW). The experiments show the FCN detectors trained by pseudo labels, have much better detection accuracies than the unsupervised detectors (6.5% for THR and 11.1% for FFW), especially in the densely forested area (more than 20% of improvement). In addition, our comparative experiments when using manually labelled samples show the proposed treetop detection framework has the potential to significantly reduce the need for training samples while keep a comparable performance.     

SegGraph: 室外场景三维点云闭环检测算法

场景流是连续动态场景之间的3D运动场,广泛应用于机器人技术和自动驾驶任务.现有方法忽略了点云点的相关性,仅关注源点云和目标点云逐点的匹配关系,由于匹配关系完全依赖于点云数据的特征信息,导致在局部特征信息不足的点上准确估计场景流仍然存在挑战.根据源点云相邻点具有相关性的特性,提出NCPUM（neighborhood consistency propagation update method）方法,在邻域内将场景流从高置信度点向低置信度点传播,从而优化局部特征信息不足点的场景流.具体来说,NCPUM包含2个模块：置信度预测模块,根据场景流先验分布图,预测源点云逐点的置信度;场景流传播模块,根据局部区域一致性的约束更新低置信度点集的场景流.NCPUM在合成数据集Flyingthings3D和真实驾驶场景数据集KITTI上进行评估,准确度上达到了国际先进水平.由于邻域一致性更符合真实激光雷达场景的先验假设,因此在KITTI数据集上的提升更加明显.

     

利用双通道卷积神经网络的图像超分辨率算法

目的 图像超分辨率算法在实际应用中有着较为广泛的需求和研究。然而传统基于样本的超分辨率算法均使用简单的图像梯度特征表征低分辨率图像块,这些特征难以有效地区分不同的低分辨率图像块。针对此问题,在传统基于样本超分辨率算法的基础上,提出双通道卷积神经网络学习低分辨率与高分辨率图像块相似度进行图像超分辨率的算法。方法 首先利用深度卷积神经网络学习得到有效的低分辨率与高分辨率图像块之间相似性度量,然后根据输入低分辨率图像块与高分辨率图像块字典基元的相似度重构出对应的高分辨率图像块。结果 本文算法在Set5和Set14数据集上放大3倍情况下分别取得了平均峰值信噪比(PSNR)为32.53 dB与29.17 dB的效果。结论 本文算法从低分辨率与高分辨率图像块相似度学习角度解决图像超分辨率问题,可以更好地保持结果图像中的边缘信息,减弱结果中的振铃现象。本文算法可以很好地适用于自然场景图像的超分辨率增强任务。     

Multifocus image fusion using convolutional neural networks in the discrete wavelet transform domain

Multimedia Tools and Applications - In this paper, a novel multifocus image fusion algorithm based on the convolutional neural network (CNN) in the discrete wavelet transform (DWT) domain is...     

Image spam filtering using convolutional neural networks

Spammers often embed text into images in order to avoid filtering by text-based spam filters, which result in a large number of advertisement spam images. Garbage image recognition has become one of the hotspots in the field of Internet spam filtering research. Its goal is to solve the problem that traditional spam information filtering methods encounter a sharp performance decline or even failure when filtering spam image information. Based on the clustering algorithm, this paper proposes a method to expand the data samples, which greatly improves the number of high-quality training samples and meets the needs of model training. Then, we train a convolutional neural networks using the enlarged data samples to recognize the SPAM in real time. The experimental results show that the accuracy of the model is increased by more than 14% after using the method of data augmentation. The accuracy of the model can be improved by 6% compared with other methods of data augmentation. Combined with convolutional neural networks and the proposed method of data augmentation, the accuracy of our SPAM filtering model is 7–11% higher than that of the traditional method.     

Cloud-based video analytics using convolutional neural networks

Object classification is a vital part of any video analytics system, which could aid in complex applications such as object monitoring and management. Traditional video analytics systems work on shallow networks and are unable to harness the power of distributed processing for training and inference. We propose a cloud-based video analytics system based on an optimally tuned convolutional neural network to classify objects from video streams. The tuning of convolutional neural network is empowered by in-memory distributed computing. The object classification is performed by comparing the target object with the prestored trained patterns, generating a set of matching scores. The matching scores greater than an empirically determined threshold reveal the classification of the target object. The proposed system proved to be robust to classification errors with an accuracy and precision of 97% and 96%, respectively, and can be used as a general-purpose video analytics system.     

基于孪生卷积神经网络的人脸追踪

由于光照、遮挡、尺度变化等原因,在真实多变场景下完成人脸追踪面临挑战。探究了基于卷积神经网络（CNN）的人脸追踪,将基本的卷积神经网络改进为孪生神经网络,在OTB数据集上采用端到端的方式,以成对图像区域作为输入,输出两者距离,通过距离评估图像区域相似性;加入边框回归算法（bounding box regression）微调追踪结果。实验结果表明,改进后的神经网络优于传统的卷积神经网络,能达到更好的人脸追踪效果。