Unsupervised fisheye image correction through bidirectional loss with geometric prior

Neural network based methods for fisheye distortion correction are effective and increasingly popular, although training network require a high amount of labeled data. In this paper, we propose an unsupervised fisheye correction network to address the aforementioned issue. During the training process, the predicted parameters are employed to correct strong distortion that exists in the fisheye image and synthesize the corresponding distortion using the original distortion-free image. Thus, the network is constrained with bidirectional loss to obtain more accurate distortion parameters. We calculate the two losses at the image level as opposed to directly minimizing the difference between the predicted and ground truth of distortion parameters. Additionally, we leverage the geometric prior that the distortion distribution depends on the geometric regions of fisheye images and the straight line should be straight in the corrected images. The network focuses more on the geometric prior regions as opposed to equally perceiving the whole image without any attention mechanisms. To generate more appealing corrected results in visual appearance, we introduce a coarse-to-fine inpainting network to fill the hole regions caused by the irreversible mapping function using distortion parameters. Each module of the proposed network is differentiable, and thus the entire framework is completely end-to-end. When compared with the previous supervised methods, our method is more flexible and shows better practical applications for distortion rectification. The experiment results demonstrate that our proposed method outperforms state-of-the-art methods on the correction performance without any labeled distortion parameters.     

Medical image categorization and retrieval for PACS using the GMM-KL framework.

This paper presents an image representation and matching framework for image categorization in medical image archives. Categorization enables one to determine automatically, based on the image content, the examined body region and imaging modality. It is a basic step in content-based image retrieval (CBIR) systems, the goal of which is to augment text-based search with visual information analysis. CBIR systems are currently being integrated with picture archiving and communication systems for increasing the overall search capabilities and tools available to radiologists. The proposed methodology is comprised of a continuous and probabilistic image representation scheme using Gaussian mixture modeling (GMM) along with information-theoretic image matching via the Kullback-Leibler (KL) measure. The GMM-KL framework is used for matching and categorizing X-ray images by body regions. A multidimensional feature space is used to represent the image input, including intensity, texture, and spatial information. Unsupervised clustering via the GMM is used to extract coherent regions in feature space that are then used in the matching process. A dominant characteristic of the radiological images is their poor contrast and large intensity variations. This presents a challenge to matching among the images, and is handled via an illumination-invariant representation. The GMM-KL framework is evaluated for image categorization and image retrieval on a dataset of 1500 radiological images. A classification rate of 97.5% was achieved. The classification results compare favorably with reported global and local representation schemes. Precision versus recall curves indicate a strong retrieval result as compared with other state-of-the-art retrieval techniques. Finally, category models are learned and results are presented for comparing images to learned category models.     

Perceptually tuned JPEG coder for echocardiac image compression

In this work, we propose an efficient framework for compressing and displaying medical images. Image compression for medical applications, due to available Digital Imaging and Communications in Medicine requirements, is limited to the standard discrete cosine transform-based joint picture expert group. The objective of this work is to develop a set of quantization tables (Q tables) for compression of a specific class of medical image sequences, namely echocardiac. The main issue of concern is to achieve a Q table that matches the specific application and can linearly change the compression rate by adjusting the gain factor. This goal is achieved by considering the region of interest, optimum bit allocation, human visual system constraint, and optimum coding technique. These parameters are jointly optimized to design a Q table that works robustly for a category of medical images. Application of this approach to echocardiac images shows high subjective and quantitative performance. The proposed approach exhibits objectively a 2.16-dB improvement in the peak signal-to-noise ratio and subjectively 25% improvement over the most useable compression techniques.     

基于自注意力特征融合的半监督生成对抗网络用于SAR目标识别

与具有大量标注数据的光学图像相比,合成孔径雷达(Synthetic Aperture Radar,SAR)图像缺乏足够的标记样本,限制了监督学习的SAR目标识别算法的性能.而无监督识别方法又难以满足实际需求,因此本文提出了基于自注意力特征融合的半监督生成对抗网路.首先,在构建生成器和判别器时引入自注意力层,克服卷积算子...     

Web and personal image annotation by mining label correlation with relaxed visual graph embedding

The number of digital images rapidly increases, and it becomes an important challenge to organize these resources effectively. As a way to facilitate image categorization and retrieval, automatic image annotation has received much research attention. Considering that there are a great number of unlabeled images available, it is beneficial to develop an effective mechanism to leverage unlabeled images for large-scale image annotation. Meanwhile, a single image is usually associated with multiple labels, which are inherently correlated to each other. A straightforward method of image annotation is to decompose the problem into multiple independent single-label problems, but this ignores the underlying correlations among different labels. In this paper, we propose a new inductive algorithm for image annotation by integrating label correlation mining and visual similarity mining into a joint framework. We first construct a graph model according to image visual features. A multilabel classifier is then trained by simultaneously uncovering the shared structure common to different labels and the visual graph embedded label prediction matrix for image annotation. We show that the globally optimal solution of the proposed framework can be obtained by performing generalized eigen-decomposition. We apply the proposed framework to both web image annotation and personal album labeling using the NUS-WIDE, MSRA MM 2.0, and Kodak image data sets, and the AUC evaluation metric. Extensive experiments on large-scale image databases collected from the web and personal album show that the proposed algorithm is capable of utilizing both labeled and unlabeled data for image annotation and outperforms other algorithms.     

基于不准确图像数据清洗的分类方法研究

在使用图像数据集训练神经网络分类模型时,需要大量标注准确的图像数据集,但实际应用中的图像数据集经常含有大量标注错误的图像,标注错误的图像不利于训练准确的神经网络分类模型。然而,标注准确的数据集制作需要消耗大量的时间和人力成本。因此,本文提出了一种基于不准确图像数据清洗的分类框架。在猫狗自然图像上的实验结果表明,具有清洗环节的分类模型的分类准确率得到提升,损失函数的损失值下降。在探讨数据集中含有标签错误图像的比例与分类准确率之间的关系中发现,较深层次的神经网络对数据集中错误图像有一定的鲁棒性,但在图像数据集中标签噪音图像的比例较高时,清洗环节的引入使得较浅的神经网络分类模型也能达到与较深层次的神经网络分类模型相当的分类效果,而较浅神经网络分类模型的运算速度更快。本文为构建快速和准确的分类模型提供了一种新思路。      

Graph regularized low-rank feature mapping for multi-label learning with application to image annotation

Automatic image annotation has emerged as a hot research topic in the last two decades due to its application in social images organization. Most studies treat image annotation as a typical multi-label classification problem, where the shortcoming of this approach lies in that in order to a learn reliable model for label prediction, it requires sufficient number of training images with accurate annotations. Being aware of this, we develop a novel graph regularized low-rank feature mapping for image annotation under semi-supervised multi-label learning framework. Specifically, the proposed method concatenate the prediction models for different tags into a matrix, and introduces the matrix trace norm to capture the correlations among different labels and control the model complexity. In addition, by using graph Laplacian regularization as a smooth operator, the proposed approach can explicitly take into account the local geometric structure on both labeled and unlabeled images. Moreover, considering the tags of labeled images tend to be missing or noisy, we introduce a supplementary ideal label matrix to automatically fill in the missing tags as well as correct noisy tags for given training images. Extensive experiments conducted on five different multi-label image datasets demonstrate the effectiveness of the proposed approach.     

A memory learning framework for effective image retrieval.

Most current content-based image retrieval systems are still incapable of providing users with their desired results. The major difficulty lies in the gap between low-level image features and high-level image semantics. To address the problem, this study reports a framework for effective image retrieval by employing a novel idea of memory learning. It forms a knowledge memory model to store the semantic information by simply accumulating user-provided interactions. A learning strategy is then applied to predict the semantic relationships among images according to the memorized knowledge. Image queries are finally performed based on a seamless combination of low-level features and learned semantics. One important advantage of our framework is its ability to efficiently annotate images and also propagate the keyword annotation from the labeled images to unlabeled images. The presented algorithm has been integrated into a practical image retrieval system. Experiments on a collection of 10,000 general-purpose images demonstrate the effectiveness of the proposed framework.     

Visual pattern image coding

A novel framework for digital image compression called visual pattern image coding, or VPIC, is presented. In VPIC, set of visual-patterns is defined independent of the images to be coded. Each visual pattern is a subimage of limited spatial support that is visually meaningful to a normal human observer. The patterns are used as a basis for efficient image representation; since it is assumed that the images to be coded are natural optical images to be viewed by human observers, visual pattern design is developed using relevant psychophysical and physiological data. Although VPIC bears certain resemblances to block truncation (BTC) and vector quantification (VQ) image coding, there are important differences. First, there is no training phase required: the visual patterns derive from models of perceptual mechanisms; second, the assignment of patterns to image regions is not based on a standard (norm) error criterion; expensive search operations are eliminated     

A doubly adaptive approach to dynamic MRI sequence estimation

Dynamic magnetic resonance imaging (MRI) refers to the acquisition of a sequence of MRI images to monitor temporal changes in tissue structure. We present a method for the estimation of dynamic MRI sequences based on two complimentary strategies: an adaptive framework for the estimation of the MRI images themselves, and an adaptive method to tailor the MRI system excitations for each data acquisition. We refer to this method as the doubly adaptive temporal update method (DATUM) for dynamic MRI. Analysis of the adaptive image estimate framework shows that calculating the optimal system excitations for each new image requires complete knowledge of the next image in the sequence. Since this is not realizable, we introduce a linear predictor to aid in determining appropriate excitations. Simulated examples using real MRI data are included to illustrate that the doubly adaptive strategy can provide estimates with lower steady state error than previously proposed methods and also the ability to recover from dramatic changes in the image sequence.     

基于深度卷积神经网络的图上半监督极化SAR图像分类算法

为实现在只有少量标记数据情况下的高质量的图像分类,本文提出了一种基于深度卷积神经网络的图上半监督极化SAR图像分类算法.该算法将极化SAR图像建模为无向图,并基于该无向图,定义了包含半监督项,卷积神经网络项和类标光滑项的能量函数.算法所采用的卷积神经网络提取抽象的数据驱动的极化特征.半监督项约束了有标记像素的类标在分类过程中保持不变.类标光滑项约束了像素间类标的光滑性.基于对PauliRGB图像进行超像素分割而产生的初始化类标图,交替迭代优化所定义的能量函数直至其收敛.在两幅真实极化SAR图像上的实验结果表明,该算法达到了优异的分类效果,其性能优于当前已有算法.     

A training framework for stack and Boolean filtering-fast optimaldesign procedures and robustness case study

A training framework is developed in this paper to design optimal nonlinear filters for various signal and image processing tasks. The targeted families of nonlinear filters are the Boolean filters and stack filters. The main merit of this framework at the implementation level is perhaps the absence of constraining models, making it nearly universal in terms of application areas. We develop fast procedures to design optimal or close to optimal filters, based on some representative training set. Furthermore, the training framework shows explicitly the essential part of the initial specification and how it affects the resulting optimal solution. Symmetry constraints are imposed on the data and, consequently, on the resulting optimal solutions for improved performance and ease of implementation. The case study is dedicated to natural images. The properties of optimal Boolean and stack filters, when the desired signal in the training set is the image of a natural scene, are analyzed. Specifically, the effect of changing the desired signal (using various natural images) and the characteristics of the noise (the probability distribution function, the mean, and the variance) is analyzed. Elaborate experimental conditions were selected to investigate the robustness of the optimal solutions using a sensitivity measure computed on data sets. A remarkably low sensitivity and, consequently, a good generalization power of Boolean and stack filters are revealed. Boolean-based filters are thus shown to be not only suitable for image restoration but also robust, making it possible to build libraries of "optimal" filters, which are suitable for a set of applications.     

Low-resource automatic cartoon image creation from limited samples

In this work, a framework that can automatically create cartoon images with low computation resources and small training datasets is proposed. The proposed system performs region segmentation and learns a region relationship tree from each learning image. The segmented regions are clustered automatically with an enhanced clustering mechanism with no prior knowledge of number of clusters. According to the topology represented by region relationship tree and clustering results, the regions are reassembled to create new images. A swarm intelligence optimization procedure is designed to coordinate the regions to the optimized sizes and positions in the created image. Rigid deformation using moving least squares is performed on the regions to generate more variety for created images. Compared with methods based on Generative Adversarial Networks, the proposed framework can create better images with limited computation resources and a very small amount of training samples.     

A new hardware Trojan detection technique using deep convolutional neural network

The involvement of external vendors in semiconductor industries increases the chance of hardware Trojan (HT) insertion in different phases of the integrated circuit (IC) design. Recently, several partial reverse engineering (RE) based HT detection techniques are reported, which attempt to reduce the time and complexity involved in the full RE process by applying machine learning or image processing techniques in IC images. However, these techniques fail to extract the relevant image features, not robust to image variations, complicated, less generalizable, and possess a low detection rate. Therefore, to overcome the above limitations, this paper proposes a new partial RE based HT detection technique that detects Trojans from IC layout images using Deep Convolutional Neural Network (DCNN). The proposed DCNN model consists of stacking several convolutional and pooling layers. It layer-wise extracts and selects the most relevant and robust features automatically from the IC images and eliminates the need to apply the feature extraction algorithm separately. To prevent the over-training of the DCNN model, a new stopping condition method and two new metrics, namely Accuracy difference measure (ADM) and Loss difference measure (LDM), are proposed that halts the training only when the performance of our model genuinely drops. Further, to combat the issue of process variations and fabrication noise generated during the RE process, we include noisy images with varying parameters in the training process of the model. We also apply the data augmentation and regularization techniques in the model to address the issues of underfitting and overfitting. Experimental evaluation shows that the proposed technique provides 99% and 97.4% accuracy on Trust-Hub and synthetic ISCAS dataset, respectively, which is on-an-average 15.83% and 21.69% higher than the existing partial RE based techniques.     

A multiplicative Nakagami speckle reduction algorithm for ultrasound images

Speckle noise of ultrasound images is of multiplicative nature which degrades the image quality in terms of resolution and contrast. While there exist a number of algorithms for reduction of multiplicative Rayleigh distributed random speckle noise, the low signal-to-noise ratio (SNR) issue of the multiplicative Rayleigh noise is still not adequately resolved. In this paper, a simple 2-dimensional (2D) local intensity smoothing method is presented which transforms the Rayleigh noise contaminated in ultrasound images to Nakagami distributed noise so as to improve the SNR of processed images. A 2D total variation regularized Nakagami speckle reduction algorithm is derived based on the maximum a posteriori estimation framework, which performs well in restoring piecewise-smooth reflectivity and preserving fine details of the image. The proposed algorithm is verified by a series of computer-simulated and real ultrasound image data. It is shown that the algorithm considerably improves the quality of ultrasound images and outperforms the Rayleigh noise based speckle reduction methods in terms of speckle SNR and contrast-to-noise ratio.     

基于数据扩维的SAR目标识别性能提升技术

卷积神经网络的出现使得深度学习在视觉领域取得了巨大的成功,并逐渐延伸到合成孔径雷达(SAR)图像识别领域。然而,SAR图像样本量不足,难以支撑卷积神经网络的训练需求,并且SAR图像包含大量相干斑噪声及不确定性,网络结构的设计较为困难。所以,深度学习在SAR图像识别领域的应用受到阻碍。针对上述问题,文中提出一种基于数据扩维的SAR目标识别性能提升方法,通过对原始SAR 图像进行相关预处理操作并把处理后图像与原始图像结合,从而将一维的原始数据扩充成多维数据来作为训练样本。该扩维方法不仅间接扩充了样本量来支撑网络训练,同时也在网络训练前加入了“主动学习冶影响,所以无需针对SAR图像特性来构建复杂卷积网络,而采用成熟、简单的网络进行训练就可以达到理想的测试精度。最后,使用MSTAR 数据对该方法进行了性能验证,实验结果显示了所提方法的有效性。     

Simulation of tissue atrophy using a topology preserving transformation model

We propose a method to simulate atrophy and other similar volumetric change effects on medical images. Given a desired level of atrophy, we find a dense warping deformation that produces the corresponding levels of volumetric loss on the labeled tissue using an energy minimization strategy. Simulated results on a real brain image indicate that the method generates realistic images of tissue loss. The method does not make assumptions regarding the mechanics of tissue deformation, and provides a framework where a pre-specified pattern of atrophy can readily be simulated. Furthermore, it provides exact correspondences between images prior and posterior to the atrophy that can be used to evaluate provisional image registration and atrophy quantification algorithms.     

面向多源遥感图像的自适应目标检测

近年来,目标检测已经在含有大量标注的数据上展现出了良好的效果,但当真实测试数据与标注数据存在域间差异时,往往会导致训练好的目标检测模型性能降低。由于相比于自然图像,多源遥感图像在成像方式和分辨率等方面存在特有的差异,而传统的方法需要将多源图像数据重新标注,这将消耗大量人力和时间,因此在遥感图像上实现自适应目标检测面临特有的挑战。针对以上问题,本文提出了一种面向多源遥感图像的自适应目标检测算法,在图像级别和语义级别上对网络进行对抗训练。此外,通过结合超分辨网络,进一步缩小了图像级别的差异,实现了自适应目标检测。本文在两个多源遥感数据集上进行实验,结果表明本文方法有效提升了目标域上的检测效果。     

Cooperative sparse representation in two opposite directions for semi-supervised image annotation

Recent studies have shown that sparse representation (SR) can deal well with many computer vision problems, and its kernel version has powerful classification capability. In this paper, we address the application of a cooperative SR in semi-supervised image annotation which can increase the amount of labeled images for further use in training image classifiers. Given a set of labeled (training) images and a set of unlabeled (test) images, the usual SR method, which we call forward SR, is used to represent each unlabeled image with several labeled ones, and then to annotate the unlabeled image according to the annotations of these labeled ones. However, to the best of our knowledge, the SR method in an opposite direction, that we call backward SR to represent each labeled image with several unlabeled images and then to annotate any unlabeled image according to the annotations of the labeled images which the unlabeled image is selected by the backward SR to represent, has not been addressed so far. In this paper, we explore how much the backward SR can contribute to image annotation, and be complementary to the forward SR. The co-training, which has been proved to be a semi-supervised method improving each other only if two classifiers are relatively independent, is then adopted to testify this complementary nature between two SRs in opposite directions. Finally, the co-training of two SRs in kernel space builds a cooperative kernel sparse representation (Co-KSR) method for image annotation. Experimental results and analyses show that two KSRs in opposite directions are complementary, and Co-KSR improves considerably over either of them with an image annotation performance better than other state-of-the-art semi-supervised classifiers such as transductive support vector machine, local and global consistency, and Gaussian fields and harmonic functions. Comparative experiments with a nonsparse solution are also performed to show that the sparsity plays an important role in the cooperation of image representations in two opposite directions. This paper extends the application of SR in image annotation and retrieval.