Image inpainting with salient structure completion and texture propagation

Image inpainting technique uses structural and textural information to repair or fill missing regions of a picture. Inspired by human visual characteristics, we introduce a new image inpainting approach which includes salient structure completion and texture propagation. In the salient structure completion step, incomplete salient structures are detected using wavelet transform, and completion order is determined through color texture and curvature features around the incomplete salient structures. Afterwards, curve fitting and extension are used to complete the incomplete salient structures. In the texture propagation step, the proposed approach first synthesizes texture information of completed salient structures. Then, the texture information is propagated into the remaining missing regions. A number of examples on real and synthetic images demonstrate the effectiveness of our algorithm in removing occluding objects. Our results compare favorably to those obtained by existing greedy inpainting techniques.     

综合距离和相关性的图像检索算法

在基于内容的图像检索中,大多数都是采用距离来测试两幅图像的相似性.提出了一种新的计算相关系数的方法并结合这种方法和距离来判断两幅图像的相似性,将其应用于CBIR(content-based image retrieval)系统.在对所提出的算法进行的实验中,用了10 000幅图像来测试了所提出的算法,实验结果表明:在同一个CBIR系统中,引入相关性能够提高图像的检索精度,解决了只用距离来判断两幅图像相似性的不足,对于基于低级特征的图像检索系统是一个很好的改进.     

基于优先权改进和块划分的图像修复

目的 针对基于样本块的Criminisi图像修复算法易发生置信项迅速下降趋于零,使优先权计算公式失效,导致修复顺序错乱造成的修复效果失真问题,以及在搜索匹配块时存在的搜索范围过大,效率过低,易出现匹配到不符合视觉效果的纹理块问题,提出一种基于优先权改进和块划分的图像修复算法。方法 首先重新定义优先权中的置信项,用样本块中的棋盘距离替代原计算公式,保证优先权一直发挥作用,从而减少因修复顺序不合理造成的错误匹配;其次根据图像纹理信息将其自适应划分为不同大小的图像块,使待修复样本块只在具有相似特征的图像块区域内搜索匹配。结果 实验结果表明,新定义的优先权,保证了修复算法的正常进行,改善了修复图像的视觉效果;由图像自适应块划分引导匹配过程,可使匹配在更少的候选块中进行,提高了算法速度。将本文方法与3种全局搜索匹配方法和1种局部搜索匹配方法进行修复结果对比分析,本文方法的修复结果视觉完整性较好,而且修复时间小于其中3种算法。结论 通过改进Criminisi算法优先权中的置信项,避免因其趋于零导致的修复顺序错乱造成的错误累积情况的发生;同时通过改进待修复匹配块的搜索范围,对整幅图像进行自适应块划分,使搜索只在相似块中进行,不仅减少了时间,而且提高了匹配的准确性。本文方法对于自然图像中大面积目标物体移除方面有较好的应用,可获得较满意的修复效果。     

Fuzzy C-Means clustering through SSIM and patch for image segmentation

In this study, we propose a new robust Fuzzy C-Means (FCM) algorithm for image segmentation called the patch-based fuzzy local similarity c-means (PFLSCM). First of all, the weighted sum distance of image patch is employed to determine the distance of the image pixel and the cluster center, where the comprehensive image features are considered instead of a simple level of brightness (gray value). Second, the structural similarity (SSIM) index takes into account similar degrees of luminance, contrast, and structure of image. The DSSIM (distance for structural similarity) metric is developed on a basis of SSIM in order to characterize the distance between two pixels in the whole image. Next a new similarity measure is proposed. Furthermore, a new fuzzy coefficient is proposed via the new similarity measure together with the weighted sum distance of image patch, and then the PFLSCM algorithm is put forward based on the idea of image patch and this coefficient. Through a collection of experimental studies using synthetic and publicly available images, we demonstrate that the proposed PFLSCM algorithm achieves improved segmentation performance in comparison with the results produced by some related FCM-based algorithms.     

一种快速图像块填充算法及其在遥感影像处理中的应用

针对Drori的图像块填充算法计算复杂度高,计算量大的缺点,提出了一个快速的改进算法。并结合光学遥感影像的特点,将其应用到遥感影像中的厚云及阴影去除上。实验结果表明,在能够取得近似相同结果的情况下,本文算法与Drori算法相比,能够较好地缩短计算时间。     

Multivariate image similarity in the compressed domain using statistical graph matching

We address the problem of image similarity in the compressed domain, using a multivariate statistical test for comparing color distributions. Our approach is based on the multivariate Wald-Wolfowitz test, a nonparametric test that assesses the commonality between two different sets of multivariate observations. Using some pre-selected feature attributes, the similarity measure provides a comprehensive estimate of the match between different images based on graph theory and the notion of minimal spanning tree (MST). Feature extraction is directly provided from the JPEG discrete cosine transform (DCT) domain, without involving full decompression or inverse DCT. Based on the zig-zag scheme, a novel selection technique is introduced that guarantees image's enhanced invariance to geometric transformations. To demonstrate the performance of the proposed method, the application on a diverse collection of images has been systematically studied in a query-by-example image retrieval task. Experimental results show that a powerful measure of similarity between compressed images can emerge from the statistical comparison of their pattern representations.     

Combinational image watermarking in the spatial and frequency domains

In order to provide more watermarks and to minimize the distortion of the watermarked image, a novel technique using the combinational spatial and frequency domains is presented in this paper. The splitting of the watermark image into two parts, respectively, for spatial and frequency insertion relies on the user's preference and data importance. Experimental results provide the comparisons when different sized watermarks are embedded into a grayscale image. The proposed combinational image watermarking possesses the following advantages. More watermark data can be inserted into the host image, so that the capacity is increased. The splitting of the watermark into two parts makes the degree of protection double. The splitting strategy can be designed even more complicated to be unable to compose. Furthermore, to enhance robustness, a random permutation of the watermark is used to defeat the attacks of signal processing, such as image crops.     

基于层次化及最小二乘的精确图像配准

针对传统对数极坐标傅立叶变换(log-polar mapping based Fourier transform, LPMFT)在大尺度、大旋转及大平移变换情况下不能精确估计图像对之间的变换参数, 提出基于层次化及最小二乘的图像配准方法(multi-resolution analysis and least square optimization, MALSO): 首先, 使用小波变换将图像分解为多分层结构, 并将每层的低频部分作为待匹配图像; 其次, 在每层中, 引入窗口函数及自适应滤波函数以减少谱泄漏, 混叠及插值误差的影响; 最后, 构建一个代价函数, 并通过最小二乘法求解最优参数. 实验表明, 该方法既满足大尺度, 大旋转及大平移参数准确估计要求, 又比LPMFT对遮挡更具鲁棒性, 有一定的理论及应用价值.     

Full-frame cosine transform image compression for medical and industrial applications

Full-frame image compression capable of a 101 ratio or higher was developed for radiological applications where block artifacts are not acceptable. Applications in machine vision with similar stringencies can utilize the same principles and hardware design. Subsecond compression speed for image sizes up to 1K × 1K × 8 can be achieved by DMA and DSP designs using off-the-shelf components and customized bus architecture. Successions of completed hardware module design and proposed enhancements are reported with technical details. The possibility of 512 × 512 × 8 image compression in real time is also explored with design examples.     

Visual word spatial arrangement for image retrieval and classification

We present word spatial arrangement (WSA), an approach to represent the spatial arrangement of visual words under the bag-of-visual-words model. It lies in a simple idea which encodes the relative position of visual words by splitting the image space into quadrants using each detected point as origin. WSA generates compact feature vectors and is flexible for being used for image retrieval and classification, for working with hard or soft assignment, requiring no pre/post processing for spatial verification. Experiments in the retrieval scenario show the superiority of WSA in relation to Spatial Pyramids. Experiments in the classification scenario show a reasonable compromise between those methods, with Spatial Pyramids generating larger feature vectors, while WSA provides adequate performance with much more compact features. As WSA encodes only the spatial information of visual words and not their frequency of occurrence, the results indicate the importance of such information for visual categorization.     

基于结构稀疏度和块差异度的目标移除图像修复

针对目标移除修复方法中存在的修复顺序不合理以及错误匹配问题,提出一种基于结构稀疏度和块差异度的图像修复方法。首先,在优先权中增加块的结构稀疏度计算,使优先权不仅依赖于目标块的几何特征,而且可以反映其邻域特征,提高了对目标块所处区域特征的辨识度,从而使修复顺序更加合理。其次,定义目标块与样本块之间的差异度,并在此基础上定义新的匹配规则,不仅对已存在像素之间的相似程度进行衡量,而且对已存在像素与填充像素之间的差异程度进行衡量,从而有效防止错误匹配以及错误累积。实验结果表明,该方法可以有效提高图像的修复效果,使修复图像更加符合视觉一致性要求。     

Locally affine patch mapping and global refinement for image super-resolution

This paper deals with the super-resolution (SR) problem based on a single low-resolution (LR) image. Inspired by the local tangent space alignment algorithm in [16] for nonlinear dimensionality reduction of manifolds, we propose a novel patch-learning method using locally affine patch mapping (LAPM) to solve the SR problem. This approach maps the patch manifold of low-resolution image to the patch manifold of the corresponding high-resolution (HR) image. This patch mapping is learned by a training set of pairs of LR/HR images, utilizing the affine equivalence between the local low-dimensional coordinates of the two manifolds. The latent HR image of the input (an LR image) is estimated by the HR patches which are generated by the proposed patch mapping on the LR patches of the input. We also give a simple analysis of the reconstruction errors of the algorithm LAPM. Furthermore we propose a global refinement technique to improve the estimated HR image. Numerical results are given to show the efficiency of our proposed methods by comparing these methods with other existing algorithms.     

Combining stationary wavelet transform and self-organizing maps for brain MR image segmentation

This study presents an image segmentation system that automatically segments and labels T1-weighted brain magnetic resonance (MR) images. The method is based on a combination of unsupervised learning algorithm of the self-organizing maps (SOM) and supervised learning vector quantization (LVQ) methods. Stationary wavelet transform (SWT) is applied to the images to obtain multiresolution information for distinguishing different tissues. Statistical information of the different tissues is extracted by applying spatial filtering to the coefficients of SWT. A multidimensional feature vector is formed by combining SWT coefficients and their statistical features. This feature vector is used as input to the SOM. SOM is used to segment images in a competitive unsupervised approach and an LVQ system is used for fine-tuning. Results are evaluated using Tanimoto similarity index and are compared with manually segmented images. Quantitative comparisons of our system with the other methods on real brain MR images using Tanimoto similarity index demonstrate that our system shows better segmentation performance for the gray matter while it gives average results for white matter.     

Ranking consistency for image matching and object retrieval

The goal of object retrieval is to rank a set of images by the similarity of their contents to those of a query image. However, it is difficult to measure image content similarity due to visual changes caused by varying viewpoint and environment. In this paper, we propose a simple, efficient method to more effectively measure content similarity from image measurements. Our method is based on the ranking information available from existing retrieval systems. We observe that images within the set which, when used as queries, yield similar ranking lists are likely to be relevant to each other and vice versa. In our method, ranking consistency is used as a verification method to efficiently refine an existing ranking list, in much the same fashion that spatial verification is employed. The efficiency of our method is achieved by a list-wise min-Hash scheme, which allows rapid calculation of an approximate similarity ranking. Experimental results demonstrate the effectiveness of the proposed framework and its applications.     

Design considerations for image segmentation quality assessment measures

Factors to consider when designing quality assessment measures for image segmentation are discussed. Quality assessment requires one manually generated segmentation (for reference) plus computer-generated segmentations corresponding to different image segmentation algorithms or algorithm parameter settings. Since true pixel class assignments are seldom available, one must typically rely on a trained human analyst to produce a reference by using a mouse to draw boundaries of perceived regions on a digital image background. Different algorithms and parameter settings can be compared by ranking computed disparities between maps of computer-generated region boundaries and region boundaries from a common reference.Proximity-based association between two boundary pixels is discussed in the context of association distance. Motivated by the concept of phase-modulated signals, a penalty factor on the degree of association is then introduced as some non-negative power (phase modulation order) of the cosine of disparity in phase (boundary direction) between two boundary pixels. Families of matching measures between maps of region boundaries are defined as functions of associations between many pairs of boundary pixels. The measures are characterized as one-way (reflecting relationships in one direction between region boundaries from two segmentations) vs. two-way (reflecting relationships in both directions). Measures of inconsistency between perceived and computed matches of computer and manually generated region boundaries are developed and exercised so that effects of association distance, phase modulation, and choice of matching measure on image segmentation quality assessment can be quantified. It is quantitatively established that consistency can be significantly improved by using two-way measures in conjunction with high-order phase modulation and moderate association distances.     

A general framework for image fusion based on multi-scale transform and sparse representation

In image fusion literature, multi-scale transform (MST) and sparse representation (SR) are two most widely used signal/image representation theories. This paper presents a general image fusion framework by combining MST and SR to simultaneously overcome the inherent defects of both the MST- and SR-based fusion methods. In our fusion framework, the MST is firstly performed on each of the pre-registered source images to obtain their low-pass and high-pass coefficients. Then, the low-pass bands are merged with a SR-based fusion approach while the high-pass bands are fused using the absolute values of coefficients as activity level measurement. The fused image is finally obtained by performing the inverse MST on the merged coefficients. The advantages of the proposed fusion framework over individual MST- or SR-based method are first exhibited in detail from a theoretical point of view, and then experimentally verified with multi-focus, visible-infrared and medical image fusion. In particular, six popular multi-scale transforms, which are Laplacian pyramid (LP), ratio of low-pass pyramid (RP), discrete wavelet transform (DWT), dual-tree complex wavelet transform (DTCWT), curvelet transform (CVT) and nonsubsampled contourlet transform (NSCT), with different decomposition levels ranging from one to four are tested in our experiments. By comparing the fused results subjectively and objectively, we give the best-performed fusion method under the proposed framework for each category of image fusion. The effect of the sliding window’s step length is also investigated. Furthermore, experimental results demonstrate that the proposed fusion framework can obtain state-of-the-art performance, especially for the fusion of multimodal images.     

一种平移旋转图像的角点匹配方法

在图像角点匹配过程中,目标图像往往存在平移旋转现象,直接影响匹配效果,为此提出了一种平移旋转图像的角点匹配方法。该方法首先利用角点检测生成自相关矩阵的梯度信息与仿射变换相结合,构造确定性退火算法中的自由能函数,然后对该自由能函数进行优化,获取待匹配角点间的仿射变换参数,最后利用该变换参数实现角点匹配。实验结果表明,该方法能够在目标图像发生平移旋转的情况下,有效实现角点匹配。