Scale and Object Aware Image Thumbnailing

In this paper we study effective approaches to create thumbnails from input images. Since a thumbnail will eventually be presented to and perceived by a human visual system, a thumbnailing algorithm should consider several important issues in the process including thumbnail scale, object completeness and local structure smoothness. To address these issues, we propose a new thumbnailing framework named scale and object aware thumbnailing (SOAT), which contains two components focusing respectively on saliency measure and thumbnail warping/cropping. The first component, named scale and object aware saliency (SOAS), models the human perception of thumbnails using visual acuity theory, which takes thumbnail scale into consideration. In addition, the “objectness” measurement (Alexe et al. 2012) is integrated in SOAS, as to preserve object completeness. The second component uses SOAS to guide the thumbnailing based on either retargeting or cropping. The retargeting version uses the thin-plate-spline (TPS) warping for preserving structure smoothness. An extended seam carving algorithm is developed to sample control points used for TPS model estimation. The cropping version searches a cropping window that balances the spatial efficiency and SOAS-based content preservation. The proposed algorithms were evaluated in three experiments: a quantitative user study to evaluate thumbnail browsing efficiency, a quantitative user study for subject preference, and a qualitative study on the RetargetMe dataset. In all studies, SOAT demonstrated promising performances in comparison with state-of-the-art algorithms.     

A framework for automatic static and dynamic video thumbnail extraction

Video thumbnails enable users to see quick snapshots of video collections. To display the video thumbnails, the first frame or a frame selected by using simple low level features in each video clip has been set to the default thumbnail for the sake of computational efficiency and implementation simplicity. However, such methods often fail to represent the gist of the clip. To overcome this limitation, we present a new framework for both static and dynamic video thumbnail extraction. First, we formulate energy functions using the features which incorporate mid-level information to obtain superior thumbnailing. Since it is considered that frames whose layouts are similar to others in the clip are relevant in video thumbnail extraction, scene layouts are also considered in computing overall energy. For dynamic thumbnail generation, a time slot is determined by finding the duration showing the minimum energy. Experimental results show that the proposed method achieves comparable performance on a variety of challenging videos, and the subjective evaluation demonstrates the effectiveness of our method.     

一种精确缩略图保持的图像加密方案

手机电脑平板等的普及,使得照片在日常生活中更容易获得,并且人们习惯将大量照片存储在云端。但是,在享受云存储带来的便利的同时,用户也容易受到隐私泄露的威胁。虽然学者们设计出许多图像加密方案用来防止隐私泄露,然而往往忽略了图像的可用性。最近,Tajik等人提出了一种精确缩略图保持的加密方案,能够很好地平衡图像的隐私与可用性。但是,该方案在加密过程中仅以2个像素为一组,效率较低。为此提出一种利用分割法加密图像的方案,该方案以3个像素为一组进行加密,用于保持密文图像的缩略图与明文图像的缩略图一致,并且该方案相比Tajik方案具有更高的效率。实验表明,这个方案能够使密文图像精确地保持与明文相同的缩略图,平衡了隐私和可用性。     

An image fusion method based on object-oriented classification

In image fusion of different spatial resolution multispectral (MS) and panchromatic (PAN) images, a spectrally mixed MS pixel superimposes multiple mixed PAN pixels and multiple pure PAN pixels. This verifies that with increased spatial resolution in imaging, a low spatial resolution spectrally mixed subpixel may be unmixed to be a pure pixel. However, spectral unmixing of mixed MS subpixels is rarely considered in current remote-sensing image fusion methods, resulting in blurred fused images. In the image fusion method proposed in this article, such spectral unmixing is realized. In this method, the MS and PAN images are jointly segmented into image objects, image objects are classified to obtain a classification map of the PAN image and each MS subpixel is fused to be a pixel matching the class of the corresponding PAN pixel. Tested on spatially degraded IKONOS MS and PAN images with a significant spatial resolution ratio of 8:1, the fusion method offered fused images with high spectral quality and deblurred visualization.     

基于序列图像的鲁棒三维重建方法

提出了基于序列图像的鲁棒三维重建方法。首先利用两幅图像的最优参数估计,然后添加新图像并采用稀疏调整,减少图像坐标测量值的最小几何误差。通过对三维结构和摄像机参数进行全局优化处理,以提高重建的鲁棒性。实验结果表明,该方法提高了重建的精度和鲁棒性,并真实地再现了物体的三维模型。     

基于Lambert-Phong模型的图像盲取证方法

由于现有的图像盲取证方法中所使用的光照模型不能有效地表征物体表面的实际光照效果,提出Lambert-Phong光照模型。该模型同时考虑光照的漫反射和镜面反射,利用该光照模型对无限光源模式下的图像进行蓄意修改检测。实验结果表明,Lambert-Phong光照模型能较准确地计算出图像中不同目标的光照方向,有效地判别出图像是否经过蓄意修改。     

Multi-sensor image super-resolution with fuzzy cluster by using multi-scale and multi-view sparse coding for infrared image

Super-resolution (SR) methods are effective for generating a high-resolution image from a single low-resolution image. However, four problems are observed in existing SR methods. (1) They cannot reconstruct many details from a low-resolution infrared image because infrared images always lack detailed information. (2) They cannot extract the desired information from images because they do not consider that images naturally come at different scales in many cases. (3) They fail to reveal different physical structures of low-resolution patch because they extract features from a single view. (4) They fail to extract all the different patterns because they use only one dictionary to represent all patterns. To overcome these problems, we propose a novel SR method for infrared images. First, we combine the information of high-resolution visible light images and low-resolution infrared images to improve the resolution of infrared images. Second, we use multiscale patches instead of fixed-size patches to represent infrared images more accurately. Third, we use different feature vectors rather than a single feature to represent infrared images. Finally, we divide training patches into several clusters, and multiple dictionaries are learned for each cluster to provide each patch with a more accurate dictionary. In the proposed method, clustering information for low-resolution patches is learnt by using fuzzy clustering theory. Experiments validate that the proposed method yields better results in terms of quantization and visual perception than the state-of-the-art algorithms.     

一种多分辨多尺度的Retinex彩色图像增强算法

针对Retinex算法的缺点与不足,提出了一种改进的中心/环绕函数,以及图像增强分辨度概念。利用Retinex算法中图像增强分辨度与尺度之间对图像增强效果的互补特性,提出了多分辨多尺度的Retinex（Multi-Resolution and Multi-Scale Retinex,MRMSR）彩色图像增强算法。实验过程中,采用图像质量主观评价法和图像质量客观评价法相结合的方式,对多幅彩色图像进行了算法验证,并与MSR、MSRCR等算法进行比较。实验表明,MRMSR算法具有较好的图像增强效果,其图像增强效果明显优于其他算法,并能够有效地降低尺度对图像增强的影响。     

Iteration-free clustering algorithm for nonstationary image database

Image database systems must effectively and efficiently handle and retrieve images from a large collection of images. A serious problem faced by these systems is the requirement to deal with the nonstationary database. In an image database system, image features are typically organized into an indexing structure, and updating the indexing structure involves many computations. In this paper, this difficult problem is converted into a constrained optimization problem, and the iteration-free clustering (IFC) algorithm based on the Lagrangian function, is presented for adapting the existing indexing structure for a nonstationary database. Experimental results concerning recall and precision indicate that the proposed method provides a binary tree that is almost optimal. Simulation results further demonstrate that the proposed algorithm can maintain 94% precision in seven-dimensional feature space, even when the number of new-coming images is one-half the number of images in the original database. Finally, our IFC algorithm outperforms other methods usually applied to image databases.     

基于模糊相似性的彩色形态学图像处理方法

为了将传统灰度图像数学形态学扩展到彩色图像,提出一种结合矢量空间模糊相似性的彩色形态学图像处理方法。首先,在RGB彩色空间中利用彩色矢量间的距离和角度定义模糊相似性测度,以刻画与人类视觉感知相一致的彩色相似程度;以上述相似性测度为准则定义彩色空间中任意一组彩色的上确界和下确界;利用中心像素及其结构单元内像素的上确界和下确界构建彩色形态学的基本操作,包括膨胀、腐蚀、开、闭等操作;进一步将提出的彩色形态学操作应用于高分辨率遥感图像,通过实验对比验证其对地物目标的形变和平滑能力,说明其实用性和有效性。     

基于图的人-物交互识别

提出了一种基于图的人与物体的交互(Human-Object Interactions,HOIs)识别方法.为了对静态图像中人与物体间丰富的交互关系进行有效的表示,采用具有强大关系建模能力的图结构为图像生成对应的人-物交互关系图.为了对图像中上下文(context)信息加以利用,提出了引入注意力机制的特征处理网络(Fea...     

面向遥感影像镶嵌的SVR色彩一致性处理

由于成像条件与环境的差异,多景待镶嵌遥感影像之间往往会出现色彩差异,针对此问题,提出一种基于支持向量回归 (SVR)的色彩一致性处理方法。采用NDVI(归一化植被指数)阈值分割并结合光谱角匹配(SAM)的方法在影像重叠区域自动选取具有"不变特征"的像素作为样本;通过SVR建立输入影像到参考影像的灰度值变换方程,并对输入影像进行处理,使得待镶嵌影像具有与参考影像相同或者相似的亮度与对比度。采用TM、SPOT、无人机(UAV)影像等多源数据进行了实验,结果表明,该方法能够有效消除由系统因素引起的色差,与线性回归方法相比,该算法在方差、辐射分辨率等方面具有优势。     

Small object detection in cluttered image using a correlation based active contour model

This paper presents a novel energy function for active contour models based on autocorrelation function, which is capable of detecting small objects against a cluttered background. In the proposed method, image features are calculated using a combination of short-term autocorrelations (STA) computed from the image pixels to represent region information. The obtained features are exploited to define an energy function for the localized region-based active contour model called normalized accumulated short-term autocorrelation (NASTA). Minimizing this energy function, we can accurately detect small objects in images containing cluttered and textured backgrounds. Moreover, the proposed method provides high robustness against random noise and can precisely locate small objects in noisy backgrounds, difficult to be detected with naked eye. Experimental results indicate remarkable advantages of our approach comparing to existing methods.     

结合相位一致与全变差模型的高分辨率遥感图像边缘检测

目的边缘检测是有效利用遥感数据开展地物目标自动识别的重要步骤。高分辨率遥感图像地物类型复杂,细节信息过于丰富,使得基于相位一致的边缘检测结果中存在过多的噪声与伪边缘。为此提出了一种结合相位一致与全变差模型的高分辨率遥感图像边缘检测方法。方法根据相位一致原理,应用Log Gabor构造的2维相位一致模型,引入全变差去噪模型对基于相位一致的边缘强度图进行改进。结果借助有界变差空间对图像光滑性的约束,实现了高分辨率遥感图像噪声去除与伪边缘抑制,利用改进后的相位一致边缘强度图可有效检测高分辨率遥感图像的边缘。结论实验结果表明,与相位一致模型、Canny算法相比,该方法能消除了高分辨率遥感图像中同类地物内部细节特征形成的噪声,抑制相位一致边缘检测结果中的伪边缘,突出地物的真实边缘,并能正确地提取地物目标的整体轮廓信息,有助于后续地物目标的自动识别。     

A new framework for remote sensing image super-resolution: Sparse representation-based method by processing dictionaries with multi-type features

Remote sensing images play an important role in many practical applications, however, due to the physical limitations of remote sensing devices, it is difficult to obtain images at an expecting high resolution level. Acquiring high-resolution(HR) images from the original low-resolution(LR) ones with super-resolution(SR) methods has always been an attractive proposition in embedded systems including various kinds of tablet PC and smart phone. SR methods based on sparse representation have been successfully used in processing remote sensing images, however, they have two major problems in common. First, they use only one type of image features to represent the low resolution(LR) images. However, one single type of features cannot accurately represent an image due to the diverse structures of the image, as a result, artifacts would be produced simultaneously. Second, many dictionary learning methods try to build a universal dictionary with only one single type of features. However, apparently, a dictionary with a single type of features is not enough to capture the different structures of a remote sensing image, without any doubt, the resultant image would turn out to be a poor one. To overcome the problems above, we propose a new framework for remote sensing image super resolution: sparse representation-based SR method by processing dictionaries with multi-type features. First, in order to represent the remote sensing image more accurately, different types of features are extracted from images. Second, to achieve a better performance, various dictionaries with multi-type features are learned to capture the essential structures of the image. Then, it’s proposed to adaptively control the weights of the high resolution(HR) patches obtained by different dictionaries. Numerous experiments validate that this proposed framework brings better results in terms of both objective quantitation and visual perception than other compared algorithms.     

Super-resolution of human face image using canonical correlation analysis

Super-resolution reconstruction of face image is the problem of reconstructing a high resolution face image from one or more low resolution face images. Assuming that high and low resolution images share similar intrinsic geometries, various recent super-resolution methods reconstruct high resolution images based on a weights determined from nearest neighbors in the local embedding of low resolution images. These methods suffer disadvantages from the finite number of samples and the nature of manifold learning techniques, and hence yield unrealistic reconstructed images.To address the problem, we apply canonical correlation analysis (CCA), which maximizes the correlation between the local neighbor relationships of high and low resolution images. We use it separately for reconstruction of global face appearance, and facial details. Experiments using a collection of frontal human faces show that the proposed algorithm improves reconstruction quality over existing state-of-the-art super-resolution algorithms, both visually, and using a quantitative peak signal-to-noise ratio assessment.     

两级上下文卷积网络宽视场图像小目标检测方法

目标检测和识别已经在输电线路巡检中被广泛采用。由于图像数据量大,小目标分辨率低,现有的图像金字塔、特征金字塔和多异构特征融合等方法虽能准确地检测目标,却非常耗时,因而快速、准确地检测宽视场图像中小目标仍是一个挑战。此算法提出一个两个Faster-RCNs级联的上下文宽视场小目标检测卷积网络,首先,针对降分辨率的宽视场图像,利用一个Faster R-CNN来检测目标的上下文区域,然后,针对上下文区域对应的高分辨率原始图像,利用Faster R-CNN来检测来小目标。我们用航拍输电线路图像数据集进行了目标检测试验,试验结果表明,小目标检测方法达到了88%的检测精度,比单级Faster R-CNN检测方法具有更高的准确率。     

土地利用分类规则集的构建与应用研究

利用高空间分辨率影像,采用面向对象的分类方法,通过多种方法确定了土地利用类型的适宜尺度,形成了多尺度的影像对象层次网络体系。对影像对象进行多特征与空间关系描述,有效集成了辅助特征和专家知识,构建了影像对象分类规则集。研究区分类结果表明:地物分布特征及其空间关系规则的应用,可以有效地提高分类精度,得到更好的语义区分和更精确的分类结果。以期仅作少许改动就可将方案应用到条件类似的高分辨率影像分类中。     

亮度—梯度联合约束的车牌图像超分辨率重建

目的 受成像距离、光照条件、动态模糊等因素影响,监控系统拍摄的车牌图像往往并不具备较高的可辨识度。为改善成像质量,提升对车牌的识别能力,提出一种基于亮度与梯度联合约束的车牌图像超分辨率重建方法。方法 首先充分结合亮度约束和梯度约束的优势,实现对运动位移和模糊函数的精确估计;为抑制重建图像中的噪声与伪影,基于车牌图像的文字化特征,进一步确定了亮度与梯度联合约束的图像先验模型。结果 为验证该方法的有效性,利用监控系统获得4组车牌图像,分别进行模拟和真实的超分辨率重建实验。在模拟实验中将联合约束图像先验重建结果与拉普拉斯、Huber-Markov（HMRF）以及总变分（TV）先验的处理结果进行对比,联合约束先验对车牌纹理信息的恢复效果优于其他3种常见图像先验;同时,在模拟和真实实验中,将本文算法与双三次插值、传统最大后验概率、非线性扩散正则化和自适应范数正则化方法的超分辨率重建结果进行比较,模拟实验的结果表明,在不添加噪声情况下,该算法峰值信噪比（PSNR）和结构相似性（SSIM）指标分别为35.326 dB和0.958,优于其他4种算法;该算法在真实实验中,能够有效增强车牌图像纹理信息,获得较优的视觉效果,通过对重建车牌图像的字符识别精度比较,本文算法重建结果的识别精度远高于其他3种算法,平均字符差距为1.3。结论 模拟和真实图像序列的实验结果证明,基于亮度—梯度联合约束的超分辨率重建方法,能够降低运动和模糊等参数的估计误差,有效减少图像中存在的模糊和噪声,提高车牌的识别精度。该算法广泛适用于因光照变化、相对运动等因素影响下的低质量车牌图像超分辨率重建。