Computationally Efficient Formulation of Sparse Color Image Recovery in the JPEG Compressed Domain

Sparse representations provide a powerful framework for various image processing tasks, among which image recovery seems to be an already classical application. While most developments of image recovery applications are focused on finding the best dictionary, the possibility of using already existing sparse image representations tends to be ignored. This is the case of the JPEG compressed image representation, which is a sparse image representation in terms of the discrete cosine transform (DCT) dictionary. The development of sparse frameworks directly on the JPEG encoded image representation can lead to computationally efficient approaches. Here we introduce a DCT-based JPEG compressed domain formulation of the color image recovery process within a sparse representation framework and we prove mathematically and experimentally not only its numerical efficiency as compared to the pixel level formulation (the processing time is reduced up to 40 %), but also the good quality of the restoration results.     

Sparse Representation Based Fisher Discrimination Dictionary Learning for Image Classification

The employed dictionary plays an important role in sparse representation or sparse coding based image reconstruction and classification, while learning dictionaries from the training data has led to state-of-the-art results in image classification tasks. However, many dictionary learning models exploit only the discriminative information in either the representation coefficients or the representation residual, which limits their performance. In this paper we present a novel dictionary learning method based on the Fisher discrimination criterion. A structured dictionary, whose atoms have correspondences to the subject class labels, is learned, with which not only the representation residual can be used to distinguish different classes, but also the representation coefficients have small within-class scatter and big between-class scatter. The classification scheme associated with the proposed Fisher discrimination dictionary learning (FDDL) model is consequently presented by exploiting the discriminative information in both the representation residual and the representation coefficients. The proposed FDDL model is extensively evaluated on various image datasets, and it shows superior performance to many state-of-the-art dictionary learning methods in a variety of classification tasks.     

Robust visual tracking with structured sparse representation appearance model

In this paper, we present a structured sparse representation appearance model for tracking an object in a video system. The mechanism behind our method is to model the appearance of an object as a sparse linear combination of structured union of subspaces in a basis library, which consists of a learned Eigen template set and a partitioned occlusion template set. We address this structured sparse representation framework that preferably matches the practical visual tracking problem by taking the contiguous spatial distribution of occlusion into account. To achieve a sparse solution and reduce the computational cost, Block Orthogonal Matching Pursuit (BOMP) is adopted to solve the structured sparse representation problem. Furthermore, aiming to update the Eigen templates over time, the incremental Principal Component Analysis (PCA) based learning scheme is applied to adapt the varying appearance of the target online. Then we build a probabilistic observation model based on the approximation error between the recovered image and the observed sample. Finally, this observation model is integrated with a stochastic affine motion model to form a particle filter framework for visual tracking. Experiments on some publicly available benchmark video sequences demonstrate the advantages of the proposed algorithm over other state-of-the-art approaches.     

Sparse feature selection based on graph Laplacian for web image annotation

Confronted with the explosive growth of web images, the web image annotation has become a critical research issue for image search and index. Sparse feature selection plays an important role in improving the efficiency and performance of web image annotation. Meanwhile, it is beneficial to developing an effective mechanism to leverage the unlabeled training data for large-scale web image annotation. In this paper we propose a novel sparse feature selection framework for web image annotation, namely sparse Feature Selection based on Graph Laplacian (FSLG)². FSLG applies the l_2,1/2-matrix norm into the sparse feature selection algorithm to select the most sparse and discriminative features. Additional, graph Laplacian based semi-supervised learning is used to exploit both labeled and unlabeled data for enhancing the annotation performance. An efficient iterative algorithm is designed to optimize the objective function. Extensive experiments on two web image datasets are performed and the results illustrate that our method is promising for large-scale web image annotation.     

Image Annotation by Graph-Based Inference With Integrated Multiple/Single Instance Representations

In most of the learning-based image annotation approaches, images are represented using multiple-instance (local) or single-instance (global) features. Their performances, however, are mixed as for certain concepts, the single-instance representations of images are more suitable, while for others, the multiple-instance representations are better. Thus this paper explores a unified learning framework that combines the multiple-instance and single-instance representations for image annotation. More specifically, we propose an integrated graph-based semi-supervised learning framework to utilize these two types of representations simultaneously. We further explore three strategies to convert from multiple-instance representation into a single-instance one. Experiments conducted on the COREL image dataset demonstrate the effectiveness and efficiency of the proposed integrated framework and the conversion strategies.      

Image fusion with morphological component analysis

Image fusion can produce a single image that describes the scene better than the individual source image. One of the keys to image fusion algorithm is how to effectively and completely represent the source images. Morphological component analysis (MCA) believes that an image contains structures with different spatial morphologies and can be accordingly modeled as a superposition of cartoon and texture components, and that the sparse representations of these components can be obtained by some specific decomposition algorithms which exploit the structured dictionary. Compared with the traditional multiscale decomposition, which has been successfully applied to pixel-level image fusion, MCA employs the morphological diversity of an image and provides more complete representation for an image. Taking advantage of this property, we propose a multi-component fusion method for multi-source images in this paper. In our method, source images are separated into cartoon and texture components, and essential fusion takes place on the representation coefficients of these two components. Our fusion scheme is verified on three kinds of images and compared with six single-component fusion methods. According to the visual perceptions and objective evaluations on the fused results, our method can produce better fused images in our experiments, compared with other single-component fusion methods.     

Gaussian mixture model learning based image denoising method with adaptive regularization parameters

Gaussian mixture model learning based image denoising as a kind of structured sparse representation method has received much attention in recent years. In this paper, for further enhancing the denoised performance, we attempt to incorporate the gradient fidelity term with the Gaussian mixture model learning based image denoising method to preserve more fine structures of images. Moreover, we construct an adaptive regularization parameter selection scheme by combing the image gradient with the local entropy of the image. Experiment results show that our proposed method performs an improvement both in visual effects and peak signal to noise values.     

Robust face recognition via occlusion dictionary learning

Sparse representation based classification (SRC) has recently been proposed for robust face recognition. To deal with occlusion, SRC introduces an identity matrix as an occlusion dictionary on the assumption that the occlusion has sparse representation in this dictionary. However, the results show that SRC's use of this occlusion dictionary is not nearly as robust to large occlusion as it is to random pixel corruption. In addition, the identity matrix renders the expanded dictionary large, which results in expensive computation. In this paper, we present a novel method, namely structured sparse representation based classification (SSRC), for face recognition with occlusion. A novel structured dictionary learning method is proposed to learn an occlusion dictionary from the data instead of an identity matrix. Specifically, a mutual incoherence of dictionaries regularization term is incorporated into the dictionary learning objective function which encourages the occlusion dictionary to be as independent as possible of the training sample dictionary. So that the occlusion can then be sparsely represented by the linear combination of the atoms from the learned occlusion dictionary and effectively separated from the occluded face image. The classification can thus be efficiently carried out on the recovered non-occluded face images and the size of the expanded dictionary is also much smaller than that used in SRC. The extensive experiments demonstrate that the proposed method achieves better results than the existing sparse representation based face recognition methods, especially in dealing with large region contiguous occlusion and severe illumination variation, while the computational cost is much lower.     

结构约束和样本稀疏表示的图像修复

本文探讨了一种利用结构约束和样本稀疏表示,对结构信息缺损较大时的图像修复方法。利用多项式曲线拟合方式修复图像边缘信息,约束结构的修复;采用样本稀疏表示的窄带模型,优先修复结构信息;利用平移块的稀疏表示方法修复纹理信息。仿真实验结果表明,该方法修复图像质量高,既可较好地修复图像的边缘结构,又能保持结构的整体平滑性。     

基于结构字典的图像修复算法

随着稀疏表示理论的日渐完善,利用信号的稀疏性对图像进行修复得到广泛应用。本文针对传统的字典仅是一种无结构的扁平的原子的集合,没有充分利用原子之间相关性的问题,提出基于结构字典的图像修复算法。实验结果表明了该算法的有效性。基于结构字典的图像修复算法不仅可以训练字典更紧致地完成图像修复任务,而且训练得到的字典具有平移不变性、尺度灵活性等优点。     

Decomposing Monomial Representations of Solvable Groups

We present an efficient algorithm that decomposes a monomial representation of a solvable groupG into its irreducible components. In contradistinction to other approaches, we also compute the decomposition matrixA in the form of a product of highly structured, sparse matrices. This factorization provides a fast algorithm for the multiplication with A. In the special case of a regular representation, we hence obtain a fast Fourier transform forG . Our algorithm is based on a constructive representation theory that we develop. The term “constructive" signifies that concrete matrix representations are considered and manipulated, rather than equivalence classes of representations as it is done in approaches that are based on characters. Thus, we present well-known theorems in a constructively refined form and derive new results on decomposition matrices of representations. Our decomposition algorithm has been implemented in the GAP share package AREP. One application of the algorithm is the automatic generation of fast algorithms for discrete linear signal transforms.     

How can a sparse representation be made applicable for very low-dimensional data?

The sparse representation has achieved notable performance in the field of pattern classification, and has been adopted in many expert and intelligent applications such as access control and surveillance. However, sparse representation does not work as well for low-dimensional data as it does for high-dimensional data. For data of very low dimensionality, sparse representation methods usually have severe drawbacks; consequently, wider applications of sparse representations are seriously restricted. In this paper, we focus on this challenging problem and propose a very effective method for using sparse representations with low-dimensional data. Compared with the conventional sparse representation method, the proposed method achieves considerable improvement of classification accuracy by increasing the dimensionality of the data. Moreover, the proposed method is mathematically tractable and quite computationally efficient.     

Single Image Super-Resolution Reconstruction Technique based on A Single Hybrid Dictionary

A new sparse domain approach is proposed in this paper to realize the single image super-resolution (SR) reconstruction based upon one single hybrid dictionary, which is deduced from the mixture of both the high resolution (HR) image patch samples and the low resolution (LR) ones. Moreover, a linear model is proposed to characterize the relationship between the sparse representations of both the HR image patches and the corresponding LR ones over the same hybrid dictionary. It is shown that, the requirement on the identical sparse representation of both HR and LR image patches over the corresponding HR dictionary and the LR dictionary can be relaxed. It is unveiled that, the use of one single hybrid dictionary can not only provide a more flexible framework to keep the similar sparse characteristics between the HR patches and the corresponding degenerated LR patches, but also to accommodate their differences. On this basis, the sparse domain based SR reconstruction problem is reformulated. Moreover, the proposed linear model between the sparse representations of both the HR patch and the corresponding LR patch over the same hybrid dictionary offers us a new method to interpret the image degeneration characteristics in sparse domain. Finally, practical experimental results are presented to test and verify the proposed SR approach.     

基于综合几何关系稀疏自注意力机制的图像标注方法研究

针对基于Transformer框架的图像标注任务中提取视觉特征容易引入噪声问题且为了进一步提高视觉的上下文信息,提出了一种基于综合几何关系稀疏自注意力机制的图像标注方法。首先通过结合图像区域的绝对位置、相对位置和空间包含关系提取详细全面的视觉表示,获取图像中潜在的上下文信息;其次提出了注意力层权重矩阵的稀疏化方法,该方法解决了Transformer忽略图像区域的局部性并引入噪声信息的问题;最后,采用了强化学习方法作为指导策略,实现模型在句子级别优化目标序列。通过在MS-COCO数据集上进行的对比实验结果表明,提出的方法在BLEU1、BLEU4、METEOR、ROUGE-L、CIDEr和SPICE指标上分别比基线模型提升了0.2、0.7、0.1、0.3、1.2和0.4,有效提升了图像自动标注的性能。     

基于稀疏分解和聚类的自适应图像去噪新方法

随着信号稀疏表示原理的深入研究,稀疏分解越来越广泛地应用于图像处理领域。针对过完备字典构造和稀疏分解运算量巨大的问题,提出一种基于稀疏分解和聚类相结合的自适应图像去噪新方法。该方法首先通过改进的K均值(K-means)聚类算法训练样本,构造过完备字典;其次,通过训练过程中每一次迭代,自适应地更新字典的原子,使字典更适应样本的稀疏表示;然后利用正交匹配追踪(OMP)算法实现图像的稀疏表示,从而达到图像去噪的目的。实验结果表明：与传统的字典训练方法相比,新算法有效地降低了运算复杂度,并取得更好的图像去噪效果。     

基于随机投影与加权稀疏表示残差的光照鲁棒人脸识别方法

针对人脸识别中的光照变化问题,利用随机投影对传统稀疏表示分类器进行改进,提出一种基于随机投影与加权稀疏表示残差的光照鲁棒人脸识别方法。通过对人脸图像进行光照规范化处理,尽量消除人脸图像上的恶劣光照,取得经光照校正的人脸样本后进行多次随机空间投影,进一步丰富样本的光照不变特征,以减小光照变化对人脸识别带来的影响。在此基础上,对利用单一残差分类的传统稀疏表示分类方法进行改进,样本经过多次随机投影和稀疏表示会产生多个样本特征和重构残差,利用样本特征的能量来确定各个重构残差的融合权值,最终得到一种稳定性和可靠性更强的加权残差。在 Yale B 和 CMU PIE 两个光照变化较大的人脸库上的实验结果表明,改进的方法具有较强的光照鲁棒性。与传统稀疏表示方法相比,本文提出的方法在Yale B人脸库上两组实验的平均识别率分别提高了25.76%和46.39%,在CMU PIE上的平均识别率提高了10%左右。     

基于分层特征融合的行人分类

针对复杂环境中的行人检测问题,提出了一种有效的基于分层稀疏编码的图像表示方法。首先通过两层稀疏编码模型结合基于K-SVD的深度学习算法来获得图像的稀疏表示,对图像块及同一区域的高阶依赖关系进行了建模,形成一个有效的无监督特征学习方法;然后将得到的稀疏表示与SIFT描述符的稀疏表示进行特征融合,得到了更加全面、更加可判别的图像表示;最后结合SVM分类器应用于行人分类任务。实验结果表明,该行人分类方法对比同类方法在性能上有明显改善。     

Robust palmprint identification based on directional representations and compressed sensing

In this paper, we propose a novel approach for palmprint recognition, which contains two interesting components: directional representation and compressed sensing. Gabor wavelets can be well represented for biometric image for their similar characteristics to human visual system. However, these Gabor-based algorithms are not robust for image recognition under non-uniform illumination and suffer from the heavy computational burden. To improve the recognition performance under the low quality conditions with a fast operation speed, we propose novel palmprint recognition approach using directional representations. Firstly, the directional representation for palmprint appearance is obtained by the anisotropy filter, which is robust to drastic illumination changes and preserves important discriminative information. Then, the principal component analysis (PCA) is used for feature extraction to reduce the dimensions of the palmprint images. At last, based on a sparse representation on PCA feature, the compressed sensing is used to distinguish palms from different hands. Experimental results on the PolyU palmprint database show the proposed algorithm have better performance than that of the Gabor based methods.     

基于混合基稀疏图像表示的压缩传感图像重构

单一基函数不能对同时包含边缘和纹理信息的自然图像进行最优压缩传感图像重构. 本文根据Meyer的卡通--纹理图像模型和生物视觉原理, 用拉普拉斯塔式分解和圆对称轮廓波分别表示图像的光滑成分和边缘成分, 并构造了窄带轮廓波变换实现纹理成分的稀疏表示. 三种稀疏变换的基函数分别与视觉皮层中的侧膝体、简单细胞及栅格细胞的感受野类似. 结合三种图像稀疏表示方法和凸集交替投影算法提出了基于混合基稀疏表示的压缩传感图像重构算法. 实验结果表明,与基于块匹配三维变换迭代收缩的图像重构算法比较, 本文算法能获得更高的图像重构质量.     

A discriminative and sparse topic model for image classification and annotation

Image classification is to assign a category of an image and image annotation is to describe individual components of an image by using some annotation terms. These two learning tasks are strongly related. The main contribution of this paper is to propose a new discriminative and sparse topic model (DSTM) for image classification and annotation by combining visual, annotation and label information from a set of training images. The essential features of DSTM different from existing approaches are that (i) the label information is enforced in the generation of both visual words and annotation terms such that each generative latent topic corresponds to a category; (ii) the zero-mean Laplace distribution is employed to give a sparse representation of images in visual words and annotation terms such that relevant words and terms are associated with latent topics. Experimental results demonstrate that the proposed method provides the discrimination ability in classification and annotation, and its performance is better than the other testing methods (sLDA-ann, abc-corr-LDA, SupDocNADE, SAGE and MedSTC) for LabelMe, UIUC, NUS-WIDE and PascalVOC07 images.