Sparse coding and dictionary learning with class-specific group sparsity

In recent years, sparse coding via dictionary learning has been widely used in many applications for exploiting sparsity patterns of data. For classification, useful sparsity patterns should have discrimination, which cannot be well achieved by standard sparse coding techniques. In this paper, we investigate structured sparse coding for obtaining discriminative class-specific group sparsity patterns in the context of classification. A structured dictionary learning approach for sparse coding is proposed by considering the \(\ell _{2,0}\) norm on each class of data. An efficient numerical algorithm with global convergence is developed for solving the related challenging \(\ell _{2,0}\) minimization problem. The learned dictionary is decomposed into class-specific dictionaries for the classification that is done according to the minimum reconstruction error among all the classes. For evaluation, the proposed method was applied to classifying both the synthetic data and real-world data. The experiments show the competitive performance of the proposed method in comparison with several existing discriminative sparse coding methods.

     

融合极端学习机的判别性分析字典学习算法

研究表明,端学习机和判别性字典学习算法在图像分类领域极具有高效和准确的优势。然而,这两种方法也具有各自的缺点,极端学习机对噪声的鲁棒性较差,判别性字典学习算法在分类过程中耗时较长。为统一这种互补性以提高分类性能,文中提出了一种融合极端学习机的判别性分析字典学习模型。该模型利用迭代优化算法学习最优的判别性分析字典和极端学习机分类器。为验证所提算法的有效性,利用人脸数据集进行分类。实验结果表明,与目前较为流行的字典学习算法和极端学习机相比,所提算法在分类过程中具有更好的效果。     

An AK-BRP dictionary learning algorithm for video frame sparse representation in compressed sensing

Sparsifying transform is an important prerequisite in compressed sensing. And it is practically significant to research the fast and efficient signal sparse representation methods. In this paper, we propose an adaptive K-BRP (AK-BRP) dictionary learning algorithm. The bilateral random projection (BRP), a method of low rank approximation, is used to update the dictionary atoms. Furthermore, in the sparse coding stage, an adaptive sparsity constraint is utilized to obtain sparse representation coefficient and helps to improve the efficiency of the dictionary update stage further. Finally, for video frame sparse representation, our adaptive dictionary learning algorithm achieves better performance than K-SVD dictionary learning algorithm in terms of computation cost. And our method produces smaller reconstruction error as well.

     

Stochastic learning of multi-instance dictionary for earth mover’s distance-based histogram comparison

Dictionary plays an important role in multi-instance data representation. It maps bags of instances to histograms. Earth mover’s distance (EMD) is the most effective histogram distance metric for the application of multi-instance retrieval. However, up to now, there is no existing multi-instance dictionary learning methods designed for EMD-based histogram comparison. To fill this gap, we develop the first EMD-optimal dictionary learning method using stochastic optimization method. In the stochastic learning framework, we have one triplet of bags, including one basic bag, one positive bag, and one negative bag. These bags are mapped to histograms using a multi-instance dictionary. We argue that the EMD between the basic histogram and the positive histogram should be smaller than that between the basic histogram and the negative histogram. Base on this condition, we design a hinge loss. By minimizing this hinge loss and some regularization terms of the dictionary, we update the dictionary instances. The experiments over multi-instance retrieval applications shows its effectiveness when compared to other dictionary learning methods over the problems of medical image retrieval and natural language relation classification.

     

Sparse Representation Based Fisher Discrimination Dictionary Learning for Image Classification

The employed dictionary plays an important role in sparse representation or sparse coding based image reconstruction and classification, while learning dictionaries from the training data has led to state-of-the-art results in image classification tasks. However, many dictionary learning models exploit only the discriminative information in either the representation coefficients or the representation residual, which limits their performance. In this paper we present a novel dictionary learning method based on the Fisher discrimination criterion. A structured dictionary, whose atoms have correspondences to the subject class labels, is learned, with which not only the representation residual can be used to distinguish different classes, but also the representation coefficients have small within-class scatter and big between-class scatter. The classification scheme associated with the proposed Fisher discrimination dictionary learning (FDDL) model is consequently presented by exploiting the discriminative information in both the representation residual and the representation coefficients. The proposed FDDL model is extensively evaluated on various image datasets, and it shows superior performance to many state-of-the-art dictionary learning methods in a variety of classification tasks.     

Collaborative representation analysis methods for feature extraction

Recently, sparse representation (SR) theory gets much success in the fields of pattern recognition and machine learning. Many researchers use SR to design classification methods and dictionary learning via reconstruction residual. It was shown that collaborative representation (CR) is the key part in sparse representation-based classification (SRC) and collaborative representation-based classification (CRC). Both SRC and CRC are good classification methods. Here, we give a collaborative representation analysis (CRA) method for feature extraction. Not like SRC-/CRC-based methods (e.g., SPP and CRP), CRA could directly extract the features like PCA and LDA. Further, a Kernel CRA (KCRA) is developed via kernel tricks. The experimental results on FERET and AR face databases show that CRA and KCRA are two effective feature extraction methods and could get good performance.

     

字典学习模型、算法及其应用研究进展

稀疏表示模型常利用训练样本学习过完备字典, 旨在获得信号的冗余稀疏表示. 设计简单、高效、通用性强的字典学习算法是目前的主要研究方向之一, 也是信息领域的研究热点. 基于综合稀疏模型的字典学习方法已经广泛应用于图像分类、图像去噪、图像超分辨率和压缩成像等领域. 近些年来, 解析稀疏模型、盲字典模型和信息复杂度模型等新模型的出现丰富了字典学习理论, 使得更广泛类型的信号能够被简单性描述. 本文详细介绍了综合字典、解析字典、盲字典和基于信息复杂度字典学习的基本模型及其算法, 阐述了字典学习的典型应用, 指出了字典学习的进一步研究方向.     

结合分块LBP与投影字典对学习的表情识别

针对字典学习和分类过程所采用的表示系数[l1]范数稀疏约束求解过程代价过高的问题,同时为获取更有效的表情相关特征来进行字典学习,提出一种结合分块LBP特征与投影字典对学习的表情识别方法。提取图像的分块LBP特征替代原始数据样本,用来训练和测试。学习一个分析字典和一个综合字典,分析字典可求得表示系数,综合字典具备重构能力。利用综合字典和分析字典求出各类别的重构误差进行分类,从而实现人脸表情识别。在JAFFE和CK+数据库上的实验结果表明,与其他方法相比,所提出的方法不仅可以大大降低训练和测试阶段的时间复杂度,而且可以在分类任务中达到更高的识别率。     

基于核字典学习的图像分类

航拍图像往往具有场景复杂、数据维度大的特点,对于该类图像的自动分类一直是研究的热点。针对航拍原始数据特征维度过高和数据线性不可分的问题,在字典学习和稀疏表示的基础上提出了一种结合核字典学习和线性鉴别分析的目标识别方法。首先学习核字典并通过核字典获取目标样本的稀疏表示,挖掘数据的内部结构;其次采用线性鉴别分析,加强稀疏表示的可分性;最后利用支持向量机对目标进行分类。实验结果表明,与传统基于子空间特征提取的算法和基于字典学习的算法相比,基于核字典学习与鉴别分析的算法分类性能优越。     

Dictionary learning for VQ feature extraction in ECG beats classification

Vector quantization(VQ) can perform efficient feature extraction from electrocardiogram (ECG) with the advantages of dimensionality reduction and accuracy increase. However, the existing dictionary learning algorithms for vector quantization are sensitive to dirty data, which compromises the classification accuracy. To tackle the problem, we propose a novel dictionary learning algorithm that employs k-medoids cluster optimized by k-means++ and builds dictionaries by searching and using representative samples, which can avoid the interference of dirty data, and thus boost the classification performance of ECG systems based on vector quantization features. We apply our algorithm to vector quantization feature extraction for ECG beats classification, and compare it with popular features such as sampling point feature, fast Fourier transform feature, discrete wavelet transform feature, and with our previous beats vector quantization feature. The results show that the proposed method yields the highest accuracy and is capable of reducing the computational complexity of ECG beats classification system. The proposed dictionary learning algorithm provides more efficient encoding for ECG beats, and can improve ECG classification systems based on encoded feature.     

Learning low-rank and discriminative dictionary for image classification

Dictionary learning plays a crucial role in sparse representation based image classification. In this paper, we propose a novel approach to learn a discriminative dictionary with low-rank regularization on the dictionary. Specifically, we apply Fisher discriminant function to the coding coefficients to make the dictionary more discerning, that is, a small ratio of the within-class scatter to between-class scatter. In practice, noisy information in the training samples will undermine the discriminative ability of the dictionary. Inspired by the recent advances in low-rank matrix recovery theory, we apply low-rank regularization on the dictionary to tackle this problem. The iterative projection method (IPM) and inexact augmented Lagrange multiplier (ALM) algorithm are adopted to solve our objective function. The proposed discriminative dictionary learning with low-rank regularization (D²L²R²) approach is evaluated on four face and digit image datasets in comparison with existing representative dictionary learning and classification algorithms. The experimental results demonstrate the superiority of our approach.     

Face recognition based on manifold constrained joint sparse sensing with K-SVD

Face recognition based on Sparse representation idea has recently become an important research topic in computer vision community. However, the dictionary learning process in most of the existing approaches suffers from the perturbations brought by the variations of the input samples, since the consistence of the learned dictionaries from similar input samples based on K-SVD are not well addressed in the existing literature. In this paper, we will propose a novel technique for dictionary learning based on K-SVD to address the consistence issue. In particular, the proposed method embeds the manifold constraints into a standard dictionary learning framework based on k-SVD and force the optimization process to satisfy the structure preservation requirement. Therefore, this new approach can consistently integrate the manifold constraints during the optimization process, and it can contribute a better solution which is robust to the variance of the input samples. Extensive experiments on several popular face databases show a consistent performance improvement in comparison to some related state-of-the-art algorithms.

     

面向运动想象脑电信号识别的多层判别字典对学习方法

面向运动想象的脑机接口（brain computer interface,BCI）能够利用自主想象的特定动作触发脑电信号直接实时控制外部电子设备。运动想象脑电信号（motor imagery electroencephalogram,MI-EEG）存在信噪比低、类内差异大、类间差异小等特点,导致MI-EEG的识别率较低且不稳定。针对这一问题,提出了多层判别字典对学习（multilayer discriminant dictionary pair learning,MDDPL）方法。与基于字典学习的MI-EEG识别方法不同,MDDPL将字典对学习融入多层学习模型,通过一系列非线性方法将数据投影到更具判别力的子空间。在综合字典和分析字典的共同作用下,前一层的编码向量作为当前层的输入,同时在每一层模型上构建基于分析字典的多分类项,以保证稀疏编码的分类误差最小化,增强模型的类别区分能力。另外,对最后一层的稀疏编码施加低秩约束,以保证同类编码的紧凑性和相似性。在目标式求解中,使用交替更新策略得到每个参数的解析解,使得参数同时得到最优解。在国际BCI竞赛数据集上的实验结果表明,MDDPL方法在所有对比算法中取得了最佳的分类性能。     

基于矢量表示组合的字典学习性能分析

在字典学习算法中,使用图像的多矢量表示相比单一矢量表示,可以获得分类精度更高且更具有鲁棒性的分类模型.本文中我们采用多种矢量表示的组合以及合理的加权对数和方案,来提升字典算法的性能.通过在公共人脸数据集上进行实验,验证了我们的方法应用于字典学习具有更高的准确度和鲁棒性.充分挖掘和利用表示多样性可以获得被观察对象的各种潜在外观以及图像高分类精度.     

Deep locality-sensitive discriminative dictionary learning for semantic video analysis

Video semantic analysis (VSA) has received significant attention in the area of Machine Learning for some time now, particularly video surveillance applications with sparse representation and dictionary learning. Studies have shown that the duo has significantly impacted on the classification performance of video detection analysis. In VSA, the locality structure of video semantic data containing more discriminative information is very essential for classification. However, there has been modest feat by the current SR-based approaches to fully utilize the discriminative information for high performance. Furthermore, similar coding outcomes are missing from current video features with the same video category. To handle these issues, we first propose an improved deep learning algorithm—locality deep convolutional neural network algorithm (LDCNN) to better extract salient features and obtain local information from semantic video. Second, we propose a novel DL method, called deep locality-sensitive discriminative dictionary learning (DLSDDL) for VSA. In the proposed DLSDDL, a discriminant loss function for the video category based on sparse coding of sparse coefficients is introduced into the structure of the locality-sensitive dictionary learning (LSDL) method. After solving the optimized dictionary, the sparse coefficients for the testing video feature samples are obtained, and then the classification result for video semantic is realized by reducing the error existing between the original and recreated samples. The experiment results show that the proposed DLSDDL technique considerably increases the efficiency of video semantic detection as against competing methods used in our experiment.     

Laplacian sparse dictionary learning for image classification based on sparse representation

Sparse representation is a mathematical model for data representation that has proved to be a powerful tool for solving problems in various fields such as pattern recognition, machine learning, and computer vision. As one of the building blocks of the sparse representation method, dictionary learning plays an important role in the minimization of the reconstruction error between the original signal and its sparse representation in the space of the learned dictionary. Although using training samples directly as dictionary bases can achieve good performance, the main drawback of this method is that it may result in a very large and inefficient dictionary due to noisy training instances. To obtain a smaller and more representative dictionary, in this paper, we propose an approach called Laplacian sparse dictionary (LSD) learning. Our method is based on manifold learning and double sparsity. We incorporate the Laplacian weighted graph in the sparse representation model and impose the l₁-norm sparsity on the dictionary. An LSD is a sparse overcomplete dictionary that can preserve the intrinsic structure of the data and learn a smaller dictionary for each class. The learned LSD can be easily integrated into a classification framework based on sparse representation. We compare the proposed method with other methods using three benchmark-controlled face image databases, Extended Yale B, ORL, and AR, and one uncontrolled person image dataset, i-LIDS-MA. Results show the advantages of the proposed LSD algorithm over state-of-the-art sparse representation based classification methods.     

Dictionary learning with the cosparse analysis model based on summation of blocked determinants as the sparseness measure

Dictionary learning is crucially important for sparse representation of signals. Most existing methods are based on the so called synthesis model, in which the dictionary is column redundant. This paper addresses the dictionary learning and sparse representation with the so-called analysis model. In this model, the analysis dictionary multiplying the signal can lead to a sparse outcome. Though it has been studied in the literature, there is still not an investigation in the context of dictionary learning for nonnegative signal representation, while the algorithms designed for general signal are found not sufficient when applied to the nonnegative signals. In this paper, for a more efficient dictionary learning, we propose a novel cost function that is termed as the summation of blocked determinants measure of sparseness (SBDMS). Based on this measure, a new analysis sparse model is derived, and an iterative sparseness maximization scheme is proposed to solve this model. In the scheme, the analysis sparse representation problem can be cast into row-to-row optimizations with respect to the analysis dictionary, and then the quadratic programming (QP) technique is used to optimize each row. Therefore, we present an algorithm for the dictionary learning and sparse representation for nonnegative signals. Numerical experiments on recovery of analysis dictionary show the effectiveness of the proposed method.     

Hyperspectral image super-resolution through clustering-based sparse representation

Promoting the spatial resolution of hyperspectral sensors is expected to improve computer vision tasks. However, due to the physical limitations of imaging sensors, the hyperspectral image is often of low spatial resolution. In this paper, we propose a new hyperspectral image super-resolution method from a low-resolution (LR) hyperspectral image and a high resolution (HR) multispectral image of the same scene. The reconstruction of HR hyperspectral image is formulated as a joint estimation of the hyperspectral dictionary and the sparse codes based on the spatial-spectral sparsity of the hyperspectral image. The hyperspectral dictionary is learned from the LR hyperspectral image. The sparse codes with respect to the learned dictionary are estimated from LR hyperspectral image and the corresponding HR multispectral image. To improve the accuracy, both spectral dictionary learning and sparse coefficients estimation exploit the spatial correlation of the HR hyperspectral image. Experiments show that the proposed method outperforms several state-of-art hyperspectral image super-resolution methods in objective quality metrics and visual performance.

     

基于稀疏表示全局字典学习的图像分类方法

针对传统的稀疏表示字典学习图像分类方法在大规模分布式环境下效率低下的问题,设计一种基于稀疏表示全局字典的图像学习方法。将传统的字典学习步骤分布到并行节点上,使用凸优化方法在节点上学习局部字典并实时更新全局字典,从而提高字典学习效率和大规模数据的分类效率。最后在MapReduce平台上进行并行化实验,结果显示该方法在不影响分类精度的情况下对大规模分布式数据的分类有明显的加速,可以更高效地运用于各种大规模图像分类任务中。