A Parameter-Free Framework for General Supervised Subspace Learning

Supervised subspace learning techniques have been extensively studied in biometrics literature; however, there is little work dedicated to: 1) how to automatically determine the subspace dimension in the context of supervised learning, and 2) how to explicitly guarantee the classification performance on a training set. In this paper, by following our previous work on unified subspace learning framework in our earlier work, we present a general framework, called parameter-free graph embedding (PFGE) to solve the above two problems by posing a general supervised subspace learning task as a semidefinite programming problem. The semipositive feature Gram matrix, namely the product of the transformation matrix and its transpose, is derived by optimizing a trace difference form of an objective function extended from that in our earlier work with the constraints that guarantee the class homogeneity within the neighborhood of each datum. Then, the subspace dimension and the feature weights are simultaneously obtained via the singular value decomposition of the feature Gram matrix. In addition, to alleviate the computational complexity, the Kronecker product approximation of the feature Gram matrix is proposed by taking advantage of the essential matrix form of image pixels. The experiments on simulated data and real-world data demonstrate the capability of the new PFGE framework in estimating the subspace dimension for supervised learning as well as the superiority in classification performance over traditional algorithms for subspace learning     

基于子空间自适应学习的粒子滤波跟踪算法

为了提高目标外观迅速变化时视觉跟踪算法的鲁棒性,提出了一种基于自适应子空间学习的粒子滤波跟踪算法。在粒子滤波构架下,建立状态判决机制,根据判决结果并结合主成分分析(PCA)子空间与正交子空间的特点,选择合适的学习方法。这样既能准确、稳定地学习到目标的低维子空间,又能迅速地学习到目标外观变化的趋势。同时,加入鲁棒估计技术处理遮挡问题,避免了对目标状态估计的影响。实验结果表明,该算法在光照变化、姿态变化、遮挡的情况下,均具有较强的鲁棒性。     

基于图信息的自监督多视角子空间聚类

多视角子空间聚类方法通常用于处理高维度、复杂结构的数据.现有的大多数多视角子空间聚类方法通过挖掘潜在图信息进行数据分析与处理,但缺乏对潜在子空间表示的监督过程.针对这一问题,本文提出一种新的多视角子空间聚类方法,即基于图信息的自监督多视角子空间聚类(SMSC).它将谱聚类与子空间表示相结合形成统一的深度学习框架.SMS...     

Subspace identification with non-steady Kalman filter parameterization

Most existing subspace identification methods use steady-state Kalman filter (SKF) in parameterization, hence, infinite data horizons are implicitly assumed to allow the Kalman gain to reach steady state. However, using infinite horizons requires collecting infinite data which is unrealistic in practice. In this paper, a subspace framework with non-steady state Kalman filter (NKF) parameterization is established to provide exact parameterization for finite data horizon identification problems. Based on this we propose a novel subspace identification method with NKF parameterization which can handle closed-loop data and avoid assumption on infinite horizons. It is shown that with finite data, the proposed parameterization method provides more accurate and consistent solutions than existing SKF based methods. The paper also reveals why it is often beneficial in practice to estimate a bank of ARX models over a single ARX model.     

多源适应多标签分类框架

多源适应学习是一种旨在提升目标学习性能的有效机器学习方法。针对多标签视觉分类问题,基于现有的研究进展,研究提出一种新颖的联合特征选择和共享特征子空间学习的多源适应多标签分类框架,在现有的图Laplacian正则化半监督学习范式中充分考虑目标视觉特征的优化处理,多标签相关信息在共享特征子空间的嵌入,以及多个相关领域的判别信息桥接利用等多个方面,并将其融为一个统一的学习模型,理论证明了其局部最优解只需通过求解一个广义特征分解问题便可分别获得,并给出了算法实现及其收敛性定理。在两个实际的多标签视觉数据分类上分别进行深入实验分析,证实了所提框架的鲁棒有效性和优于现有相关方法的分类性能。     

Target detection based on a dynamic subspace

For hyperspectral target detection, it is usually the case that only part of the targets pixels can be used as target signatures, so can we use them to construct the most proper background subspace for detecting all the probable targets? In this paper, a dynamic subspace detection (DSD) method which establishes a multiple detection framework is proposed. In each detection procedure, blocks of pixels are calculated by the random selection and the succeeding detection performance distribution analysis. Manifold analysis is further used to eliminate the probable anomalous pixels and purify the subspace datasets, and the remaining pixels construct the subspace for each detection procedure. The final detection results are then enhanced by the fusion of target occurrence frequencies in all the detection procedures. Experiments with both synthetic and real hyperspectral images (HSI) evaluate the validation of our proposed DSD method by using several different state-of-the-art methods as the basic detectors. With several other single detectors and multiple detection methods as comparable methods, improved receiver operating characteristic curves and better separability between targets and backgrounds by the DSD methods are illustrated. The DSD methods also perform well with the covariance-based detectors, showing their efficiency in selecting covariance information for detection.     

Kernelized multi-view subspace clustering via auto-weighted graph learning

Multi-view subspace clustering has been an important and powerful tool for partitioning multi-view data, especially multi-view high-dimensional data. Despite great success, most of the existing multi-view subspace clustering methods still suffer from three limitations. First, they often recover the subspace structure in the original space, which can not guarantee the robustness when handling multi-view data with nonlinear structure. Second, these methods mostly regard subspace clustering and affinity matrix learning as two independent steps, which may not well discover the latent relationships among data samples. Third, many of them ignore the different importance of multiple views, whose performance may be badly affected by the low-quality views in multi-view data. To overcome these three limitations, this paper develops a novel subspace clustering method for multi-view data, termed Kernelized Multi-view Subspace Clustering via Auto-weighted Graph Learning (KMSC-AGL). Specifically, the proposed method implicitly maps the multi-view data from linear space into nonlinear space via kernel-induced functions, so as to exploit the nonlinear structure hidden in data. Furthermore, our method aims to enhance the clustering performance by learning a set of view-specific representations and their affinity matrix in a general framework. By integrating the view weighting strategy into this framework, our method can automatically assign the weights to different views, while learning an optimal affinity matrix that is well-adapted to the subsequent spectral clustering. Extensive experiments are conducted on a variety of multi-view data sets, which have demonstrated the superiority of the proposed method.

     

Incremental Tensor Subspace Learning and Its Applications to Foreground Segmentation and Tracking

Appearance modeling is very important for background modeling and object tracking. Subspace learning-based algorithms have been used to model the appearances of objects or scenes. Current vector subspace-based algorithms cannot effectively represent spatial correlations between pixel values. Current tensor subspace-based algorithms construct an offline representation of image ensembles, and current online tensor subspace learning algorithms cannot be applied to background modeling and object tracking. In this paper, we propose an online tensor subspace learning algorithm which models appearance changes by incrementally learning a tensor subspace representation through adaptively updating the sample mean and an eigenbasis for each unfolding matrix of the tensor. The proposed incremental tensor subspace learning algorithm is applied to foreground segmentation and object tracking for grayscale and color image sequences. The new background models capture the intrinsic spatiotemporal characteristics of scenes. The new tracking algorithm captures the appearance characteristics of an object during tracking and uses a particle filter to estimate the optimal object state. Experimental evaluations against state-of-the-art algorithms demonstrate the promise and effectiveness of the proposed incremental tensor subspace learning algorithm, and its applications to foreground segmentation and object tracking.     

A unified perspective and new results on RHT computing, mixture based learning, and multi-learner based problem solving

On one hand, multiple object detection approaches of Hough transform (HT) type and randomized HT type have been extended into an evidence accumulation featured general framework for problem solving, with five key mechanisms elaborated and several extensions of HT and RHT presented. On the other hand, another framework is proposed to integrate typical multi-learner based approaches for problem solving, particularly on Gaussian mixture based data clustering and local subspace learning, multi-sets mixture based object detection and motion estimation, and multi-agent coordinated problem solving. Typical learning algorithms, especially those that base on rival penalized competitive learning (RPCL) and Bayesian Ying-Yang (BYY) learning, are summarized from a unified perspective with new extensions. Furthermore, the two different frameworks are not only examined with one viewed crossly from a perspective of the other, with new insights and extensions, but also further unified into a general problem solving paradigm that consists of five basic mechanisms in terms of acquisition, allocation, amalgamation, admission, and affirmation, or shortly A5 paradigm.     

基于双重低秩分解的不完整多视图子空间学习

多视图数据在现实世界中应用广泛,各种视角和不同的传感器有助于更好的数据表示,然而,来自不同视图的数据具有较大的差异,尤其当多视图数据不完整时,可能导致训练效果较差甚至失败。为了解决该问题,本文提出了一个基于双重低秩分解的不完整多视图子空间学习算法。所提算法通过两方面来解决不完整多视图问题:一方面,基于双重低秩分解子空间框架,引入潜在因子来挖掘多视图数据中缺失的信息;另一方面,通过预先学习的多视图数据低维特征获得更好的鲁棒性,并以有监督的方式来指导双重低秩分解。实验结果证明,所提算法较之前的多视图子空间学习算法有明显优势;即使对于不完整的多视图数据,该算法也具有良好的分类性能。     

A game theoretic fault detection filter

The fault detection process is approximated with a disturbance attenuation problem. The solution to this problem, for both linear time-varying and time-invariant systems, leads to a game theoretic filter which bounds the transmission of all exogenous signals except the fault to be detected. In the limit, when the disturbance attenuation bound is brought to zero, a complete transmission block is achieved by embedding the nuisance inputs into an unobservable, invariant subspace. Since this is the same invariant subspace structure seen in some types of detection filters, we can claim that the asymptotic game filter is itself a detection filter. One can also make use of this subspace structure to reduce the order of the limiting game theoretic filter by factoring this invariant subspace out of the state space. The resulting lower dimensional filter will then be sensitive only to the failure to be detected. A pair of examples given at the end of the paper demonstrate the effectiveness of the filter for time-invariant and time-varying problems in both full-order and reduced-order forms     

基于子空间和直方图的多记忆自适应相关滤波目标跟踪算法

为了增强相关滤波算法(CF)在目标遮挡或背景干扰情况下跟踪的鲁棒性,提出基于子空间和直方图的多记忆自适应相关滤波目标跟踪算法.首先,针对CF使用的模板单一无法应对不同时期相邻帧目标表现的差异,提出利用随机更新策略学习多个目标模板,应对不同时期的目标变化.然后,针对不同的更新模板得到多个候选目标,利用子空间学习上一帧的表示系数,综合判断候选目标的准确性.同时,因为CF与子空间表示均利用模板判断跟踪结果,对背景杂乱等情况判断容易造成偏差,所以引入颜色直方图,利用统计特征作为独立的判断依据,增强算法对候选目标判断结果的准确性.在标准视频集上的实验表明,文中算法具备一定的抗遮挡及抗背景干扰能力.     

Analysis of the Kalman filter based estimation algorithm: an orthogonal decomposition approach

In this paper we shall provide new analysis on some fundamental properties of the Kalman filter based parameter estimation algorithms using an orthogonal decomposition approach based on the excited subspace. A theoretical analytical framework is established based on the decomposition of the covariance matrix, which appears to be very useful and effective in the analysis of a parameter estimation algorithm with the existence of an unexcited subspace. The sufficient and necessary condition for the boundedness of the covariance matrix in the Kalman filter is established. The idea of directional tracking is proposed to develop a new class of algorithms to overcome the windup problem. Based on the orthogonal decomposition approach two kinds of directional tracking algorithms are proposed. These algorithms utilize a time-varying covariance matrix and can keep stable even in the case of unsufficient and/or unbounded excitation.     

面向特征的ICA和HP滤波实现视频异常事件检测

针对室外环境下光照亮度变化、阴影和树木遮挡等问题,对利用隐马尔可夫模型进行视频异常事件检测的影响,提出基于独立分量分析（ICA）和HP（Hodrick-Prescott）滤波器的隐马尔可夫模型视频异常事件检测方法。该方法首先利用ICA构造正常视频的特征子空间,将图像序列投影到特征子空间上得到投影序列,实现数据降维;然后利用HP滤波器滤除投影序列中环境变化引起的趋势分量;最终克服不利的环境因素,有效改善隐马尔可夫模型的视频异常事件检测性能。机动车辆禁行路段视频的检测实验表明,该方法能够在复杂的室外环境下较好地检测出异常事件。     

Learning multi-kernel multi-view canonical correlations for image recognition

Computational Visual Media - canonical correlations (M2CCs) framework for subspace learning. In the proposed framework, the input data of each original view are mapped into multiple higher...     

Towards learning a semantic-consistent subspace for cross-modal retrieval

A great many of approaches have been developed for cross-modal retrieval, among which subspace learning based ones dominate the landscape. Concerning whether using the semantic label information or not, subspace learning based approaches can be categorized into two paradigms, unsupervised and supervised. However, for multi-label cross-modal retrieval, supervised approaches just simply exploit multi-label information towards a discriminative subspace, without considering the correlations between multiple labels shared by multi-modalities, which often leads to an unsatisfactory retrieval performance. To address this issue, in this paper we propose a general framework, which jointly incorporates semantic correlations into subspace learning for multi-label cross-modal retrieval. By introducing the HSIC-based regularization term, the correlation information among multiple labels can be not only leveraged but also the consistency between the modality similarity from each modality is well preserved. Besides, based on the semantic-consistency projection, the semantic gap between the low-level feature space of each modality and the shared high-level semantic space can be balanced by a mid-level consistent one, where multi-label cross-modal retrieval can be performed effectively and efficiently. To solve the optimization problem, an effective iterative algorithm is designed, along with its convergence analysis theoretically and experimentally. Experimental results on real-world datasets have shown the superiority of the proposed method over several existing cross-modal subspace learning methods.

     

量子谱回归算法

子空间学习是机器学习领域的重要研究方向.为了降低子空间学习的复杂度,Cai等人提出了谱回归降维框架,并针对结合标签构造对应图的子空间学习提出了高效谱回归.近年来,量子计算的发展使进一步降低子空间学习算法的复杂度成为了可能.Meng等人率先提出了量子谱回归算法(MYXZ算法).MYXZ算法用了稀疏哈密顿量模拟技术来处理由权重矩阵生成的矩阵,但这个矩阵在较多的情况下是稠密矩阵.针对这种情况,指出了MYXZ算法的局限性,提出了一个改进的量子谱回归算法.改进算法采用了量子奇异值估计技术,在处理稠密矩阵时相对MYXZ算法有多项式加速.另外,提出了一个新的量子算法,对经典的高效谱回归进行加速.新算法能处理的这类问题是MYXZ算法无法处理的.新算法利用了量子岭回归和量子矩阵向量乘技术,在相同的参数条件下相对经典算法具有多项式加速效果.     

Adaptive subspace learning: an iterative approach for document clustering

The performance of clustering in document space can be influenced by the high dimension of the vectors, because there exists a great deal of redundant information in the high-dimensional vectors, which may make the similarity between vectors inaccurate. Hence, it is very considerable to derive a low-dimensional subspace that contains less redundant information, so that document vectors can be grouped more reasonably. In general, learning a subspace and clustering vectors are treated as two independent steps; in this case, we cannot estimate whether the subspace is appropriate for the method of clustering or vice versa. To overcome this drawback, this paper combines subspace learning and clustering into an iterative procedure named adaptive subspace learning (ASL). Firstly, the intracluster similarity and the intercluster separability of vectors can be increased via the initial cluster indicators in the step of subspace learning, and then affinity propagation is adopted to partition the vectors into a specific number of clusters, so as to update the cluster indicators and repeat subspace learning. In ASL, the obtained subspace can become more suitable for the clustering with the iterative optimization. The proposed method is evaluated using NG20, Classic3 and K1b datasets, and the results are shown to be superior to the conventional methods of document clustering.     

引入低秩约束先验的深度子空间聚类

大多数子空间聚类算法将高维数据映射到低维子空间时不能较好捕获数据间几何结构.针对上述问题,文中提出引入低秩约束先验的深度子空间聚类算法,兼顾数据全局和局部结构信息.算法结合低秩表示与深度自编码器,利用低秩约束捕获数据全局结构,并将约束神经网络的潜在特征表示为低秩.自编码通过最小化重构误差进行非线性低维子空间映射,保留数据的局部特性.以多元逻辑回归函数作为判别模型,预测子空间分割.整个算法在无监督联合学习框架下进行优化.在5个数据集上的实验验证文中方法的有效性.