Graph-based relational reasoning in a latent space for skeleton-based action recognition

Motivated by the powerful capability of deep neural networks in feature learning, a new graph-based neural network is proposed to learn local and global relational information on skeleton sequences represented as spatio-temporal graphs (STGs). The pipeline of our network architecture consists of three main stages. As the first stage, spatial–temporal sub-graphs (sub-STGs) are projected into a latent space in which every point is represented as a linear subspace. The second stage is based on message passing to acquire the localized correlated features of the nodes in the latent space. The third stage relies on graph convolutional networks (GCNs) to reason the long-range spatio-temporal dependencies through a graph representation of the latent space. Finally, the average pooling layer and the softmax classifier are then employed to predict the action categories based on the extracted local and global correlations. We validate our model in terms of action recognition using three challenging datasets: the NTU RGB+D, Kinetics Motion, and SBU Kinect Interaction datasets. The experimental results demonstrate the effectiveness of our approach and show that our proposed model outperforms the state-of-the-art methods.     

Semi-supervised Robust Feature Selection with eq-Norm Graph for Multiclass Classification

Flexible manifold embedding (FME) is a semi-supervised dimension reduction framework.It has been extended into feature selection by using different loss functions and sparse regularization methods.However,these kind of methods used the quadratic form of graph embedding,thus the results are sensitive to noise and outliers.In this paper,we propose a general semi-supervised feature selection model that optimizes an eq-norm of FME to decrease the noise sensitivity.Compare to the fixed parameter model,the eq-norm graph brings flexibility to balance the manifold smoothness and the sensitivity to noise by tuning its parameter.We present an efficient iterative algorithm to solve the proposed eq-norm graph embedding based semi-supervised feature selection problem,and offer a rigorous convergence analysis.Experiments performed on typical image and speech emotion datasets demonstrate that our method is effective for the multiclass classification task,and outperforms the related state-of-the-art methods.     

基于稀疏贝叶斯的流形学习

针对当前监督学习算法在流形数据集上分类性能的缺陷,如分类精度低且稀疏性有限,本文在稀疏贝叶斯方法和流行正则化框架的基础上,提出一种稀疏流形学习算法（Manifold Learning Based on Sparse Bayesian Approach,MLSBA）.该算法是对稀疏贝叶斯模型的扩展,通过在模型的权值上定义稀疏流形先验,有效利用了样本数据的流形信息,提高了算法的分类准确率.在多种数据集上进行实验,结果表明：MLSBA不仅在流形数据集上取得良好的分类性能,而且在非流形数据集上效果也比较好;同时算法在两类数据集上均具有良好的稀疏性能.     

基于流形等距映射的矩阵信息几何检测方法

雷达目标检测常面临复杂的杂波特性,经典的检测方法通常适合于某些特定的场景,当检测背景发生变化时,其检测性能急剧下降。为有效提升不同杂波背景下的检测性能,提出一种基于流形等距映射(ISOMAP)的矩阵信息几何检测器。该方法首先将信号检测问题转化为矩阵流形上两点之间的区分性问题;然后基于样本数据和流形等距映射原理,自适应地学习出矩阵流形的投影变换矩阵,将矩阵流形变换为可区分的低维流形,最大程度地保持每一个矩阵与其邻域内矩阵之间几何距离大小,增强矩阵流形的可分性;最后利用仿真杂波和实测数据对算法进行验证。实验结果表明,相比于经典的检测方法,所提方法能有效提升目标检测性能。     

基于流形学习的三维空间数据网格剖分方法

 三维空间数据的三角网格剖分实质是寻找嵌入在三维空间中的二维流形,通过建立流形学习与网格剖分的本质联系,提出基于流形学习的空间数据网格剖分方法.依据流形学习的重构误差准则,实现三维空间数据的维数约简;对生成的二维数据按照Delaunay准则划分;将二维数据之间的拓扑关系映射到对应的三维数据点集.相对于其它数据降维方法,流形学习更能保持数据之间的本质联系,使重构的三角网格与物体表面拓扑差异性更小.实验表明,该方法对于非同胚于球物体的表面重建能够取得良好的效果.     

一种基于T-分布随机近邻嵌入的聚类集成方法

该文将T-分布随机近邻嵌入(TSNE)引入到聚类集成问题中,提出一种基于TSNE的聚类集成方法。首先通过TSNE最小化超图邻接矩阵的行对应的高维数据点与低维映射点分布之间的KL散度,使得高维空间结构在低维空间得以保持,然后在低维空间运行层次聚类算法获得最终的聚类结果。在基准数据集上的实验结果表明: TSNE能够提高层次聚类算法的聚类质量,该文方法获得了优于主流聚类集成方法的结果。     

Skeleton-Based Human Action Recognition via Screw Matrices

With the recent advent of low-cost acqui-sition depth cameras, extracting 3D body skeleton has be-come relatively easier, which significantly lighten many dif-ficulties in 2D videos including occlusions, shadows and background extraction, etc. Directly perceived features, for example points, lines and planes, can be easily ex-tracted from 3D videos such that we can employ rigid motions to represent skeletal motions in a geometric way. We apply screw matrices, acquired by using rotations and translations in 3D space, to model single and multi-body rigid motion. Since screw matrices are members of the special Euclidean group SE(3), an action can be repre-sented as a point on a Lie group, which is a differen-tiable manifold. Using Lie-algebraic properties of screw al-gebra, isomorphic to se(3), the classical algorithms of ma-chine learning in vector space can be expanded to man-ifold space. We evaluate our approached on three public 3D action datasets: MSR Action3D dataset, UCF Kinect dataset and Florence3D-Action Dataset. The experimental results show that our approaches either match or exceed state-of-the-art skeleton-based human action recognition approaches.     

一种基于多尺度实例学习的视频事件检测算法

现有的大多数视频事件检测方法首先从视频帧或视频快照中提取特征,然后对特征进行量化和汇集,进而为整个视频生成一个向量表示.最后的汇集步骤虽然简单高效,但是可能丢失时间局部信息,而这些信息对于确定长视频中事件发生的位置具有重要作用,从而削弱了事件检测的准确性.为此,本文首先将每个视频表示为多个“实例”,并将其定义为不同时间间隔的视频段.然后,针对每个视频的正实例比例已知和未知两种情况,提出基于多尺度实例学习的检测算法,在将实例标签看成隐藏潜在变量的同时推断出实例标签以及实例尺度的事件检测模型.最后,利用大规模视频事件数据集进行了全面的仿真实验,结果证明了本文算法具有显著的性能提升.此外,算法还可以确定视频中导致正检测的时间段的位置,进而对检测结果做出解释.     

基于图像块整体稀疏性与流形投影的压缩成像

如何利用自然图像本身固有的先验知识来提高重构图像质量是压缩成像系统的一个关键问题.本文在压缩成像系统中融合图像块整体稀疏性与流形特性,提出了一种高质量压缩成像算法.在该算法中,图像块由字典稀疏表示,同时还可由一组与图像块位于同一低维流形上的近邻点线性逼近,从而使稀疏重建信号分布在原始信号所处的流形附近.另外本文充分利用了图像中任意位置处图像块的稀疏性先验知识,使得压缩成像算法在低采样率下能重构出质量较高的图像.     

基于无监督特征选择的高光谱图像分类方法

在高光谱图像分类中,丰富的数据提升了其地物 识别能力。然而,由于样本特 征数大且有标记训练样本点少,导致“维度灾难”问题。本文提出一种基于无监督特征选择 的高光谱图像分类方 法,该方法同时考虑数据的流形嵌入映射和稀疏表达,将特征选择问题转化为一个优 化问题,数据的流形嵌入和稀疏表达作为约束项加入目标函数。设计了三个目标函 数,第一个目标函数描述流形学习的局部性原则,第二个目标函数将原始样本点回归 到低维嵌入空间,第三个目标函数对回归系数进行正则化。针对目标函数非凸的问 题,用迭代的方法来解这个约束优化问题,给出了解该优化问题的算法。优选特征用 于参与后续的分类识别任务。在真实的高光谱数据集上的实验表明,新方法能够提高 分类的精度。     

AND-OR tree-based mode selection for video coding

The conventional video coding approach is pragmatic in that researchers design different methods and compare their results to state-of-the-art methods. Although the H.264/AVC baseline is effective, there is a need to develop an abstract view of video coding, in the hope that new insights can be derived from the abstraction. In this paper, we propose a preliminary approach based on an AND-OR tree representation of video coding. We show that the H.264/AVC baseline can be represented as an AND-OR tree structure. Based on the AND-OR tree representation, we propose two video coding systems: one is a T+2D wavelet codec based on a motion-compensated temporal filtering (MCTF) lifting structure, and the other is the AND-OR tree implementation of the H.264/AVC baseline. We also compare the proposed systems’ coding performance in terms of the PSNR with that of H.264/AVC JM 16.2.     

Kernel collaboration representation-based manifold regularized model for unconstrained face recognition

Most recent researches have demonstrated the effectiveness of using kernel function into sparse representation and collaborative representation, which can overcome the problem of ignoring the nonlinear relationship of samples in face recognition and other classification problems. Considering the fact that space structure information (i.e., manifold structure or spatial consistence) can help a lot in robust sparse coding by nonlinear kernel metrics. In our paper, we present a kernel collaborative representation-based manifold regularized method, where we apply kernel collaborative representation with \({{\ell }_{2}}\)-regularization-based classifier and add spatial similarity structure to collaborative representation for benefiting classification accuracy. Meanwhile, the local binary patterns feature is used to increase discrimination of classifier and reduce the sensitivity to unconstrained case (i.e., occlusion or noise). So our method is a joint model of linear and nonlinear, local feature and distance metrics, kernel subspace structure and manifold structure. Experiments show that the proposed method outperforms several similar state-of-the-art methods in terms of accuracy and time cost.     

Video quality assessment using space–time slice mappings

We develop a full-reference (FR) video quality assessment framework that integrates analysis of space–time slices (STSs) with frame-based image quality measurement (IQA) to form a high-performance video quality predictor. The approach first arranges the reference and test video sequences into a space–time slice representation. To more comprehensively characterize space–time distortions, a collection of distortion-aware maps are computed on each reference–test video pair. These reference-distorted maps are then processed using a standard image quality model, such as peak signal-to-noise ratio (PSNR) or Structural Similarity (SSIM). A simple learned pooling strategy is used to combine the multiple IQA outputs to generate a final video quality score. This leads to an algorithm called Space–TimeSlice PSNR (STS-PSNR), which we thoroughly tested on three publicly available video quality assessment databases and found it to deliver significantly elevated performance relative to state-of-the-art video quality models. Source code for STS-PSNR is freely available at: http://live.ece.utexas.edu/research/Quality/STS-PSNR_release.zip.     

Newton type algorithm on Riemannian manifolds applied to robot vision, and suggestions for improvement of its performance

In visual servo control of a robot, we often encounter the structure-from-motion problem. To study the structure-from-motion problem, we are led to finding a minimum of a real valued function defined on a product Riemannian manifold, e.g., special orthogonal groups and unit sphere. To take advantage of its Riemannian structure, we consider the Newton algorithm on this manifold. In particular, we focus on improving the algorithm to be more robust and faster than the existing Newton algorithm on Riemannian manifolds. For this, we exploit the sparseness of the Hessian matrix and suggest how to choose the step size during the optimisation procedure, which can be considered as extensions of those for vector space optimisation algorithms.     

Joint interference alignment precoding based on the optimization algorithm on the Grassmannian manifold

To solve the interference problem between users of the multiuser MIMO system, we first transform the system sum rate maximization problem into joint optimization of interference signal power and useful signal power. On this basis, we propose a weighted interference alignment objective function, causing the system to obtain a higher sum rate by adjusting the weight with different signal-to-noise ratios. Then, we model the transmit subspace and the interference subspace on the Grassmannian manifold and propose joint interference alignment precoding based on the Grassmannian conjugacy gradient algorithm (GCGA-JIAP algorithm). In contrast to conventional interference alignment algorithms, our proposed algorithm can reduce the computational cost by transforming the constrained optimization of the complex Euclidean space into unconstrained optimization with the degenerate dimension on the Grassmannian manifold. Computer simulation shows that the proposed algorithm improves the convergence of the iterative optimization of the transmitter precoding matrix and the receiver postprocessing matrix and also improves the sum rate performance of the multiuser MIMO interference system.     

Tomographic reconstruction using an adaptive tetrahedral mesh defined by a point cloud

Medical images in nuclear medicine are commonly represented in three dimensions as a stack of two-dimensional images that are reconstructed from tomographic projections. Although natural and straightforward, this may not be an optimal visual representation for performing various diagnostic tasks. A method for three-dimensional (3-D) tomographic reconstruction is developed using a point cloud image representation. A point cloud is a set of points (nodes) in space, where each node of the point cloud is characterized by its position and intensity. The density of the nodes determines the local resolution allowing for the modeling of different parts of the image with different resolution. The reconstructed volume, which in general could be of any resolution, size, shape, and topology, is represented by a set of nonoverlapping tetrahedra defined by the nodes. The intensity at any point within the volume is defined by linearly interpolating inside a tetrahedron from the values at the four nodes that define the tetrahedron. This approach creates a continuous piecewise linear intensity over the reconstruction domain. The reconstruction provides a distinct multiresolution representation, which is designed to accurately and efficiently represent the 3-D image. The method is applicable to the acquisition of any tomographic geometry, such as parallel-, fan-, and cone-beam; and the reconstruction procedure can also model the physics of the image detection process. An efficient method for evaluating the system projection matrix is presented. The system matrix is used in an iterative algorithm to reconstruct both the intensity and location of the distribution of points in the point cloud. Examples of the reconstruction of projection data generated by computer simulations and projection data experimentally acquired using a Jaszczak cardiac torso phantom are presented. This work creates a framework for voxel-less multiresolution representation of images in nuclear medicine.     

Sparsity induced convex nonnegative matrix factorization algorithm with manifold regularization

To address problems that the effectiveness of feature learned from real noisy data by classical nonnegative matrix factorization method,a novel sparsity induced manifold regularized convex nonnegative matrix factorization algorithm (SGCNMF) was proposed.Based on manifold regularization,the L_2,1norm was introduced to the basis matrix of low dimensional subspace as sparse constraint.The multiplicative update rules were given and the convergence of the algorithm was analyzed.Clustering experiment was designed to verify the effectiveness of learned features within various of noisy environments.The empirical study based on K-means clustering shows that the sparse constraint reduces the representation of noisy features and the new method is better than the 8 similar algorithms with stronger robustness to a variable extent.     

Affine hull based target representation for visual tracking

Handling appearance variations is a challenging issue in visual tracking. Existing appearance models are usually built upon a linear combination of templates. With such kind of representation, accurate visual tracking is not desirable when heavy appearance variations are in presence. Under the framework of particle filtering, we propose a novel target representation for tracking. Namely, the target candidates are represented by affine combinations of a template set, which leads to better capability in describing unseen target appearances. Additionally, in order to adapt this representation to dynamic contexts across a video sequence, a novel template update scheme is presented. Different from conventional approaches, the scheme considers both the importance of one template to a target candidate in the current frame and the recentness of the template that is kept in the template set. Comprehensive experiments show that the proposed algorithm achieves superior performances in comparison with state-of-the-art works.