Accurate 3D action recognition using learning on the Grassmann manifold

In this paper we address the problem of modeling and analyzing human motion by focusing on 3D body skeletons. Particularly, our intent is to represent skeletal motion in a geometric and efficient way, leading to an accurate action–recognition system. Here an action is represented by a dynamical system whose observability matrix is characterized as an element of a Grassmann manifold. To formulate our learning algorithm, we propose two distinct ideas: (1) in the first one we perform classification using a Truncated Wrapped Gaussian model, one for each class in its own tangent space. (2) In the second one we propose a novel learning algorithm that uses a vector representation formed by concatenating local coordinates in tangent spaces associated with different classes and training a linear SVM. We evaluate our approaches on three public 3D action datasets: MSR-action 3D, UT-kinect and UCF-kinect datasets; these datasets represent different kinds of challenges and together help provide an exhaustive evaluation. The results show that our approaches either match or exceed state-of-the-art performance reaching 91.21% on MSR-action 3D, 97.91% on UCF-kinect, and 88.5% on UT-kinect. Finally, we evaluate the latency, i.e. the ability to recognize an action before its termination, of our approach and demonstrate improvements relative to other published approaches.     

View-based 3D model retrieval via supervised multi-view feature learning

With the development of the processing technologies of 3D model and the increasing of 3D model in different application flieds, 3D model retrieval is attracting more and more people’s attention. In order to handle this problem, most of approaches focus on the feature extraction form different virtual view. It is hard to guarantee the robustness and also ignore the correlation between both views. Thus, we propose an effective view-based 3D model retrieval method via supervised multi-view feature learning (SMFL). First, the subspace dimension of viusal feature is generated through Singular Value Decomposition (SVD) algorithm. This step is used to select main information from multi-view in order to reduce the final amount of calculation; Secondly, we consider the relationship of multi-view from same class and the correlation between two different classes to make the feature mapping in order to reduce the different of views from the same class and increase the different of views from the difference class; Finally, the projection mapping corresponding to the inner product of each 3D model helps to calculate the similarities between two different 3D models. The extensive experiments are conducted on popular ETH, NTU, MV-RED and PSB 3D model datasets with Zernike moments. The comparative results or The experimental results with existing 3D model retrieval methods show the superiority of the proposed method.     

Free-hand sketches for 3D model retrieval using cascaded LSDA

Multimedia Tools and Applications - This study proposes a novel cascaded 3D model retrieval framework for automatically and accurately fitting 3D models to user freehand 2D sketches (or object...     

Image retrieval using nonlinear manifold embedding

The huge number of images on the Web gives rise to the content-based image retrieval (CBIR) as the text-based search techniques cannot cater to the needs of precisely retrieving Web images. However, CBIR comes with a fundamental flaw: the semantic gap between high-level semantic concepts and low-level visual features. Consequently, relevance feedback is introduced into CBIR to learn the subjective needs of users. However, in practical applications the limited number of user feedbacks is usually overwhelmed by the large number of dimensionalities of the visual feature space. To address this issue, a novel semi-supervised learning method for dimensionality reduction, namely kernel maximum margin projection (KMMP) is proposed in this paper based on our previous work of maximum margin projection (MMP). Unlike traditional dimensionality reduction algorithms such as principal component analysis (PCA) and linear discriminant analysis (LDA), which only see the global Euclidean structure, KMMP is designed for discovering the local manifold structure. After projecting the images into a lower dimensional subspace, KMMP significantly improves the performance of image retrieval. The experimental results on Corel image database demonstrate the effectiveness of our proposed nonlinear algorithm.     

Sparse representation-based 3D model retrieval

Multimedia Tools and Applications - In this study, we leveraged the sparse representation for multi-modal information fusion to handle 3D model retrieval problem. First, SIFT feature is extracted...     

三维模型检索中的预处理技术

随着三维建模技术和数字化技术的发展,基于内容的三维模型检索技术应运而生.由于三维模型是一种自由度很高的信息,在特征提取之前对三维模型进行坐标系归一化预处理,从而保证所提取特征的平移不变性、比例不变性和旋转不变性,使得同样的三维模型在不同的坐标系下具有相同的相似度,这样可以提高基于内容的三维模型相似性比较算法的准确性.而预处理的效果对检索性能有着很大的影响,对PCA和各向异性变换这两种主要预处理技术进行了研究.     

Convolutional deep learning for 3D object retrieval

In recent years, with the development of 3D technologies, 3D model retrieval has become a hot topic. The key point of 3D model retrieval is to extract robust feature for 3D model representation. In order to improve the effectiveness of method on 3D model retrieval, this paper proposes a feature extraction model based on convolutional neural networks (CNN). First, we extract a set of 2D images from 3D model to represent each 3D object. SIFT detector is utilized to detect interesting points from each 2D image and extract interesting patches to represent local information of each 3D model. X-means is leveraged to generate the CNN filters. Second, a single CNN layer learns low-level features which are then given as inputs to multiple recursive neural networks (RNN) in order to compose higher order features. RNNs can generate the final feature for 2D image representation. Finally, nearest neighbor is used to compute the similarity between different 3D models in order to handle the retrieval problem. Extensive comparison experiments were on the popular ETH and MV-RED 3D model datasets. The results demonstrate the superiority of the proposed method.     

3D model retrieval using principal plane analysis and dynamic programming

Three dimensional models play an important role in many applications; the problem is how to select the appropriate models from a 3D database rapidly and accurately. In recent years, a variety of shape representations, statistical methods, and geometric algorithms have been proposed for matching 3D shapes or models. In this paper, we propose a 3D shape representation scheme based on a combination of principal plane analysis and dynamic programming. The proposed 3D shape representation scheme consists of three steps. First, a 3D model is transformed into a 2D image by projecting the vertices of the model onto its principal plane. Second, the convex hall of the 2D shape of the model is further segmented into multiple disjoint triangles using dynamic programming. Finally, for each triangle, a projection score histogram and moments are extracted as the feature vectors for similarity searching. Experimental results showed the robustness of the proposed scheme, which resists translation, rotation, scaling, noise, and destructive attacks. The proposed 3D model retrieval method performs fairly well in retrieving models having similar characteristics from a database of 3D models.     

基于相似性序列分析的三维模型检索性能评价

普林斯顿形状基准数据库已经成为当前三维模型检索研究人员使用最为广泛的基准数据库之一。但是该基准数据库存在着一个重要的缺陷,那便是模型类别之间的相似性没有被考虑,这导致在对检索结果进行性能评价时出现与人类直观感觉不一致的结论。为了克服这个缺陷,提出了一种新的基准数据库分类方案,并且在此基础上提出了一种基于相似性序列分析的检索性能评估方法。     

3D head model retrieval in kernel feature space using HSOM

In this paper, we propose a novel 3D head model retrieval framework. Specifically, to facilitate better classification and retrieval, the original 3D head model representations are embedded into another kernel feature space in which kernel principal component analysis (kernel PCA) is then performed to search for the optimal basis representation. Based on the extracted nonlinear features, a hierarchical indexing structure for 3D model retrieval is constructed using the hierarchical self organizing map (HSOM). The proposed indexing structure clusters the database into a hierarchy so that head models are partitioned by coarse features initially and then by finer scale features at lower levels. The main motivation of adopting this approach is that subspace technique like kernel PCA provides an elegant mechanism to describe the 3D head models on multiple resolutions based on the choices for reconstruction error and the orthogonal property of the produced eigenvectors. To further enhance the performance, a fuzzy metric between the query and the feature vector associated with each node on the SOMs is adopted instead of the usual Euclidean metric. Only nodes that possess high fuzzy measure values will be considered further for retrieval. In this way, the fuzzy measure approach is able to pick up potential relevant models even though they may be distributed across a number of neighbouring nodes. In addition to model categorization, the topology-preserving property of HSOM also facilitates the exploration of the model database with the possibility for further knowledge discovery. The effectiveness of the proposed approach is verified by a set of simulation examples on a 3D head model database.     

基于图像的三维扫描系统

本文新实现一种基于图像的三维扫描系统,各支撑杆根据高度分层设有相应扫描装置;当人体位于支撑杆中间时,通过通过放置一周的带有高清摄像头的树莓派同时拍照被测人后,合成被测者的人体三维模型,用作虚拟试衣和后续使用.本三维扫描系统成本低廉,克服了现有技术中扫描系统使用成本高的问题.     

利用骨架模型和格拉斯曼流形的3D人体动作识别

为了通过几何特征的有效方法描述人体骨骼运动,构建3D人体动作识别系统,提出一种基于3D骨骼关节空间建模方法。首先,使用自回归和移动平均模型（ARMA）描述每个随着时间变化的运动轨迹,成功捕捉了时空动态运动信息。同时,将该模型的观察矩阵生成的子空间作为格拉斯曼流形中一个点;然后,通过学习控制切线（CT）描述每个类的均值,映射学习过程中的观察变量到所有CT形成局部切丛（LTB）,LTB流形数据点可直接在分类器上完成分类;最后,提出的方法使用SVM分类器完成训练和分类。MSR-action 3D、Weizmann和UCF-Kinect三个数据库的实验结果验证了该方法的有效性,与几种基于深度数据的算法相比,该方法获得了最高的识别率,在延迟性方面的性能也表现最优,当帧数为30时,识别率达到97.91%,在延迟较高时,可达到期望识别率。     

基于电场通量特征的3D模型检索方法

对基于内容的三维模型检索的应用背景和现有算法进行了研究。结合物理学的基本原理,提出了一种静电场电通量描述子,在三维模型的模拟电场中设置测试球壳,计算各个测试球壳内部的电通量,用以描述三维模型。该描述子具有平移、旋转和缩放不变的特征,并对噪声有较强的鲁棒性。实验证明该描述子具有较强的检索能力。