ROSy+: 3D Object Pose Normalization Based on PCA and Reflective Object Symmetry with Application in 3D Object Retrieval

A novel pose normalization method based on 3D object reflective symmetry is presented. It is a general purpose global pose normalization method; in this paper it is used to enhance the performance of a 3D object retrieval pipeline. Initially, the axis-aligned minimum bounding box of a rigid 3D object is modified by requiring that the 3D object is also in minimum angular difference with respect to the normals to the faces of its bounding box. To estimate the modified axis-aligned bounding box, a set of predefined planes of symmetry are used and a combined spatial and angular distance, between the 3D object and its symmetric object, is calculated. By minimizing the combined distance, the 3D object fits inside its modified axis-aligned bounding box and alignment with the coordinate system is achieved. The proposed method is incorporated in a hybrid scheme, that serves as the alignment method in a 3D object retrieval system. The effectiveness of the 3D object retrieval system, using the hybrid pose normalization scheme, is evaluated in terms of retrieval accuracy and demonstrated using both quantitative and qualitative measures via an extensive consistent evaluation on standard benchmarks. The results clearly show performance boost against current approaches.     

利用对称性实现三维人脸姿态的自动估计*

描述了一个能够快速精确地对三维人脸姿态进行自动估计的系统,提出了利用人脸的反射对称特性自动估计三维人脸姿态的方法,通过扩展高斯图像及最小包围球来得到三维人脸对称平面,利用搜索得到的鼻尖顶点对人脸进行估计,然后对估计在规定范围内进行修正,最终得到精确的估计结果。以三维扫描仪扫描的真实人脸数据作为输入对系统进行了验证,实验表明该方法不但具有很好的精确性和鲁棒性,而且能够很好地应用到实际应用中。     

3D object retrieval via range image queries in a bag-of-visual-words context

3D object retrieval based on range image queries that represent partial views of real 3D objects is presented. The complete 3D models of the database are described by a set of panoramic views, and a Bag-of-Visual-Words model is built using SIFT features extracted from them. To address the problem of partial matching, we suggest a histogram computation scheme, on the panoramic views, that represents local information by taking into account spatial context. Furthermore, a number of optimization techniques are applied throughout the process for enhancing the retrieval performance. Its superior performance is shown by evaluating it against state-of-the-art methods on standard datasets.     

CM-BOF: visual similarity-based 3D shape retrieval using Clock Matching and Bag-of-Features

Content-based 3D object retrieval has become an active topic in many research communities. In this paper, we propose a novel visual similarity-based 3D shape retrieval method (CM-BOF) using Clock Matching and Bag-of-Features. Specifically, pose normalization is first applied to each object to generate its canonical pose, and then the normalized object is represented by a set of depth-buffer images captured on the vertices of a given geodesic sphere. Afterwards, each image is described as a word histogram obtained by the vector quantization of the image’s salient local features. Finally, an efficient multi-view shape matching scheme (i.e., Clock Matching) is employed to measure the dissimilarity between two models. When applying the CM-BOF method in non-rigid 3D shape retrieval, multidimensional scaling (MDS) should be utilized before pose normalization to calculate the canonical form for each object. This paper also investigates several critical issues for the CM-BOF method, including the influence of the number of views, codebook, training data, and distance function. Experimental results on five commonly used benchmarks demonstrate that: (1) In contrast to the traditional Bag-of-Features, the time-consuming clustering is not necessary for the codebook construction of the CM-BOF approach; (2) Our methods are superior or comparable to the state of the art in applications of both rigid and non-rigid 3D shape retrieval.     

3D Model Retrieval Using Bipartite Graph Matching Based on Attention

In this paper, we propose an attention-based bipartite graph 3D model retrieval algorithm, where many-to-many matching method, the weighted bipartite graph matching, is employed for comparison between two 3D models. Considering the panoramic views can donate the spatial and structural information, in this work, we use panoramic views to represent each 3D model. Attention mechanism is used to generate the weight of all views of each model. And then, we construct a weighted bipartite graph with the views of those models and the weight of each view. According to the bipartite graph, the matching result is used to measure the similarity between two 3D models. We experiment our method on ModelNet, NTU and ETH datasets, and the experimental results and comparison with other methods show the effectiveness of our method.

     

Efficient 3D reflection symmetry detection: A view-based approach

Symmetries exist in many 3D models while efficiently finding their symmetry planes is important and useful for many related applications. This paper presents a simple and efficient view-based reflection symmetry detection method based on the viewpoint entropy features of a set of sample views of a 3D model. Before symmetry detection, we align the 3D model based on the Continuous Principal Component Analysis (CPCA) method. To avoid the high computational load resulting from a directly combinatorial matching among the sample views, we develop a fast symmetry plane detection method by first generating a candidate symmetry plane based on a matching pair of sample views and then verifying whether the number of remaining matching pairs is within a minimum number. Experimental results and two related applications demonstrate better accuracy, efficiency, robustness and versatility of our algorithm than state-of-the-art approaches.     

一种基于人工免疫聚类的三维模型检索算法

为提高对三维模型检索的性能,通过借鉴生物免疫系统中的克隆选择原理和记忆机制,将一种具有更快收敛速度和更高收敛精度的人工免疫K-Means混合聚类算法应用于三维模型对称特性提取上,经过采样、计算得出三维模型的实际对称平面,以此作为特征信息进行三维模型的检索。对比实验表明,该方法可以获得较优的检索性能。     

快速的人脸轮廓检测及姿态估计算法

提出一种基于人脸特征区域划分的人脸轮廓检测方法和快速人脸姿态估计方法.该方法根据特征点在人脸的分布情况将人脸划分为9个区域.对于每个选定的区域,首先检测出其初始轮廓线,然后用三次多项式对其进行曲线拟合处理,最后把不同区域的轮廓线连接起来得到完整的人脸轮廓.此外,为了快速、准确地估计出人脸的姿态,本文从人脸的对称性出发,提出了进行人脸姿态估计的面积模型和近似平面模型.实验表明,本文所提出的轮廓检测方法对于复杂背景中具有不同姿态的人脸图像可以得到较满意的检测结果.和其它检测方法相比,本文方法具有模型简单、计算速度快等优点.     

Alignment of 3D models

In this paper we present a new method for alignment of 3D models. This approach is based on two types of symmetries of the models: the reflective symmetry and the local translational symmetry along a direction. Inspired by the work on the principal component analysis (PCA), we select the best optimal alignment axes within the PCA-axes, the plane reflection symmetry being used as a selection criterion. This pre-processing transforms the alignment problem into an indexing scheme based on the number of the retained PCA-axes. In order to capture the local translational symmetry of a shape along a direction, we introduce a new measure we call the local translational invariance cost (LTIC). The mirror planes of a model are also used to reduce the number of candidate coordinate frames when looking for the one which corresponds to the user’s perception. Experimental results show that the proposed method finds the rotation that best aligns a 3D mesh.     

Globally Optimal Point Set Registration by Joint Symmetry Plane Fitting

The present work proposes a solution to the challenging problem of registering two partial point sets of the same object with very limited overlap. We leverage the fact that most objects found in man-made environments contain a plane of symmetry. By reflecting the points of each set with respect to the plane of symmetry, we can largely increase the overlap between the sets and therefore boost the registration process. However, prior knowledge about the plane of symmetry is generally unavailable or at least very hard to find, especially with limited partial views. Finding this plane could strongly benefit from a prior alignment of the partial point sets. We solve this chicken-and-egg problem by jointly optimizing the relative pose and symmetry plane parameters. We present a globally optimal solver by employing the branch-and-bound paradigm and thereby demonstrate that joint symmetry plane fitting leads to a great improvement over the current state of the art in globally optimal point set registration for common objects. We conclude with an interesting application of our method to dense 3D reconstruction of scenes with repetitive objects.

     

Pose estimation for objects with planar surfaces using eigenimage and range data analysis

In this paper we present a novel method for estimating the object pose for 3D objects with well-defined planar surfaces. Specifically, we investigate the feasibility of estimating the object pose using an approach that combines the standard eigenspace analysis technique with range data analysis. In this sense, eigenspace analysis was employed to constrain one object rotation and reject surfaces that are not compatible with a model object. The remaining two object rotations are estimated by computing the normal to the surface from the range data. The proposed pose estimation scheme has been successfully applied to scenes defined by polyhedral objects and experimental results are reported.     

Multi-view 3D Human Pose Estimation in Complex Environment

We introduce a framework for unconstrained 3D human upper body pose estimation from multiple camera views in complex environment. Its main novelty lies in the integration of three components: single-frame pose recovery, temporal integration and model texture adaptation. Single-frame pose recovery consists of a hypothesis generation stage, in which candidate 3D poses are generated, based on probabilistic hierarchical shape matching in each camera view. In the subsequent hypothesis verification stage, the candidate 3D poses are re-projected into the other camera views and ranked according to a multi-view likelihood measure. Temporal integration consists of computing K-best trajectories combining a motion model and observations in a Viterbi-style maximum-likelihood approach. Poses that lie on the best trajectories are used to generate and adapt a texture model, which in turn enriches the shape likelihood measure used for pose recovery. The multiple trajectory hypotheses are used to generate pose predictions, augmenting the 3D pose candidates generated at the next time step.     

Subspace methods for retrieval of general 3D models

In statistical shape analysis, subspace methods such as PCA, ICA and NMF are commonplace, whereas they have not been adequately investigated for indexing and retrieval of generic 3D models. The main roadblock to the wider employment of these methods seems to be their sensitivity to alignment, itself an ambiguous task in the absence of common natural landmarks. We present a retrieval scheme based comparatively on three subspaces, PCA, ICA and NMF, extracted from the volumetric representations of 3D models. We find that the most propitious 3D distance transform leading to discriminative subspace features is the inverse distance transform. We mitigate the ambiguity of pose normalization with continuous PCA coupled with the use of all feasible axis labeling and reflections. The performance of the subspace-based retrieval methods on Princeton Shape Benchmark is on a par with the state-of-the-art methods.     

A central profile-based 3D face pose estimation

In this study, we develop a central profile-based 3D face pose estimation algorithm. The central profile is a unique curve on a 3D face surface that starts from forehead center, goes down through nose ridge, nose tip, mouth center, and ends at a chin tip. The points on the central profile are co-planar and belong to a symmetry plane that separates human face into two identical parts. The central profile is protrusive and has a certain length. Most importantly, the normal vectors of the central profile points are parallel to the symmetry plane. Based on the properties of the central profile, Hough transform is employed to determine the symmetry plane by invoking a voting procedure. An objective function is introduced in the parameter space to quantify the vote importance for face profile points and map the central profile to an accumulator cell with the maximal value. Subsequently, a nose model matching algorithm is used to detect nose tip on the central profile. A pitch angle estimation algorithm is also proposed. The pose estimation experiments completed for a synthetic 3D face model and the FRGC v2.0 3D database demonstrate the effectiveness of the proposed pose estimation algorithm. The obtained central profile detection rate is 99.9%, and the nose tip detection rate has reached 98.16% with error not larger than 10 mm.     

3D object pose form clustering with multiple views

A candidate pose algorithm is described which computes object pose from an assumed correspondence between a pair of 2D image points and a pair of 3D model points. By computing many pose candidates actual object pose can usually be determined by detecting a cluster in the space of all candidates. Cluster space can receive candidate pose parameters from independent computations in different camera views. It is shown that use of of geometric constraint can be sufficient for reliable pose detection, but use of other knowledge, such as edge presence and type, can be easily added for increased efficiency.     

3D model alignment based on minimum projection area

3D model alignment is an important step for applications such as 3D model retrieval and 3D model recognition. In this paper, we propose a novel Minimum Projection Area-based (MPA) alignment method for pose normalization. Our method finds three principal axes to align a model: the first principal axis gives the minimum projection area when we perform an orthographic projection of the model in the direction parallel to this axis, the second axis is perpendicular to the first axis and gives the minimum projection area, and the third axis is the cross product of the first two axes. We devise an optimization method based on Particle Swarm Optimization to efficiently find the axis with minimum projection area. For application in retrieval, we further perform axis ordering and orientation in order to align similar models in similar poses. We have tested MPA on several standard databases which include rigid/non-rigid and open/watertight models. Experimental results demonstrate that MPA has a good performance in finding alignment axes which are parallel to the ideal canonical coordinate frame of models and aligning similar models in similar poses under different conditions such as model variations, noise, and initial poses. In addition, it achieves a better 3D model retrieval performance than several commonly used approaches such as CPCA, NPCA, and PCA.