A differential-geometric approach for 2D and 3D object grasping and manipulation

This article presents an expository work on a differential-geometric treatment of fundamental problems of 2D and 3D object grasping and manipulation by a pair of robot fingers with multi-joints under holonomic or nonholonomic constraints. First, Lagrange’s equation of motion of a fingers-object system whose motion is confined to a vertical plane is derived under holonomic constraints when rolling contacts between finger-ends and object surfaces are permitted. Then, a class of control signals called “blind grasping” and constructed without knowing the object kinematics or using any external sensing like vision or tactile sensation is shown to realize stable object grasping in a dynamic sense. Stability of motion and its convergence to an equibrium manifold are treated on the basis of differential geometry of solution trajectories of the closed-loop dynamics on the constraint manifolds. Second, a mathematical model of 3D object grasping and manipulation by a pair of multi-joint robot fingers is derived under the assumption that spinning motion of rotation around the opposing axis between contact points does no more arise. It is shown that, differently from the 2D case, the instantaneous axis of rotation of the object is time-varying, which induces a nonholonomic constraint expressed as a linear differential equation of rotational motion of the pinched object. It is shown that there is a class of control signals constructed without knowing the object kinematics or using external sensings that can realize “blind grasping” in a dynamic sense. Finally, it is shown that the proposed differential geometric treatment of stability can naturally cope with redundancy resolution problems of surplus degrees-of-freedom (d.f.) of the overall fingers-object system, which is closely related to Bernstein’s d.f. problem.     

Human shape recognition performance for 3D tactile display

The paper describes the relationship between the pin-matrix density of a tactile display and the recognition performance of displayed 3D shapes. Three types of pin-matrix tactile display, that generate 3D shapes, were used for the experiment. The pitch of pins was 2 mm, 3 mm, 5 mm each. We assumed that surfaces, edges, and vertices were primitive 3D shape information, so tested shapes were classified into these three categories. We assumed two types of finger touching mode: 1) fingertip-only, allowed full use of spatial shape information given to the fingertip; and 2) allowed tracing of the object. Recognition time and the classified error rate were measured. We obtained results on the relationship between pin pitch and recognition performance data. Regression curves for pin pitch and recognition time were plotted. A significance test of recognition time versus pin pitch was done. The error rate of identification versus pin pitch was described. Our results provide basic knowledge for developing tactile presentation devices     

Biometric recognition using 3D ear shape

Previous works have shown that the ear is a promising candidate for biometric identification. However, in prior work, the preprocessing of ear images has had manual steps and algorithms have not necessarily handled problems caused by hair and earrings. We present a complete system for ear biometrics, including automated segmentation of the ear in a profile view image and 3D shape matching for recognition. We evaluated this system with the largest experimental study to date in ear biometrics, achieving a rank-one recognition rate of 97.8 percent for an identification scenario and an equal error rate of 1.2 percent for a verification scenario on a database of 415 subjects and 1,386 total probes.     

Synthesized affine invariant function for 2D shape recognition

By defining the weighted wavelet synthesis, the synthesized feature signals of an interesting shape are extracted to derive the innovative synthesized affine invariant function (SAIF). The synthesized feature signals hold the shape information with minimum loss by excluding simply the translation dependent and noise-contaminated bands. The SAIF is shown excellent in the invariance property and representative in describing the original shape for automated recognition. Experimental results demonstrate that automated shape recognition based on the SAIF achieves high correctness and significantly outperforms those using conventional wavelet affine invariant functions.     

3D shape recognition based on multi-modal information fusion

The classification and retrieval of 3D models have been widely used in the field of multimedia and computer vision. With the rapid development of computer graphics, different algorithms corresponding to different representations of 3D models have achieved the best performance. The advances in deep learning also encourage various deep models for 3D feature representation. For multi-view, point cloud, and PANORAMA-view, different models have shown significant performance on 3D shape classification. However, There’s not a way to consider utilizing the fusion information of multi-modal for 3D shape classification. In our opinion, We propose a novel multi-modal information fusion method for 3D shape classification, which can fully utilize the advantage of different modal to predict the label of class. More specifically, the proposed can effectively fuse more modal information. it is easy to utilize in other similar applications. We have evaluated our framework on the popular dataset ModelNet40 for the classification task on 3D shape. Series experimental results and comparisons with state-of-the-art methods demonstrate the validity of our approach.

     

A dyadic wavelet affine invariant function for 2D shape recognition

Dyadic wavelet transform has been used to derive an affine invariant function. First, an invariant function using two dyadic levels is derived. Then, this invariant function is used to derive another invariant function using six dyadic levels. We introduce the wavelet based conic equation. The invariant function is based on analyzing the object boundary using the dyadic wavelet transform. Experimental results on both synthetic and real data are used to demonstrate the discriminating power of the proposed invariant function. It has also been compared with some traditional methods. The stability of the proposed invariant function is examined. In addition, the stability under large perspective transformation is tested     

3D palmprint recognition using shape index representation and fragile bits

Recent years have witnessed a growing interesting in developing automatic palmprint recognition methods. Most of the previous works have concentrated on two dimensional (2D) palmprint recognition in the past decade. However, the shape information is lost in 2D plamprint images. What’s more, 2D plamprint recognition is not robust enough in practice since its data could be easily counterfeited or contaminated by noise. Consequently, three dimensional (3D) palmprint recognition is treated as an important alternative road to both enhance the performance and robustness of current available palmprint recognition systems. In this paper, we first explore geometrical information of 3D palmprint data by employing shape index formulation, from which Gabor wavelet features are then extracted. Furthermore, we first discover that by incorporating fragile bits information, the performance of coding strategy related 3D recognition method can be further improved. Experiments conducted on the public available 3D plamprint database validate that our method can obtain the highest recognition performance among the state-of-the-art methods estimated.     

Applications of approximate string matching to 2D shape recognition

A new method for the recognition of arbitrary two-dimensional (2D) shapes is described. It is based on string edit distance computation. The recognition method is invariant under translation, rotation, scaling and partial occlusion. A set of experiments are described demonstrating the robustness and reliability of the proposed approach.     

Instance-based object recognition in 3D point clouds using discriminative shape primitives

3D local shapes are a critical cue for object recognition in 3D point clouds. This paper presents an instance-based 3D object recognition method via informative and discriminative shape primitives. We propose a shape primitive model that measures geometrical informativity and discriminativity of 3D local shapes of an object. Discriminative shape primitives of the object are extracted automatically by model parameter optimization. We achieve object recognition from 2.5/3D scenes via shape primitive classification and recover the 3D poses of the identified objects simultaneously. The effectiveness and the robustness of the proposed method were verified on popular instance-based 3D object recognition datasets. The experimental results show that the proposed method outperforms some existing instance-based 3D object recognition pipelines in the presence of noise, varying resolutions, clutter and occlusion.     

结合三维几何形状信息和二维纹理的3D目标匹配

为了实现场景中三维目标与模型之间的匹配,提出了一种结合三维几何形状信息和二维纹理的三维目标匹配方法。首先提取场景中深度图像的尺度不变特征变换(SIFT)特征,用SIFT算法与三维模型重建时所用到的一系列2.5维深度图像进行一一匹配,找到与场景中目标姿态最为相似的深度图像,提取此深度图像的三维几何形状特征与模型进行匹配,实现模型的初始化,即将模型重置到与场景目标相接近的姿态。最后用融合二维纹理信息的迭代就近点（ICP）算法实现场景中目标与模型之间的匹配,从而得到场景中三维目标的准确姿态。实验结果验证了方法的可行性与精确性。     

3D foot shape generation from 2D information

Two methods to generate an individual 3D foot shape from 2D information are proposed. A standard foot shape was first generated and then scaled based on known 2D information. In the first method, the foot outline and the foot height were used, and in the second, the foot outline and the foot profile were used. The models were developed using 40 participants and then validated using a different set of 40 participants. Results show that each individual foot shape can be predicted within a mean absolute error of 1.36 mm for the left foot and 1.37 mm for the right foot using the first method, and within a mean absolute error of 1.02 mm for the left foot and 1.02 mm for the right foot using the second method. The second method shows somewhat improved accuracy even though it requires two images. Both the methods are relatively cheaper than using a scanner to determine the 3D foot shape for custom footwear design.     

2D and 3D shape retrieval using skeleton filling rate

As an increasing number of digital images are generated, a demand for an efficient and effective image retrieval mechanisms grows. In this work, we present a new skeleton-based algorithm for 2D and 3D shape retrieval. The algorithm starts by drawing circles (spheres for 3D) of increasing radius around skeletons. Since each skeleton corresponds to the center of a maximally inscribed circle (sphere), this process results in circles (spheres) that are partially inside the shape. Computing the ratio between pixels that lie within the shape and the total number of pixels allows us to distinguish shapes with similar skeletons. Experimental evaluation of the proposed approach including a comprehensive comparison with the previous techniques demonstrates both effectiveness and robustness of our algorithm for shape retrieval using several 2D and 3D datasets.     

Distance metric between 3D models and 2D images for recognition andclassification

Similarity measurements between 3D objects and 2D images are useful for the tasks of object recognition and classification. The authors distinguish between two types of similarity metrics: metrics computed in image-space (image metrics) and metrics computed in transformation-space (transformation metrics). Existing methods typically use image metrics; namely, metrics that measure the difference in the image between the observed image and the nearest view of the object. Example for such a measure is the Euclidean distance between feature points in the image and their corresponding points in the nearest view. (This measure can be computed by solving the exterior orientation calibration problem.) In this paper the authors introduce a different type of metrics: transformation metrics. These metrics penalize for the deformations applied to the object to produce the observed image. In particular, the authors define a transformation metric that optimally penalizes for “affine deformations” under weak-perspective. A closed-form solution, together with the nearest view according to this metric, are derived. The metric is shown to be equivalent to the Euclidean image metric, in the sense that they bound each other from both above and below. It therefore provides an easy-to-use closed-form approximation for the commonly-used least-squares distance between models and images. The authors demonstrate an image understanding application, where the true dimensions of a photographed battery charger are estimated by minimizing the transformation metric     

人脸识别:从二维到三维

人脸识别是生物特征识别技术的一个重要方向。虽然目前大部分研究都还只是针对二维人脸图像,但是3D人脸模型包含更丰富的人脸信息,有助于机器对人脸的识别。从二维到三维,人脸识别研究进入了一个新的阶段。从3D人脸数据的获取方式入手,介绍最近提出的一系列3D人脸识别算法,并进行归类。最后提出"有针对性地获取3D人脸模型数据是进行有效识别的基础"这一结论。     

2D face fitting-assisted 3D face reconstruction for pose-robust face recognition

Recent face recognition algorithm can achieve high accuracy when the tested face samples are frontal. However, when the face pose changes largely, the performance of existing methods drop drastically. Efforts on pose-robust face recognition are highly desirable, especially when each face class has only one frontal training sample. In this study, we propose a 2D face fitting-assisted 3D face reconstruction algorithm that aims at recognizing faces of different poses when each face class has only one frontal training sample. For each frontal training sample, a 3D face is reconstructed by optimizing the parameters of 3D morphable model (3DMM). By rotating the reconstructed 3D face to different views, pose virtual face images are generated to enlarge the training set of face recognition. Different from the conventional 3D face reconstruction methods, the proposed algorithm utilizes automatic 2D face fitting to assist 3D face reconstruction. We automatically locate 88 sparse points of the frontal face by 2D face-fitting algorithm. Such 2D face-fitting algorithm is so-called Random Forest Embedded Active Shape Model, which embeds random forest learning into the framework of Active Shape Model. Results of 2D face fitting are added to the 3D face reconstruction objective function as shape constraints. The optimization objective energy function takes not only image intensity, but also 2D fitting results into account. Shape and texture parameters of 3DMM are thus estimated by fitting the 3DMM to the 2D frontal face sample, which is a non-linear optimization problem. We experiment the proposed method on the publicly available CMUPIE database, which includes faces viewed from 11 different poses, and the results show that the proposed method is effective and the face recognition results toward pose variants are promising.     

2D representation of facial surfaces for multi-pose 3D face recognition

The increasing availability of 3D facial data offers the potential to overcome the intrinsic difficulties faced by conventional face recognition using 2D images. Instead of extending 2D recognition algorithms for 3D purpose, this letter proposes a novel strategy for 3D face recognition from the perspective of representing each 3D facial surface with a 2D attribute image and taking the advantage of the advances in 2D face recognition. In our approach, each 3D facial surface is mapped homeomorphically onto a 2D lattice, where the value at each site is an attribute that represents the local 3D geometrical or textural properties on the surface, therefore invariant to pose changes. This lattice is then interpolated to generate a 2D attribute image. 3D face recognition can be achieved by applying the traditional 2D face recognition techniques to obtained attribute images. In this study, we chose the pose invariant local mean curvature calculated at each vertex on the 3D facial surface to construct the 2D attribute image and adopted the eigenface algorithm for attribute image recognition. We compared our approach to state-of-the-art 3D face recognition algorithms in the FRGC (Version 2.0), GavabDB and NPU3D database. Our results show that the proposed approach has improved the robustness to head pose variation and can produce more accurate 3D multi-pose face recognition.