Neural networks for the recognition and pose estimation of 3D objects from a single 2D perspective view

In this paper we present a neural network (NN) based system for recognition and pose estimation of 3D objects from a single 2D perspective view. We develop an appearance based neural approach for this task. First the object is represented in a feature vector derived by a principal component network. Then a NN classifier trained with Resilient backpropagation (Rprop) algorithm is applied to identify it. Next pose parameters are obtained by four NN estimators trained on the same feature vector. Performance on recognition and pose estimation for real images under occlusions are shown. Comparative studies with two other approaches are carried out.     

Design of two-dimensional recursive filters by using neuralnetworks

A new design method for two-dimensional (2-D) recursive digital filters is investigated. The design of the 2-D filter is reduced to a constrained minimization problem the solution of which is achieved by the convergence of an appropriate neural network. The method is tested on a numerical example and compared with previously published methods when applied to the same example. Advantages of the proposed method over the existing ones are discussed as well.     

Robust pose estimation from a planar target

In theory, the pose of a calibrated camera can be uniquely determined from a minimum of four coplanar but noncollinear points. In practice, there are many applications of camera pose tracking from planar targets and there is also a number of recent pose estimation algorithms which perform this task in real-time, but all of these algorithms suffer from pose ambiguities. This paper investigates the pose ambiguity for planar targets viewed by a perspective camera. We show that pose ambiguities - two distinct local minima of the according error function - exist even for cases with wide angle lenses and close range targets. We give a comprehensive interpretation of the two minima and derive an analytical solution that locates the second minimum. Based on this solution, we develop a new algorithm for unique and robust pose estimation from a planar target. In the experimental evaluation, this algorithm outperforms four state-of-the-art pose estimation algorithms     

Model-free non-rigid head pose tracking by joint shape and pose estimation

Head pose estimation under non-rigid face movement is particularly useful in applications relating to eye-gaze tracking in less constrained scenarios, where the user is allowed to move naturally during tracking. Existing vision-based head pose estimation methods often require accurate initialisation and tracking of specific facial landmarks, while methods that handle non-rigid face deformations typically necessitate a preliminary training phase prior to head pose estimation. In this paper, we propose a method to estimate the head pose in real-time from the trajectories of a set of feature points spread randomly over the face region, without requiring a training phase or model-fitting of specific facial features. Conversely, our method exploits the 3-dimensional shape of the surface of interest, recovered via shape and motion factorisation, in combination with Kalman and particle filtering to determine the contribution of each feature point to the estimation of head pose based on a variance measure. Quantitative and qualitative results reveal the capability of our method in handling non-rigid face movement without deterioration of the head pose estimation accuracy.     

单幅图像刚体目标姿态估计方法综述

刚体目标姿态作为计算机视觉技术的重点研究方向之一,旨在确定场景中3维目标的位置平移和方位旋转等多个自由度,越来越多地应用在工业机械臂操控、空间在轨服务、自动驾驶和现实增强等领域。本文对基于单幅图像的刚体目标姿态过程、方法分类及其现存问题进行了整体综述。通过利用单幅刚体目标图像实现多自由度姿态估计的各类方法进行总结、分类及比较,重点论述了姿态估计的一般过程、估计方法的演进和划分、常用数据集及评估准则、研究现状与展望。目前,多自由度刚体目标姿态估计方法主要针对单一特定应用场景具有较好的效果,还没有通用于复合场景的方法,且现有方法在面对多种光照条件、杂乱遮挡场景、旋转对称和类间相似性目标时,估计精度和效率下降显著。结合现存问题及当前深度学习技术的助推影响,从场景级多目标推理、自监督学习方法、前端检测网络、轻量高效的网络设计、多信息融合姿态估计框架和图像数据表征空间等6个方面对该领域的发展趋势进行预测和展望。     

Review and analysis of solutions of the three point perspective pose estimation problem

In this paper, the major direct solutions to the three point perspective pose estimation problems are reviewed from a unified perspective beginning with the first solution which was published in 1841 by a German mathematician, continuing through the solutions published in the German and then American photogrammetry literature, and most recently in the current computer vision literature. The numerical stability of these three point perspective solutions are also discussed. We show that even in case where the solution is not near the geometric unstable region, considerable care must be exercised in the calculation. Depending on the order of the substitutions utilized, the relative error can change over a thousand to one. This difference is due entirely to the way the calculations are performed and not due to any geometric structural instability of any problem instance. We present an analysis method which produces a numerically stable calculation.     

Real-time 6D pose estimation from a single RGB image

We propose an end-to-end deep learning architecture for simultaneously detecting objects and recovering 6D poses in an RGB image. Concretely, we extend the 2D detection pipeline with a pose estimation module to indirectly regress the image coordinates of the object's 3D vertices based on 2D detection results. Then the object's 6D pose can be estimated using a Perspective-n-Point algorithm without any post-refinements. Moreover, we elaborately design a backbone structure to maintain spatial resolution of low level features for pose estimation task. Compared with state-of-the-art RGB based pose estimation methods, our approach achieves competitive or superior performance on two benchmark datasets at an inference speed of 25 fps on a GTX 1080Ti GPU, which is capable of real-time processing.     

一种高精度鲁棒的基于直线对应的位姿估计迭代算法

2D-3D特征对应位姿估计问题算法多基于点对应,而基于直线对应求解比基于点对应求解更具有优势。从欧氏空间这一新颖角度出发,提出了一种从直线对应求解位姿估计问题的迭代算法,算法思想是先迭代求解出最优的旋转矩阵,然后再得到平移向量。针对不同的直线组情形,给出了相应的迭代初始值计算方法。仿真实验数据表明,算法具有对初始值较不敏感、高精度、鲁棒性好等特点。     

Fast and globally convergent pose estimation from video images

Determining the rigid transformation relating 2D images to known 3D geometry is a classical problem in photogrammetry and computer vision. Heretofore, the best methods for solving the problem have relied on iterative optimization methods which cannot be proven to converge and/or which do not effectively account for the orthonormal structure of rotation matrices. We show that the pose estimation problem can be formulated as that of minimizing an error metric based on collinearity in object (as opposed to image) space. Using object space collinearity error, we derive an iterative algorithm which directly computes orthogonal rotation matrices and which is globally convergent. Experimentally, we show that the method is computationally efficient, that it is no less accurate than the best currently employed optimization methods, and that it outperforms all tested methods in robustness to outliers     

On the improvement of anthropometry and pose estimation from a single uncalibrated image

Recently, we developed a technique that allows semi-automatic estimation of anthropometry and pose from a single image. However, estimation was limited to a class of images for which an adequate number of human body segments were almost parallel to the image plane. In this paper, we present a generalization of that estimation algorithm that exploits pairwise geometric relationships of body segments to allow estimation from a broader class of images. In addition, we refine our search space by constructing a fully populated discrete hyper-ellipsoid of stick human body models in order to capture the variance of the statistical anthropometric information. As a result, a better initial estimate can be computed by our algorithm and thus the number of iterations needed during minimization are reduced tenfold. We present our results over a variety of images to demonstrate the broad coverage of our algorithm.Published online: 1 September 2003     

Hierarchical topology based hand pose estimation from a single depth image

Hand pose estimation benefits large human computer interaction applications. The hand pose has high dimensions of freedom (dof) for joints, and various hand poses are flexible. Hand pose estimation is still a challenge problem. Since hand joints on the hand skeleton topology model have strict relationships between each other, we propose a hierarchical topology based approach to estimate 3D hand poses. First, we determine palm positions and palm orientations by detecting hand fingertips and calculating their directions in depth images. It is the global topology of hand poses. Moreover, we define connection relationships of finger joints as the local topology of hand model. Based on hierarchical topology, we extract angle features to describe hand poses, and adopt the regression forest algorithm to estimate 3D coordinates of hand joints. We further use freedom forrest algorithm to refine ambiguous poses in estimation to solve error accumulation problem. The hierarchical topology based approach ensures estimated hand poses in a reasonable topology, and improves estimation accuracy. We evaluate our approach on two public databases, and experiments illustrate its efficiency. Compared with state-of-the-art approaches, our approach improves estimation accuracy.     

Automatic model construction and pose estimation from photographsusing triangular splines

This paper addresses the automatic construction of complex spline object models from a few photographs. Our approach combines silhouettes from registered images to construct a G¹-continuous triangular spline approximation of an object with unknown topology. We apply a similar optimization procedure to estimate the pose of a modeled object from a single image. Experimental examples of model construction and pose estimation are presented for several complex objects     

Real-time dance evaluation by markerless human pose estimation

Multimedia Tools and Applications - This paper presents a unified framework that evaluates dance performance by markerless estimation of human poses. Dance involves complicated poses such as...     

Recognition of human head orientation based on artificial neuralnetworks

Humans easily recognize where another person is looking and often use this information for interspeaker coordination. We present a method based on three neural networks of the local linear map type which enables a computer to identify the head orientation of a user by learning from examples. One network is used for color segmentation, a second for localization of the face, and the third for the final recognition of the head orientation. The system works at a frame rate of one image per second on a common workstation, We analyze the accuracy achieved at different processing steps and discuss the usability of the approach in the context of a visual human-machine interface.     

Multiple human 3D pose estimation from multiview images

Multimedia Tools and Applications - Multiple human 3D pose estimation is a challenging task. It is mainly because of large variations in the scale and pose of humans, fast motions, multiple persons...     

Global hand pose estimation by multiple camera ellipse tracking

Immersive virtual environments with life-like interaction capabilities have very demanding requirements including high-precision motion capture and high-processing speed. These issues raise many challenges for computer vision-based motion estimation algorithms. In this study, we consider the problem of hand tracking using multiple cameras and estimating its 3D global pose (i.e., position and orientation of the palm). Our interest is in developing an accurate and robust algorithm to be employed in an immersive virtual training environment, called “Virtual GloveboX” (VGX) (Twombly et al. in J Syst Cybern Inf 2:30–34, 2005), which is currently under development at NASA Ames. In this context, we present a marker-based, hand tracking and 3D global pose estimation algorithm that operates in a controlled, multi-camera, environment built to track the user’s hand inside VGX. The key idea of the proposed algorithm is tracking the 3D position and orientation of an elliptical marker placed on the dorsal part of the hand using model-based tracking approaches and active camera selection. It should be noted that, the use of markers is well justified in the context of our application since VGX naturally allows for the use of gloves without disrupting the fidelity of the interaction. Our experimental results and comparisons illustrate that the proposed approach is more accurate and robust than related approaches. A byproduct of our multi-camera ellipse tracking algorithm is that, with only minor modifications, the same algorithm can be used to automatically re-calibrate (i.e., fine-tune) the extrinsic parameters of a multi-camera system leading to more accurate pose estimates.     

Connected line drawing extraction from a perspective view of a polyhedron

To extract line drawings with positional information from perspective veiws of three-dimensional objects is essential in image analysis and understanding. A new heuristic-search algorithm driven by a priori knowledge contained in a world model is presented which extracts a connected line drawing from a perspective view of a polyhedron. A main feature of our algorithm is that the search is concentrated on local areas centered at corners found with a corner finder. Therefore, the search time is significantly reduced and so are the positional errors in the extracted line drawing. An iterative process removes the false corners and lines and thus guarantees that our algorithm will work stably and reliably even in a noisy environment. Experimental results are presented.     

Robust real-time 3D head pose estimation from range data

In this paper a real-time 3D pose estimation algorithm using range data is described. The system relies on a novel 3D sensor that generates a dense range image of the scene. By not relying on brightness information, the proposed system guarantees robustness under a variety of illumination conditions, and scene contents. Efficient face detection using global features and exploitation of prior knowledge along with novel feature localization and tracking techniques are described. Experimental results demonstrate accurate estimation of the six degrees of freedom of the head and robustness under occlusions, facial expressions, and head shape variability.     

Cascaded hierarchical CNN for 2D hand pose estimation from a single color image

Due to severe articulation, self-occlusion, various scales, and high dexterity of the hand, hand pose estimation is more challenging than body pose estimation. Recently-developed body pose estimation algorithms are not suitable for addressing the unique challenges of hand pose estimation because they are trained without explicitly modeling structural relationships between keypoints. In this paper, we propose a novel cascaded hierarchical CNN(CH-HandNet) for 2D hand pose estimation from a single color image. The CH-HandNet includes three modules, hand mask segmentation, preliminary 2D hand pose estimation, and hierarchical estimation. The first module obtains a hand mask by hand mask segmentation network. The second module connects the hand mask and the intermediate image features to estimate the 2D hand heatmaps. The last module connects hand heatmaps with the intermediate image features and hand mask to estimate finger and palm heatmaps hierarchically. Finally, the extracted Finger(pinky,ring,middle,index) and Palm(thumb and palm) feature information are fused to estimate 2D hand pose. Experimental results on three datasets - OneHand 10k, Panoptic, and Eric.Lee, consistently shows that our proposed CH-HandNet outperforms previous state-of-the-art hand pose estimation methods.