基于关键点的类别级三维可形变目标姿态估计

为了解决类别级三维可形变目标姿态估计问题,基于目标的关键点,提出了一种面向类别的三维可形变目标姿态估计方法。该方法设计了一种基于关键点的端到端深度学习框架,框架以PointNet++为后端网络,通过特征提取、部位分割、关键点提取和基于关键点的姿态估计部分实现可形变目标的姿态估计,具有计算精度高、鲁棒性强等优势。同时,基于ANCSH方法设计了适用于K-AOPE网络的关键点标准化分层表示方法,该方法仅需提取目标少量的关键点即可表示类别物体。为了验证方法的有效性,在公共数据集shape2motion上进行测试。实验结果显示,提出的姿态估计方法(以眼镜类别为例)在旋转角上的误差分别为2.3°、3.1°、3.7°,平移误差分别为0.034、0.030、0.046,连接状态误差为2.4°、2.5°,连接参数误差为1.2°、0.9°,0.008、0.010。与ANCSH方法相比,所提方法具有较高的准确性和鲁棒性。     

3D pose estimation for articulated vehicles using Kalman-filter based tracking

Knowledge about relative poses within a tractor/trailer combination is a vital prerequisite for kinematic modelling and trajectory estimation. In case of autonomous vehicles or driver assistance systems, for example, the monitoring of an attached passive trailer is crucial for operational safety. We propose a camerabased 3D pose estimation system based on a Kalman-filter. It is evaluated against previously published methods for the same problem.     

Estimating workpiece pose using the feature points method

The ability of a robot to estimate workpiece pose is necessary for many industrial tasks. An automatic method of visually estimating the pose of a workpiece in a robot hand has been investigated. This method is applicable to workpieces that have six continuous unknown degrees of freedom. Furthermore, partial occlusion of the workpiece by the robot hand is allowed. Workpiece pose was estimated using the three-dimensional locations of at least three noncollinear workpiece feature points. A second view of the workpiece was obtained by rotating the workpiece in front of the camera. Corresponding image features on the second view were found by searching along lines on the image which were determined by the rotation of the workpiece. A very accurate camera calibration procedure has been developed to compute rays in space from image locations. The location of workpiece features was computed by trigonometric relations between rays of corresponding image features from two views of the workpiece. The pose of a workpiece was estimated by the correspondence between the workpiece features located and the feature points of a workpiece model. Experiments have been performed on a complete functional system. The major limitation of the current system was due to the image feature extraction algorithm which only detected corners and small holes.     

Viewpoint invariant characteristics of articulated objects

The aim of this paper is to investigate whether it is possible to construct invariants for articulated objects—these are objects that are composed of different rigid parts which are allowed to perform a restricted motion with respect to each other—which use only partial information from each component. To this end, the transformation group describing the deformations of the image of an articulated object due to relative motions of the components, and/or changes in the position of the camera, is identified. It turns out that for a planar articulated object with two rigid components that are allowed to move within the object plane, this transformation group is (anti-isomorphic to) the semi-direct product of the group one would obtain if the object was rigid, and its smallest normal subgroup containing the transformations due to the relative motions of the components. Depending on the projection model, different answers to the question above evoke. For instance, when using perspective projection no other invariants exist than those obtained by considering each part separately as a rigid object, whereas in the pseudo-orthographic case simpler invariants (using only partial information from each component) do exist. Examples of such invariants are given.Postdoctoral Research Fellow of the Belgian National Fund for Scientific Research (N.F.W.O.).     

Multi-scale 2D tracking of articulated objects using hierarchical spring systems

This paper presents a flexible framework to build a target-specific, part-based representation for arbitrary articulated or rigid objects. The aim is to successfully track the target object in 2D, through multiple scales and occlusions. This is realized by employing a hierarchical, iterative optimization process on the proposed representation of structure and appearance. Therefore, each rigid part of an object is described by a hierarchical spring system represented by an attributed graph pyramid. Hierarchical spring systems encode the spatial relationships of the features (attributes of the graph pyramid) describing the parts and enforce them by spring-like behavior during tracking. Articulation points connecting the parts of the object allow to transfer position information from reliable to ambiguous parts. Tracking is done in an iterative process by combining the hypotheses of simple trackers with the hypotheses extracted from the hierarchical spring systems.     

Recognition of articulated and occluded objects

A model-based automatic target recognition system is developed to recognize articulated and occluded objects in synthetic aperture radar (SAR) images, based on invariant features of the objects. Characteristics of SAR target image scattering centers, azimuth variation, and articulation invariants are presented. The basic elements of the new recognition system are described and performance results are given for articulated, occluded and occluded articulated objects, and they are related to the target articulation invariance and percent unoccluded     

Estimating 3D hand pose using hierarchical multi-label classification

This paper presents an analysis of the design of classifiers for use in a hierarchical object recognition approach. In this approach, a cascade of classifiers is arranged in a tree in order to recognize multiple object classes. We are interested in the problem of recognizing multiple patterns as it is closely related to the problem of locating an articulated object. Each different pattern class corresponds to the hand in a different pose, or set of poses. For this problem obtaining labelled training data of the hand in a given pose can be problematic. Given a parametric 3D model, generating training data in the form of example images is cheap, and we demonstrate that it can be used to design classifiers almost as good as those trained using non-synthetic data. We compare a variety of different template-based classifiers and discuss their merits.     

Constrained pose refinement of parametric objects

Pose refinement is an essential task for computer vision systems that require the calibration and verification of model and camera parameters. Typical domains include the real-time tracking of objects and verification in model-based recognition systems. A technique is presented for recovering model and camera parameters of 3D objects from a single two-dimensional image. This basic problem is further complicated by the incorporation of simple bounds on the model and camera parameters and linear constraints restricting some subset of object parameters to a specific relationship. It is demonstrated in this paper that this constrained pose refinement formulation is no more difficult than the original problem based on numerical analysis techniques, including active set methods and lagrange multiplier analysis. A number of bounded and linearly constrained parametric models are tested and convergence to proper values occurs from a wide range of initial error, utilizing minimal matching information (relative to the number of parameters and components). The ability to recover model parameters in a constrained search space will thus simplify associated object recognition problems.     

Occlusion-aware multi-view reconstruction of articulated objects for manipulation

We present an algorithm called Procrustes-Lo-RANSAC (PLR) to recover complete 3D models of articulated objects. Structure-from-motion techniques are used to capture 3D point cloud models of an object in two different configurations. Procrustes analysis, combined with a locally optimized RANSAC sampling strategy, facilitates a straightforward geometric approach to recovering the joint axes, as well as classifying them automatically as either revolute or prismatic. With the resulting articulated model, a robotic system is then able to manipulate the object along its joint axes at a specified grasp point in order to exercise its degrees of freedom. Because the models capture all sides of the object, they are occlusion-aware, meaning that the robot has knowledge of parts of the object that are not visible in the current view. Our algorithm does not require prior knowledge of the object, nor does it make any assumptions about the planarity of the object or scene. Experiments with a PUMA 500 robotic arm demonstrate the effectiveness of the approach on a variety of real-world objects containing both revolute and prismatic joints.     

Interpolation synthesis of articulated figure motion

Most conventional media depend on engaging and appealing characters. Empty spaces and buildings would not fare well as television or movie programming, yet virtual reality usually offers up such spaces. The problem lies in the difficulty of creating computer generated characters that display real time, engaging interaction and realistic motion. Articulated figure motion for real time computer graphics offers one solution to this problem. A common approach stores a set of motions and lets you choose one particular motion at a time. The article describes a process that greatly expands the range of possible motions. Mixing motions selected from a database lets you create a new motion to exact specifications. The synthesized motion retains the original motions' subtle qualities, such as the realism of motion capture or the expressive, exaggerated qualities of artistic animation. Our method provides a new way to achieve inverse kinematics capability-for example, placing the hands or feet of an articulated figure in specific positions. It proves useful for both real time graphics and prerendered animation production. The method, called interpolation synthesis, is based on motion capture data and it provides real time character motion for interactive entertainment or avatars in virtual worlds     

Motion analysis of articulated objects from monocular images

This paper presents a new method of motion analysis of articulated objects from feature point correspondences over monocular perspective images without imposing any constraints on motion. An articulated object is modeled as a kinematic chain consisting of joints and links, and its 3D joint positions are estimated within a scale factor using the connection relationship of two links over two or three images. Then, twists and exponential maps are employed to represent the motion of each link, including the general motion of the base link and the rotation of other links around their joints. Finally, constraints from image point correspondences, which are similar to that of the essential matrix in rigid motion, are developed to estimate the motion. In the algorithm, the characteristic of articulated motion, i.e., motion correlation among links, is applied to decrease the complexity of the problem and improve the robustness. A point pattern matching algorithm for articulated objects is also discussed in this paper. Simulations and experiments on real images show the correctness and efficiency of the algorithms.     

Stereo Pictorial Structure for 2D articulated human pose estimation

In this paper, we consider the problem of 2D human pose estimation on stereo image pairs. In particular, we aim at estimating the location, orientation and scale of upper-body parts of people detected in stereo image pairs from realistic stereo videos that can be found in the Internet. To address this task, we propose a novel pictorial structure model to exploit the stereo information included in such stereo image pairs: the Stereo Pictorial Structure (SPS). To validate our proposed model, we contribute a new annotated dataset of stereo image pairs, the Stereo Human Pose Estimation Dataset (SHPED), obtained from YouTube stereoscopic video sequences, depicting people in challenging poses and diverse indoor and outdoor scenarios. The experimental results on SHPED indicates that SPS improves on state-of-the-art monocular models thanks to the appropriate use of the stereo information.     

Linearized motion estimation for articulated planes

In this paper, we describe the explicit application of articulation constraints for estimating the motion of a system of articulated planes. We relate articulations to the relative homography between planes and show that these articulations translate into linearized equality constraints on a linear least-squares system, which can be solved efficiently using a Karush-Kuhn-Tucker system. The articulation constraints can be applied for both gradient-based and feature-based motion estimation algorithms and to illustrate this, we describe a gradient-based motion estimation algorithm for an affine camera and a feature-based motion estimation algorithm for a projective camera that explicitly enforces articulation constraints. We show that explicit application of articulation constraints leads to numerically stable estimates of motion. The simultaneous computation of motion estimates for all of the articulated planes in a scene allows us to handle scene areas where there is limited texture information and areas that leave the field of view. Our results demonstrate the wide applicability of the algorithm in a variety of challenging real-world cases such as human body tracking, motion estimation of rigid, piecewise planar scenes, and motion estimation of triangulated meshes.     

Genetic programming for articulated figure motion

We present an approach to articulated figure motion in which motion tasks are defined in terms of goals and ratings. The agents are dynamically-controlled robots whose behaviour is determined by robotic controller programs. The controller programs for the robots are evaluated at each time step to yield torque values which drive the dynamic simulation of the motion. We use the AI technique of genetic programming (GP) to automatically derive control programs for the agents which achieve the goals. This type of motion specification is an alternative to key framing which allows a highly automated, learning-based approach to generation of motion. This method of motion control is very general (it can be applied to any type of motion), yet it allows for specifications of the types of specific motion which are desired for a high quality animation. We show that complex, specific, physically plausible and aesthetically appealing motion can be generated using these methods.     

Reprint of: Multi-scale 2D tracking of articulated objects using hierarchical spring systems

This paper presents a flexible framework to build a target-specific, part-based representation for arbitrary articulated or rigid objects. The aim is to successfully track the target object in 2D, through multiple scales and occlusions. This is realized by employing a hierarchical, iterative optimization process on the proposed representation of structure and appearance. Therefore, each rigid part of an object is described by a hierarchical spring system represented by an attributed graph pyramid. Hierarchical spring systems encode the spatial relationships of the features (attributes of the graph pyramid) describing the parts and enforce them by spring-like behavior during tracking. Articulation points connecting the parts of the object allow to transfer position information from reliable to ambiguous parts. Tracking is done in an iterative process by combining the hypotheses of simple trackers with the hypotheses extracted from the hierarchical spring systems.     

On using CAD models to compute the pose of curved 3D objects

A new approach is presented for explicitly relating image observables to models of curved three-dimensional objects. This relationship is used for object recognition and positioning. Object models consist of collections of parametric surface patches. The image observables considered are raw range data, surface normal and Gaussian curvature, raw image intensity and intensity gradient, raw image contours, and contour orientation and curvature. Elimination theory provides a method for constructing an implicit equation that relates these observables to the three-dimensional position and orientation of object models. Determining the unknown pose parameters is reduced to a fitting problem between the implicit equation and the observed data points. By considering translation-independent observables such as surface normal and curvature, this process is further decomposed into first determining orientation and then determining translation. Applications to object recognition are described, and an implementation is presented.     

SAM-animation software for simulating articulated motion

A collection of software used for interactive specification, motion control and graphics simulation of articulated objects of arbitrary complexity is described. While used primarily for simulating and evaluating robotic manipulators, it has also been applied to the animation of biological models. One of the key issues discussed involves a flexible and intuitive approach to the motion specification for that large class of objects which possess redundant degrees of freedom.     

Accurate constrained pose estimation for small objects

The paper proposes a novel method for accurate pose estimation of small objects. A range sensor is used to find object orientation that constraints a PnP solver. The method finds the global minimum of a cost function (as opposed to traditional PnP algorithms) in closed form. This approach was used to enable robust robotic operation with sub millimeter precision.     

Constraint fusion for recognition and localization of articulated objects

This paper presents a method for localization and interpretation of modeled objects that is general enough to cover articulated and other types of constrained models. The flexibility between the components of the model is expressed as spatial constraints that are fused into the pose estimation during the interpretation process. The constraint fusion assists in obtaining a precise and stable pose of each of the object's components and in finding the correct interpretation. The proposed method can handle any constraint (including inequalities) between any number of different components of the model. The framework is based on Kalman filtering.