A sensor-based SLAM algorithm for camera tracking in virtual studio

This paper addresses a sensor-based simultaneous localization and mapping （SLAM） algorithm for camera tracking in a virtual studio environment. The traditional camera tracking methods in virtual studios are vision-based or sensor-based. However, the chroma keying process in virtual studios requires color cues, such as blue background, to segment foreground objects to be inserted into images and videos. Chroma keying limits the application of vision-based tracking methods in virtual studios since the background cannot provide enough feature information. Furthermore, the conventional sensor-based tracking approaches suffer from the jitter, drift or expensive computation due to the characteristics of individual sensor system. Therefore, the SLAM techniques from the mobile robot area are first investigated and adapted to the camera tracking area. Then, a sensor-based SLAM extension algorithm for two dimensional （2D） camera tracking in virtual studio is described. Also, a technique called map adjustment is proposed to increase the accuracy＇ and efficiency of the algorithm. The feasibility and robustness of the algorithm is shown by experiments. The simulation results demonstrate that the sensor-based SLAM algorithm can satisfy the fundamental 2D camera tracking requirement in virtual studio environment.     

未标定摄像机姿态估计的矢量求解算法*

提出一种基于矢量运算未标定摄像机姿态估计的求解算法。重点研究了P5P问题,将五个控制点组成四个矢量,根据成像过程,由控制点矢量运算逐步构建摄像机姿态和相机矩阵的线性约束方程。依据线性理论及旋转阵R的正交性化简约束方程,通过矢量运算给出未标定P5P问题摄像机姿态和相机矩阵的解析解。给出有足够约束条件的PnP问题的求解过程。模拟和真实实验都验证了该方法的有效性。     

摄像机位姿的高精度快速求解

目的 针对对应点个数大于等于6的摄像机位姿估计问题,提出一种既适用于已标定也适用于未标定摄像机的时间复杂度为 的高精度快速算法。 方法 首先选取四个非共面虚拟控制点,并根据空间点和虚拟控制点的空间关系以及空间点的图像建立线性方程组,以此求解虚拟控制点的图像坐标及摄像机内参,再由POSIT算法根据虚拟控制点及其图像坐标求解旋转矩阵和平移向量。 结果 模拟数据实验和真实图像实验表明该算法时间复杂度和计算精度均优于现有的已标定摄像机位姿的高精度快速求解算法EPnP。 结论 该算法能够同时估计摄像机内外参数,而且比现有算法具有更好的速度和精度。     

Efficient lookup table based camera pose estimation for augmented reality

Until now, exising camera pose estimation methods for the widely used square marker‐based augmented reality (AR) are either highly sensitive to noise or much time consuming, and developers have to work hard to find the proper trade‐off between computational speed and quality in mobile AR applications where computational resources are limited. The major difficulty is that only the four corner points of the square AR marker are available, and no redundant point correspondences can be used for a stable estimation. To solve this problem, an efficient lookup table (LUT)‐based non‐iterative solution is presented in this paper that achieves high stability in the presence of noise better than the most robust and accurate iterative solutions in the field, with the same level of accuracy and a much lower computational complexity. Our central idea consists of extracting a key parameter β from the camera pose and creating a LUT for β by taking the symmetrical structure of the square marker into account, thereby exploiting additional information. Copyright © 2011 John Wiley & Sons, Ltd.     

Global hand pose estimation by multiple camera ellipse tracking

Immersive virtual environments with life-like interaction capabilities have very demanding requirements including high-precision motion capture and high-processing speed. These issues raise many challenges for computer vision-based motion estimation algorithms. In this study, we consider the problem of hand tracking using multiple cameras and estimating its 3D global pose (i.e., position and orientation of the palm). Our interest is in developing an accurate and robust algorithm to be employed in an immersive virtual training environment, called “Virtual GloveboX” (VGX) (Twombly et al. in J Syst Cybern Inf 2:30–34, 2005), which is currently under development at NASA Ames. In this context, we present a marker-based, hand tracking and 3D global pose estimation algorithm that operates in a controlled, multi-camera, environment built to track the user’s hand inside VGX. The key idea of the proposed algorithm is tracking the 3D position and orientation of an elliptical marker placed on the dorsal part of the hand using model-based tracking approaches and active camera selection. It should be noted that, the use of markers is well justified in the context of our application since VGX naturally allows for the use of gloves without disrupting the fidelity of the interaction. Our experimental results and comparisons illustrate that the proposed approach is more accurate and robust than related approaches. A byproduct of our multi-camera ellipse tracking algorithm is that, with only minor modifications, the same algorithm can be used to automatically re-calibrate (i.e., fine-tune) the extrinsic parameters of a multi-camera system leading to more accurate pose estimates.     

Homography-based camera pose estimation with known gravity direction for UAV navigation

Relative pose estimation has become a fundamental and important problem in visual simulta-neous localization and mapping.This paper statistically optimizes the ...     

A performance study for camera pose estimation using visual marker based tracking

Vision-based tracking systems are widely used for augmented reality (AR) applications. Their registration can be very accurate and there is no delay between real and virtual scene. However, vision-based tracking often suffers from limited range, errors, heavy processing time and present erroneous behavior due to numerical instability. To address these shortcomings, robust method are required to overcome these problems. In this paper, we survey classic vision-based pose computations and present a method that offers increased robustness and accuracy in the context of real-time AR tracking. In this work, we aim to determine the performance of four pose estimation methods in term of errors and execution time. We developed a hybrid approach that mixes an iterative method based on the extended Kalman filter (EKF) and an analytical method with direct resolution of pose parameters computation. The direct method initializes the pose parameters of the EKF algorithm which performs an optimization of these parameters thereafter. An evaluation of the pose estimation methods was obtained using a series of tests and an experimental protocol. The analysis of results shows that our hybrid algorithm improves stability, convergence and accuracy of the pose parameters.     

Robust camera pose estimation by viewpoint classification using deep learning

Computational Visual Media - Camera pose estimation with respect to target scenes is an important technology for superimposing virtual information in augmented reality (AR). However, it is...     

基于自适应粒子滤波的摄像机位姿估计方法

提出一种基于自适应粒子滤波的摄像机位姿估计方法。该方法首先利用相邻两帧传递模型的噪声方差动态调整传递模型,接着利用内点统计方法计算粒子权值,在对权值作归一化运算之后,利用粒子加权和计算摄像机位置和姿态。实验结果表明该方法很大程度上提高了基于标识的摄像机位姿估计系统的健壮性与稳定性。     

Hybrid three-dimensional camera pose estimation using particle filter sensor fusion

To properly align objects in the real and virtual worlds in an augmented reality (AR) space it is essential to keep tracking the camera's exact three-dimensional position and orientation (camera pose). State-of-the-art analysis shows that traditional vision-based or inertial sensor-based solutions are not adequate when used individually. Sensor fusion for hybrid tracking has become an active research direction during the past few years, although how to do it in a robust and principled way is still an open problem. In this paper, we develop a hybrid camera pose-tracking system that combines vision and inertial sensor technologies. We propose to use the particle filter framework for the sensor fusion system. Particle filters are sequential Monte-Carlo methods based upon a point mass (or 'particle') representation of probability densities, which can be applied to any state space model and which generalize the traditional Kalman filtering methods. We have tested our algorithm to evaluate its performance and have compared the results obtained by the particle filter with those given by a classical extended Kalman filter. Experimental results are presented     

Fast and robust absolute camera pose estimation with known focal length

Some 3D computer vision techniques such as structure from motion (SFM) and augmented reality (AR) depend on a specific perspective-n-point (PnP) algorithm to estimate the absolute camera pose. However, existing PnP algorithms are difficult to achieve a good balance between accuracy and efficiency, and most of them do not make full use of the internal camera information such as focal length. In order to attack these drawbacks, we propose a fast and robust PnP (FRPnP) method to calculate the absolute camera pose for 3D compute vision. In the proposed FRPnP method, we firstly formulate the PnP problem as the optimization problem in the null space that can avoid the effects of the depth of each 3D point. Secondly, we can easily get the solution by the direct manner using singular value decomposition. Finally, the accurate information of camera pose can be obtained by optimization strategy. We explore four ways to evaluate the proposed FRPnP algorithm with synthetic dataset, real images, and apply it in the AR and SFM system. Experimental results show that the proposed FRPnP method can obtain the best balance between computational cost and precision, and clearly outperforms the state-of-the-art PnP methods.

     

GeoPose3K: Mountain landscape dataset for camera pose estimation in outdoor environments

We introduce a new dataset called GeoPose3K¹ which contains over three thousand precise camera poses of mountain landscape images. In addition to camera location and orientation, we provide data for the training and evaluation of computer vision methods and applications in the context of outdoor scenes; synthetic depth maps, normal maps, illumination simulation and semantic labels. In order to illustrate properties of the dataset, we compare results achieved by state-of-the-art visual geo-localization method on GeoPose3K with results achieved on an existing dataset for visual geo-localization. So as to foster research of computer vision algorithms for outdoor environments, several novel future use-cases of our new GeoPose3K dataset are proposed.     

A virtual studio environment for design integration

In this paper the authors attempt to stress the social dimension of design and the role of explicit support for human-level interaction during design systems integration. A human-centred approach is proposed by taking design integration as the collaborative use of design artefacts, and a virtual studio environment (VSE) framework is presented as an integration vehicle to link the social and technical dimensions. A VSE consists of two subsystems: the VSE base system and the domain resources. While common generic facilities for human–human interaction are embedded within the VSE base system, the domain-specific resources are loosely coupled into VSE via resource agents. A VSE prototype for the domain of building design is described, and a demonstration of the use of the VSE prototype is presented. This is then followed by some discussion on related research and further work. © 1998 Published by Elsevier Science Ltd. All rights reserved.     

Linear N-point camera pose determination

The determination of camera position and orientation from known correspondences of 3D reference points and their images is known as pose estimation in computer vision and space resection in photogrammetry. It is well-known that from three corresponding points there are at most four algebraic solutions. Less appears to be known about the cases of four and five corresponding points. We propose a family of linear methods that yield a unique solution to 4- and 5-point pose determination for generic reference points. We first review the 3-point algebraic method. Then we present our two-step, 4-point and one-step, 5-point linear algorithms. The 5-point method can also be extended to handle more than five points. Finally, we demonstrate our methods on both simulated and real images. We show that they do not degenerate for coplanar configurations and even outperform the special linear algorithm for coplanar configurations in practice     

Robust pose estimation

Standard least-squares (LS) methods for pose estimation of objects are sensitive to outliers which can occur due to mismatches. Even a single mismatch can severely distort the estimated pose. This paper describes a least-median of squares (LMedS) approach to estimating pose using point matches. It is both robust (resistant to up to 50% outliers) and efficient (linear in the number of points). The basic algorithm is then extended to improve performance in the presence of two types of noise: 1) type I which perturbs all data values by small amounts (e.g., Gaussian) and 2) type II which can corrupt a few data values by large amounts     

Reliable pose estimation of underwater dock using single camera: a scene invariant approach

It is well known that docking of Autonomous Underwater Vehicle (AUV) provides scope to perform long duration deep-sea exploration. A large amount of literature is available on vision-based docking which exploit mechanical design, colored markers to estimate the pose of a docking station. In this work, we propose a method to estimate the relative pose of a circular-shaped docking station (arranged with LED lights on periphery) up to five degrees of freedom (5-DOF, neglecting roll effect). Generally, extraction of light markers from underwater images is based on fixed/adaptive choice of threshold, followed by mass moment-based computation of individual markers as well as center of the dock. Novelty of our work is the proposed highly effective scene invariant histogram-based adaptive thresholding scheme (HATS) which reliably extracts positions of light sources seen in active marker images. As the perspective projection of a circle features a family of ellipses, we then fit an appropriate ellipse for the markers and subsequently use the ellipse parameters to estimate the pose of a circular docking station with the help of a well-known method in Safaee-Rad et al. (IEEE Trans Robot Autom 8(5):624–640, 1992). We analyze the effectiveness of HATS as well as proposed approach through simulations and experimentation. We also compare performance of targeted curvature-based pose estimation with a non-iterative efficient perspective-n-point (EPnP) method. The paper ends with a few interesting remarks on vantages with ellipse fitting for markers and utility of proposed method in case of non-detection of all the light markers.     

Gravitational pose estimation

Problem of relative pose estimation between a camera and rigid object, given an object model with feature points and image(s) with respective image points (hence known correspondence) has been extensively studied in the literature. We propose a “correspondenceless” method called gravitational pose estimation (GPE), which is inspired by classical mechanics. GPE can handle occlusion and uses only one image (i.e., perspective projection of the object). GPE creates a simulated gravitational field from the image and lets the object model move and rotate in that force field, starting from an initial pose. Experiments were carried out with both real and synthetic images. Results show that GPE is robust, consistent, and fast (runs in less than a minute). On the average (including up to 30% occlusion cases) it finds the orientation within 6° and the position within 17% of the object’s diameter. SoftPOSIT was so far the best correspondenceless method in the literature that works with a single image and point-based object model like GPE. However, SoftPOSIT’s convergence to a result is sensitive to the choice of initial pose. Even “random start SoftPOSIT,” which performs multiple runs of SoftPOSIT with different initial poses, can often fail. However, SoftPOSIT finds the pose with great precision when it is able to converge. We have also integrated GPE and SoftPOSIT into a single method called GPEsoftPOSIT, which finds the orientation within 3° and the position within 10% of the object’s diameter even under occlusion. In GPEsoftPOSIT, GPE finds a pose that is very close to the true pose, and then SoftPOSIT is used to enhance accuracy of the result. Unlike SoftPOSIT, GPE also has the ability to work with three points as well as planar object models.     

Hand pose estimation and tracking in real and virtual interaction:A review

Vision-based 3D hand tracking is a key and popular component for interaction studies in a broad range of domains such as virtual reality (VR), augmented reality (AR) and natural human-computer interaction (HCI). While this research field has been well studied in the last decades, most approaches have considered the human hand in isolation and not in action or in interaction with the surrounding environment. Even the common collaborative and strong interactions with the other hand have been ignored. However, many of today's computer applications require more and more hand-object interactions. Furthermore, employing contextual information about the object in the hand (e.g. the shape, the texture, and the pose) can remarkably constrain the tracking problem. The most studied contextual constraints involve interaction with real objects and not with virtual objects which is still a very big challenge. The goal of this survey is to develop an up-to-date taxonomy of the state-of-the-art vision-based hand pose estimation and tracking methods with a new classification scheme: hand-object interaction constraints. This taxonomy allows us to examine the strengths and weaknesses of the current state of the art and to highlight future trends in the domain.     

A low-complexity sensor fusion algorithm based on a fiber-optic gyroscope aided camera pose estimation system

Visual tracking, as a popular computer vision technique, has a wide range of applications, such as camera pose estimation. Conventional methods for it are mostly based on vision only, which are complex for image processing due to the use of only one sensor. This paper proposes a novel sensor fusion algorithm fusing the data from the camera and the fiber-optic gyroscope. In this system, the camera acquires images and detects the object directly at the beginning of each tracking stage; while the relative motion between the camera and the object measured by the fiber-optic gyroscope can track the object coordinate so that it can improve the effectiveness of visual tracking. Therefore, the sensor fusion algorithm presented based on the tracking system can overcome the drawbacks of the two sensors and take advantage of the sensor fusion to track the object accurately. In addition, the computational complexity of our proposed algorithm is obviously lower compared with the existing approaches(86% reducing for a 0.5 min visual tracking). Experiment results show that this visual tracking system reduces the tracking error by 6.15% comparing with the conventional vision-only tracking scheme(edge detection), and our proposed sensor fusion algorithm can achieve a long-term tracking with the help of bias drift suppression calibration.     

基于全局姿态感知的轻量级人体姿态估计

人体姿态估计是近年来人机交互领域的热点话题.当前,常见的人体姿态估计方法集中在通过增加网络的复杂性来提高精度,却忽视了模型的效益问题,导致模型在实际应用中精度高但计算资源消耗巨大.针对这一问题设计了一个基于全局姿态感知的轻量级人体姿态估计模型,其在MSCOCO数据集上精度达68.2％AP,速度保持在255 fps,参数...