Inside looking out camera pose estimation for virtual studio

This paper studies the inside looking out camera pose estimation for the virtual studio. The camera pose estimation process, the process of estimating a camera’s extrinsic parameters, is based on closed-form geometrical approaches which use the benefit of simple corner detection of 3D cubic-like virtual studio landmarks. We first look at the effective parameters of the camera pose estimation process for the virtual studio. Our studies include all characteristic landmark parameters like landmark lengths, landmark corner angles and their installation position errors and some camera parameters like lens focal length and CCD resolution. Through computer simulation we investigate and analyze all these parameters’ efficiency in camera extrinsic parameters, including camera rotation and position matrixes. Based on this work, we found that the camera translation vector is affected more than other camera extrinsic parameters because of the noise of effective camera pose estimation parameters. Therefore, we present a novel iterative geometrical noise cancellation method for the closed-form camera pose estimation process. This is based on the collinearity theory that reduces the estimation error of the camera translation vector, which plays a major role in camera extrinsic parameters estimation errors. To validate our method, we test it in a complete virtual studio simulation. Our simulation results show that they are in the same order as those of some commercial systems, such as the BBC and InterSense IS-1200 VisTracker.     

Invisible Marker–Based Augmented Reality

To estimate camera pose, existing augmented reality (AR) techniques usually require fiducial markers with known geometry. However, placing a marker in the workspace of the user can be very visually intrusive. To overcome this limitation, a nonintrusive AR method using invisible markers that are created/drawn with an infrared (IR) ink is proposed and an IR marker tracking system is presented. This system includes additional algorithms to maintain reliable performance in a cluttered background. Then, the working conditions of IR markers are examined and compared with those of visual markers, and the optimal working conditions of IR markers are discussed. Next, the qualitative evaluation of an IR marker–based AR is presented through a variety of experiments and user evaluations. Finally, the potential applications of IR marker–based AR are explored.     

STag: A stable fiducial marker system

Fiducial markers provide better-defined features than the ones naturally available in the scene. For this reason, they are widely utilized in computer vision applications where reliable pose estimation is required. Factors such as imaging noise and subtle changes in illumination induce jitter on the estimated pose. Jitter impairs robustness in vision and robotics applications, and deteriorates the sense of presence and immersion in AR/VR applications. In this paper, we propose STag, a fiducial marker system that provides stable pose estimation. STag is designed to be robust against jitter factors, thus sustains pose stability better than the existing solutions. This is achieved by utilizing geometric features that can be localized more repeatably. The outer square border of the marker is used for detection and homography estimation. This is followed by a novel homography refinement step using the inner circular border. After refinement, the pose can be estimated stably and robustly across viewing conditions. These features are demonstrated with a comprehensive set of experiments, including comparisons with the state of the art fiducial marker systems.     

Digital techniques to mitigate feedback impairment and noise of look‐up table digital predistortion

In this article, some new effects of feedback impairment and noise on look‐up table (LUT) digital predistortion (DPD) are presented. Several digital techniques are proposed to mitigate these effects. A smoothing filter (SMF) for LUTs is used to eliminate fluctuations of the transfer function of the overall DPD amplification system. By combining SMF method with iterative average (IA) method, the LUTs iterative process becomes stable and well converged. A postcompensator is established according to the proposed two‐box model for feedback impairment. For the demonstration, both simulation and experiments are carried out based on a Hammerstein‐type PA. Simulation results give some preliminary cognition of these new effects and the effectiveness of proposed techniques. Experimental tests are performed on an S‐band amplifier excited with single‐carrier WCDMA signal. The adjacent channel power ratio (ACPR) at 5‐MHz offset is ?48 dBc after DPD with SMF method and postcompensator. A total of 8 dB extra improvement of ACPR is obtained compared with that with neither SMF nor postcompensator. The result clearly shows that the proposed digital techniques are qualified for LUT DPD system, especially when it suffers significant feedback impairment and noise. © 2011 Wiley Periodicals, Inc. Int J RF and Microwave CAE, 2011.     

SoftPOSIT: Simultaneous Pose and Correspondence Determination

The problem of pose estimation arises in many areas of computer vision, including object recognition, object tracking, site inspection and updating, and autonomous navigation when scene models are available. We present a new algorithm, called SoftPOSIT, for determining the pose of a 3D object from a single 2D image when correspondences between object points and image points are not known. The algorithm combines the iterative softassign algorithm (Gold and Rangarajan, 1996; Gold et al., 1998) for computing correspondences and the iterative POSIT algorithm (DeMenthon and Davis, 1995) for computing object pose under a full-perspective camera model. Our algorithm, unlike most previous algorithms for pose determination, does not have to hypothesize small sets of matches and then verify the remaining image points. Instead, all possible matches are treated identically throughout the search for an optimal pose. The performance of the algorithm is extensively evaluated in Monte Carlo simulations on synthetic data under a variety of levels of clutter, occlusion, and image noise. These tests show that the algorithm performs well in a variety of difficult scenarios, and empirical evidence suggests that the algorithm has an asymptotic run-time complexity that is better than previous methods by a factor of the number of image points. The algorithm is being applied to a number of practical autonomous vehicle navigation problems including the registration of 3D architectural models of a city to images, and the docking of small robots onto larger robots.     

Fast and robust absolute camera pose estimation with known focal length

Some 3D computer vision techniques such as structure from motion (SFM) and augmented reality (AR) depend on a specific perspective-n-point (PnP) algorithm to estimate the absolute camera pose. However, existing PnP algorithms are difficult to achieve a good balance between accuracy and efficiency, and most of them do not make full use of the internal camera information such as focal length. In order to attack these drawbacks, we propose a fast and robust PnP (FRPnP) method to calculate the absolute camera pose for 3D compute vision. In the proposed FRPnP method, we firstly formulate the PnP problem as the optimization problem in the null space that can avoid the effects of the depth of each 3D point. Secondly, we can easily get the solution by the direct manner using singular value decomposition. Finally, the accurate information of camera pose can be obtained by optimization strategy. We explore four ways to evaluate the proposed FRPnP algorithm with synthetic dataset, real images, and apply it in the AR and SFM system. Experimental results show that the proposed FRPnP method can obtain the best balance between computational cost and precision, and clearly outperforms the state-of-the-art PnP methods.

     

Pose Estimation using Point and Line Correspondences

The problem of a real-time pose estimation between a 3D scene and a single camera is a fundamental task in most 3D computer vision and robotics applications such as object tracking, visual servoing, and virtual reality. In this paper we present two fast methods for estimating the 3D pose using 2D to 3D point and line correspondences. The first method is based on the iterative use of a weak perspective camera model and forms a generalization of DeMenthon's method (1995) which consists of determining the pose from point correspondences. In this method the pose is iteratively improved with a weak perspective camera model and at convergence the computed pose corresponds to the perspective camera model. The second method is based on the iterative use of a paraperspective camera model which is a first order approximation of perspective. We describe in detail these two methods for both non-planar and planar objects. Experiments involving synthetic data as well as real range data indicate the feasibility and robustness of these two methods. We analyse the convergence of these methods and we conclude that the iterative paraperspective method has better convergence properties than the iterative weak perspective method. We also introduce a non-linear optimization method for solving the pose problem.     

Iterative Pose Estimation Using Coplanar Feature Points

This paper presents a new method for the computation of the position and orientation of a camera with respect to a known object, using four or morecoplanarfeature points. Starting with the scaled orthographic projection approximation, this method iteratively refines up to two different pose estimates, and provides an associated quality measure for each pose. When the camera distance is large compared with the object depth, or when the accuracy of feature point extraction is low because of image noise, the quality measures for the two poses are similar, and the two pose estimates are plausible interpretations of the available information. In contrast, known methods using a closed form pose solution for four coplanar points are not robust for distant objects in the presence of image noise because they provide only one of the two possible poses and may choose the wrong pose.     

未标定相机的高精度位姿估计方法

针对未标定相机的位姿估计问题,提出了一种焦距和位姿同时迭代的高精度位姿估计算法。现有的未标定相机的位姿估计算法是焦距和相机位姿单独求解,焦距估计精度较差。提出的算法首先通过现有算法得到相机焦距和位姿的初始参数;然后在正交迭代的基础上推导了焦距和位姿最小化函数,将焦距和位姿同时作为初始值进行迭代计算;最后得到高精度的焦距和位姿参数。仿真实验表明提出的算法在点数为10,噪声标准差为2的情况下,角度相对误差小于1%,平移相对误差小于4%,焦距相对误差小于3%;真实实验表明提出的算法与棋盘标定方法的精度相当。与现有算法相比,能够对未标定相机进行高精度的焦距和位姿估计。     

Estimating the generalized autoregression model parameters for unknown noise distribution

We solve the problem of estimating the autoregressive parameters of a nonlinear stable stochastic process with discrete time of the AR(p)/ARCH(p) type with unknown ARCH(p) process parameters. For the AR(1)/ARCH(1) model, we solve the estimation problem for all unknown process parameters, i.e., the autoregression parameter and two parameters of the noise process ARCH(1). We assume that the noise distributions are unknown. We show that the least square estimates are strongly consistent.     

Pose estimation from multiple cameras based on Sylvester’s equation

In this paper, we introduce a method to estimate the object’s pose from multiple cameras. We focus on direct estimation of the 3D object pose from 2D image sequences. Scale-Invariant Feature Transform (SIFT) is used to extract corresponding feature points from adjacent images in the video sequence. We first demonstrate that centralized pose estimation from the collection of corresponding feature points in the 2D images from all cameras can be obtained as a solution to a generalized Sylvester’s equation. We subsequently derive a distributed solution to pose estimation from multiple cameras and show that it is equivalent to the solution of the centralized pose estimation based on Sylvester’s equation. Specifically, we rely on collaboration among the multiple cameras to provide an iterative refinement of the independent solution to pose estimation obtained for each camera based on Sylvester’s equation. The proposed approach to pose estimation from multiple cameras relies on all of the information available from all cameras to obtain an estimate at each camera even when the image features are not visible to some of the cameras. The resulting pose estimation technique is therefore robust to occlusion and sensor errors from specific camera views. Moreover, the proposed approach does not require matching feature points among images from different camera views nor does it demand reconstruction of 3D points. Furthermore, the computational complexity of the proposed solution grows linearly with the number of cameras. Finally, computer simulation experiments demonstrate the accuracy and speed of our approach to pose estimation from multiple cameras.     

Critical Issues on Kalman Filter with Colored and Correlated System Noises

The Kalman filtering (KF) is optimal under the assumption that both process and observation noises are independent white Gaussian noise. However, this assumption is not always satisfied in real‐world navigation campaigns. In this paper, two types of KF methods are investigated, i.e. augmented KF (AKF) and the second moment information based KF (SMIKF) with colored system noises, including process and observation noises. As a popular noise‐whitening method, the principle of AKF is briefly reviewed for dealing with the colored system noises. The SMIKF method is developed for the colored and correlated system noises, which directly compensates for the covariance through stochastic model in the sense of minimum mean square error. To accurately implement the SMIKF, a refined SMIKF is further derived regarding the continuous‐time dynamic model rather than the discrete one. The computational burdens of the proposed SMIKF along with representative methods are analyzed and compared. The simulation results demonstrate the performances of proposed methods.     

Robust camera pose tracking for augmented reality using particle filtering framework

In this paper, we present new solutions for the problem of estimating the camera pose using particle filtering framework. The proposed approach is suitable for real-time augmented reality (AR) applications in which the camera is held by the user. This work demonstrates that particle filtering improve the robustness of the tracking comparing to existing approaches, such as those based on the Kalman filter. We propose a tracking framework for both points and lines features, the particle filter is used to compute the posterior density for the camera 3D motion parameters. We also analyze the sensitivity of our technique when outliers are present in the match data. Outliers arise frequently due to incorrect correspondences which occur because of either image noise or occlusion. Results from real data in an augmented reality setup are then presented, demonstrating the efficiency and robustness of the proposed method.     

Map Building Fusing Acoustic and Visual Information using Autonomous Underwater Vehicles

We present a system for automatically building three‐dimensional (3‐D) maps of underwater terrain fusing visual data from a single camera with range data from multibeam sonar. The six‐degree‐of‐freedom location of the camera relative to the navigation frame is derived as part of the mapping process, as are the attitude offsets of the multibeam head and the onboard velocity sensor. The system uses pose graph optimization and the square root information smoothing and mapping framework to simultaneously solve for the robot's trajectory, the map, and the camera location in the robot's frame. Matched visual features are treated within the pose graph as images of 3‐D landmarks, while multibeam bathymetry submap matches are used to impose relative pose constraints linking robot poses from distinct tracklines of the dive trajectory. The navigation and mapping system presented works under a variety of deployment scenarios on robots with diverse sensor suites. The results of using the system to map the structure and the appearance of a section of coral reef are presented using data acquired by the Seabed autonomous underwater vehicle.     

基于移动增强现实的地震科普馆导览系统

增强现实技术是一种将虚拟信息无缝融合到真实世界的新技术,近年来受到国内外研究者的广泛关注。随着便携性、智能化移动设备的快速发展,增强现实技术越来越多地应用到移动设备上,具有很好的发展前景。以识别特定标志物为目的,研究并实现了一个基于移动增强现实的地震科普馆导览系统。利用Wi-Fi定位和图像处理对标志物进行检测识别;利用标志物的四个顶点,通过确定坐标系之间的变换关系,计算摄像机位姿,实现摄像机跟踪定位;通过坐标系之间的变换,确定虚拟信息在成像平面上的位置,实现虚拟信息的注册。测试结果表明,该系统能够实现地震科普馆导览功能,达到了预期的研究目标。     

Linear pose estimation from points or lines

Estimation of camera pose from an image of n points or lines with known correspondence is a thoroughly studied problem in computer vision. Most solutions are iterative and depend on nonlinear optimization of some geometric constraint, either on the world coordinates or on the projections to the image plane. For real-time applications, we are interested in linear or closed-form solutions free of initialization. We present a general framework which allows for a novel set of linear solutions to the pose estimation problem for both n points and n lines. We then analyze the sensitivity of our solutions to image noise and show that the sensitivity analysis can be used as a conservative predictor of error for our algorithms. We present a number of simulations which compare our results to two other recent linear algorithms, as well as to iterative approaches. We conclude with tests on real imagery in an augmented reality setup.     

Data‐based iterative learning mechanism for unknown input‐output coupling parameters/matrices

In this paper, we explore how to get the information of input‐output coupling parameters (IOCPs) for a class of uncertain discrete‐time systems by using iterative learning technique. Firstly, by taking advantage of repetitiveness of control system and informative input and output data, we design an iterative learning scheme for unknown IOCPs. It is shown that we can get the exact values of IOCPs one by one through running the repetitive system T+1 times if the control system is with identical initial state and noise free. Secondly, we give the iterative learning scheme for unknown IOCPs in the presence of measurement noise, system noise, or initial state drift and analyze the influence factors on the performance of developed iterative learning scheme. Meanwhile, we introduce the maximum allowable control deviation into the iterative learning mechanism to minimize the negative impact of noise on the performance of learning scheme and to enhance the robust of iterative learning scheme. Thirdly, for a class of multiple‐input–multiple‐output systems, we also develop iterative learning mechanism for unknown input‐output coupling matrices. Finally, an illustrative example is given to demonstrate the effectiveness of proposed iterative learning scheme.     

Way-Finder: guided tours through complex walkthrough models

The exploration of complex walkthrough models is often a difficult task due to the presence of densely occluded regions which pose a serious challenge to online navigation. In this paper we address the problem of algorithmic generation of exploration paths for complex walkthrough models. We present a characterization of suitable properties for camera paths and we discuss an efficient algorithm for computing them with little or no user intervention. Our approach is based on identifying the free‐space structure of the scene (represented by a cell and portal graph) and an entropy‐based measure of the relevance of a view‐point. This metric is key for deciding which cells have to be visited and for computing critical way‐points inside each cell. Several results on different model categories are presented and discussed.     

A dual‐patch polarization rotation reflective surface and its application to wideband wide‐beam low‐profile circularly polarized patch antennas

This study investigates the use of a polarization rotation reflective surface (PRRS) to construct a wideband, wide‐beam, low‐profile circularly polarized (CP) patch antenna. The device is composed of a feeding monopole antenna and a novel PRRS‐based dual‐patch artificial magnetic conductor (AMC) cell structure. The PRRS has two polarization rotation (PR) frequency points, generated by properly adjusting the width between square and L‐shaped metallic patches. A large PR band of 35.5% (5.1‐7.3 GHz) was achieved by combining two adjacent PR frequency points. The PRRS‐based patch antenna impedance bandwidth was measured to be 28.6% (5.1‐6.35 GHz), with a 3 dB axial ratio (AR) bandwidth of 21.8% (4.8‐6.4 GHz) and a profile of 0.045λ₀. Additionally, the proposed antenna exhibited the largest AR beamwidth (to our knowledge) of 175° and 128° in the xoz and yoz planes, respectively. It also produced a high broadside gain of 6.7 dBic within the operational bandwidth.     

Robust camera pose and scene structure analysis for service robotics

Successful path planning and object manipulation in service robotics applications rely both on a good estimation of the robot’s position and orientation (pose) in the environment, as well as on a reliable understanding of the visualized scene. In this paper a robust real-time camera pose and a scene structure estimation system is proposed. First, the pose of the camera is estimated through the analysis of the so-called tracks. The tracks include key features from the imaged scene and geometric constraints which are used to solve the pose estimation problem. Second, based on the calculated pose of the camera, i.e. robot, the scene is analyzed via a robust depth segmentation and object classification approach. In order to reliably segment the object’s depth, a feedback control technique at an image processing level has been used with the purpose of improving the robustness of the robotic vision system with respect to external influences, such as cluttered scenes and variable illumination conditions. The control strategy detailed in this paper is based on the traditional open-loop mathematical model of the depth estimation process. In order to control a robotic system, the obtained visual information is classified into objects of interest and obstacles. The proposed scene analysis architecture is evaluated through experimental results within a robotic collision avoidance system.