Visual motion ambiguities of a plane in 2-D FS sonar motion sequences

Sonar is the most common imaging modality in underwater, and high-resolution high data rate 2-D video systems have been emerging in recent years. As for visually guided terrestrial robot navigation and target-based positioning, the estimation of 3-D motion by tracking features in recorded 2-D sonar images is also a highly desirable capability for submersible platforms. Additionally, theoretical results dealing with robustness and multiplicity of solution constitute important fundamental findings due to nature of sonar data, namely, high noise level, narrow field of view coverage, scarcity of robust features, and incorrect matches.This paper explores the inherent ambiguities of 3-D motion and scene structure interpretation from 2-D forward-scan sonar image sequences. Analyzing the sonar image motion transformation model, which depends on the affine components of the projective transformation (or homography) of two plane views, we show that two interpretations are commonly inferred. The true and spurious planes form mirror images relative to the zero-elevation plane of the sonar reference frame. Even under each of pure rotation or translation, a spurious motion exists comprising both translational and rotational components. In some cases, the two solutions share certain motion components, where the imaged surface becomes parallel to a plane defined by two of the sonar coordinate axes. A unique solution exists under the very special condition where the sonar motion aligns the imaged plane with the zero-elevation planes. We also derive the relationship between the two interpretations, thus allowing closed-form computation of both solutions.     

Epipolar geometry of opti-acoustic stereo imaging

Optical and acoustic cameras are suitable imaging systems to inspect underwater structures, both in regular maintenance and security operations. Despite high resolution, optical systems have limited visibility range when deployed in turbid waters. In contrast, the new generation of high-frequency (MHz) acoustic cameras can provide images with enhanced target details in highly turbid waters, though their range is reduced by one to two orders of magnitude compared to traditional low-/midfrequency (10s-100s KHz) sonar systems. It is conceivable that an effective inspection strategy is the deployment of both optical and acoustic cameras on a submersible platform, to enable target imaging in a range of turbidity conditions. Under this scenario and where visibility allows, registration of the images from both cameras arranged in binocular stereo configuration provides valuable scene information that cannot be readily recovered from each sensor alone. We explore and derive the constraint equations for the epipolar geometry and stereo triangulation in utilizing these two sensing modalities with different projection models. Theoretical results supported by computer simulations show that an opti-acoustic stereo imaging system outperforms a traditional binocular vision with optical cameras, particularly for increasing target distance and (or) turbidity.     

基于切片图像的三维声纳目标识别

研究了为水下机器人声视觉设计的一套三维成像声纳系统的目标识别方法,该声纳可以在一次发射中获取一系列二维切片图像。根据面阵水平、垂直分辨率的不同和图像中目标的切片数量,结合水声环境对水下成像的影响,分析了几种不同目标切片图像的特点,提出一种以直方图中目标能量为特征的识别方法,该方法在算法实现上避免了三维重建的步骤,使识别速度显著提高,在对水池数据和海试数据的分类识别试验中证明其能有效地识别水下目标。     

3-D moment forms: their construction and application to objectidentification and positioning

The 3-D moment method is applied to object identification and positioning. A general theory of deriving 3-D moments invariants is proposed. The notion of complex moments is introduced. Complex moments are defined as linear combinations of moments with complex coefficients and are collected into multiplets such that each multiplet transforms irreducibly under 3-D rotations. The application of the 3-D moment method to motion estimation is also discussed. Using group-theoretic techniques, various invariant scalars are extracted from compounds of complex moments via Clebsch-Gordon expansion. Twelve moment invariants consisting of the second-order and third-order moments are explicitly derived. Based on a perturbation formula, it is shown that the second-order moment invariants can be used to predict whether the estimation using noisy data is reliable or not. The new derivation of vector forms also facilities the calculation of motion estimation in a tensor approach. Vectors consisting of the third-order moments can be derived in a similar manner     

Vision-based 3-D trajectory tracking for unknown environments

This paper describes a vision-based system for 3-D localization of a mobile robot in a natural environment. The system includes a mountable head with three on-board charge-coupled device cameras that can be installed on the robot. The main emphasis of this paper is on the ability to estimate the motion of the robot independently from any prior scene knowledge, landmark, or extra sensory devices. Distinctive scene features are identified using a novel algorithm, and their 3-D locations are estimated with high accuracy by a stereo algorithm. Using new two-stage feature tracking and iterative motion estimation in a symbiotic manner, precise motion vectors are obtained. The 3-D positions of scene features and the robot are refined by a Kalman filtering approach with a complete error-propagation modeling scheme. Experimental results show that good tracking and localization can be achieved using the proposed vision system.     

Silhouette-based 3-D model reconstruction from multiple images

The goal of this study is to investigate the reconstruction of three-dimensional (3-D) graphical models of real objects in a controlled imaging environment and present the work done in our group based on silhouette-based reconstruction. Although many parts of the whole system have been well-known in the literature and in practice, the main contribution of the paper is that it describes a complete, end-to-end system explained in detail. Based on a multi-image calibration method, an algorithm to extract the rotation axis of a turn-table has been developed. Furthermore, this can be extended to estimate robustly the initial bounding volume of the object to be modeled. The disadvantages of the silhouette-based reconstruction can be removed by an algorithm using photoconsistency. This algorithm has a simpler visibility check, and it eliminates the selection of threshold existing in similar algorithms. Besides, in order to construct the appearance, we use the concept of particles. The reconstruction results are shown both on real world and synthetic objects.     

On 3-D scene flow and structure recovery from multiview image sequences

Two novel systems computing dense three-dimensional (3-D) scene flow and structure from multiview image sequences are described in this paper. We do not assume rigidity of the scene motion, thus allowing for nonrigid motion in the scene. The first system, integrated model-based system (IMS), assumes that each small local image region is undergoing 3-D affine motion. Non-linear motion model fitting based on both optical flow constraints and stereo constraints is then carried out on each local region in order to simultaneously estimate 3-D motion correspondences and structure. The second system is based on extended gradient-based system (EGS), a natural extension of two-dimensional (2-D) optical flow computation. In this method, a new hierarchical rule-based stereo matching algorithm is first developed to estimate the initial disparity map. Different available constraints under a multiview camera setup are further investigated and utilized in the proposed motion estimation. We use image segmentation information to adopt and maintain the motion and depth discontinuities. Within the framework for EGS, we present two different formulations for 3-D scene flow and structure computation. One formulation assumes that initial disparity map is accurate, while the other does not. Experimental results on both synthetic and real imagery demonstrate the effectiveness of our 3-D motion and structure recovery schemes. Empirical comparison between IMS and EGS is also reported.     

New 3-D display that can display 3-D images at long distances and that can control their 3-D positions using changing size as a cue to depth perception

This paper presents a new three-dimensional (3-D) display that can display 3-D images at long distances of tens or hundreds of meters in the depth direction and that can control their 3-D positions to meet new requirements for outdoor use. The proposed display uses changing size as a cue to depth perception, i.e., the smoothly expanding motion of virtual images formed with optical systems according to the forward movements of the users to display 3-D images at more distant positions in the depth direction than positions where virtual images are formed with optical systems because conventional 3-D displays that use binocular disparity are only able to display 3-D images at short distances in the depth direction. The feasibility of the proposed display was evaluated by subjective tests using a moving minivan in which observers viewed a test pattern that overlapped the real view ahead of the automobile observed through the windshield. The results obtained from the subjective tests revealed that the test pattern was observed at long distances over tens and hundreds of meters in the depth direction and that the position in the depth direction of the test pattern could be controlled by changing the rate at which the motion of the test pattern smoothly expanded. These results demonstrated that the proposed display was feasible.     

Tracking 3-D motion from straight lines with trifocal tensors

We present a novel approach to track the position and orientation of a stereo camera using line features in the images. The method combines the strengths of trifocal tensors and Bayesian filtering. The trifocal tensor provides a geometric constraint to lock line features among every three frames. It eliminates the explicit reconstruction of the scene even if the 3-D scene structure is not known. Such a trifocal constraint thus makes the algorithm fast and robust. The twist motion model is applied to further improve its computation efficiency. Another major contribution is that our approach can obtain the 3-D camera motion using as little as 2 line correspondences instead of 13 in the traditional approaches. This makes the approach attractive for realistic applications. The performance of the proposed method has been evaluated using both synthetic and real data with encouraging results. Our algorithm is able to estimate 3-D camera motion in real scenarios accurately having little drifting from an image sequence longer than a 1,000 frames.     

用二维点对应数据分割和估计多刚体运动

已知含有多个三维刚体的场景,在运动前后的二维点对应数据集合,其中可以包含高斯噪声和出格点数据,发展了初始部分匹配的生成-生长技术并运用刚性约束.将上述二维点对应数据集合.分割成多个分别对应于不同刚体运动的二维点对应数据子集,并能分离出所有出格点数据.再利用单刚体运动估计算法就可估计出各个刚体运动参数.实验结果表明了算法的有效性.     

三维传感网空间RSS与AOA混合测量的精确定位方法

由于位置坐标参数的增加,三维传感网空间的定位难度较二维平面有所增大.单一的依靠接收信号强度(RSS)确定节点位置坐标的方法将使定位的不确定性增加,定位误差也较大.新型的阵列与智能天线的出现为节点间的到达角度(AOA)测量提供了方便,为此本文提出了一种三维传感网空间RSS与AOA混合测量的精确定位方法.将采用混合测量建立的非线性优化模型转化为线性方程,分别提出了节点位置坐标估计的非约束线性最小二乘(ULLS)及约束线性最小二乘(CLLS)方法.仿真测试了所设计算法的有效性,分析了不同测量噪声对位置坐标估计误差的影响.仿真表明所设计的ULLS和CLLS方法的计算速度快,相比于ULLS方法,采用约束后的CLLS方法的定位误差更小.在较小测量噪声范围内,ULLS和CLLS估计方法具有较高的稳定性和定位精度.     

3-D Tag-Based RFID System for Recognition of Object

Generally, object recognition in robotics has used vision, sonar, laser range sensors, etc. In this paper, the authors propose a new sensing technology based on radio frequency identification (RFID) to facilitate the autonomous recognition of objects. This technology uses a novel tag, named the 3-D tag, with which an object can be identified and its orientation together with location can be estimated. This paper presents the method for object recognition by analyzing the characteristics of the 3-D tag, and then validates the 3-D tag through several demonstrations showing its usefulness as an independent sensor for recognizing objects.     

Three-dimensional motion estimation via matrix completion

Three-dimensional motion estimation from multiview video sequences is of vital importance to achieve high-quality dynamic scene reconstruction. In this paper, we propose a new 3-D motion estimation method based on matrix completion. Taking a reconstructed 3-D mesh as the underlying scene representation, this method automatically estimates motions of 3-D objects. A "separating + merging" framework is introduced to multiview 3-D motion estimation. In the separating step, initial motions are first estimated for each view with a neighboring view. Then, in the merging step, the motions obtained by each view are merged together and optimized by low-rank matrix completion method. The most accurate motion estimation for each vertex in the recovered matrix is further selected by three spatiotemporal criteria. Experimental results on data sets with synthetic motions and real motions show that our method can reliably estimate 3-D motions.     

无对应点三维运动参灵敏估计

本文提出了一种不依赖于对应点数据的三维运动参数估计算法,可以处理包含高以处理包含高斯噪声和出格点(outlier)的观察数据.首先,我们构造一个代价函数来表达三维点集间的部分匹配,将运动参数估计问题转化为一个全局最优化问题,求最大部分匹配.在优化过程中,我们采用了函数逼近技术.大量计算机模拟实验结果充分证明了算法的有效性和鲁棒性.     

Dense 3-D Reconstruction of an Outdoor Scene by Hundreds-Baseline Stereo Using a Hand-Held Video Camera

Three-dimensional (3-D) models of outdoor scenes are widely used for object recognition, navigation, mixed reality, and so on. Because such models are often made manually with high costs, automatic 3-D reconstruction has been widely investigated. In related work, a dense 3-D model is generated by using a stereo method. However, such approaches cannot use several hundreds images together for dense depth estimation because it is difficult to accurately calibrate a large number of cameras. In this paper, we propose a dense 3-D reconstruction method that first estimates extrinsic camera parameters of a hand-held video camera, and then reconstructs a dense 3-D model of a scene. In the first process, extrinsic camera parameters are estimated by tracking a small number of predefined markers of known 3-D positions and natural features automatically. Then, several hundreds dense depth maps obtained by multi-baseline stereo are combined together in a voxel space.So, we can acquire a dense 3-D model of the outdoor scene accurately by using several hundreds input images captured by a hand-held video camera.     

基于WebGL的医学图像三维可视化研究

现今医学图像往往与大型的硬件设备和复杂的软件联系在一起, 然而随着互联网技术的发展, 越来越多互联网应用的出现改变了人们对传统本地软件的依赖, 现今医学图像在互联网领域才刚刚起步, 提出了一种在浏览器中实现医学图像的三维可视化的方法, 能够通过成熟的本地医学图像平台(比如3DSlicer)获取医学图像数据, 结合HTML5以及WebGL(Web Graphics Library)来实现医学图像的三维可视化.     

Integrated 3-D analysis and analysis-guided synthesis of flightimage sequences

This paper is concerned with three-dimensional (3D) analysis, and analysis-guided syntheses, of images showing 3-D motion of an observer relative to a scene. There are two objectives of the paper. First, it presents an approach to recovering 3D motion and structure parameters from multiple cues present in a monocular image sequence, such as point features, optical flow, regions, lines, texture gradient, and vanishing line. Second, it introduces the notion that the cues that contribute the most to 3-D interpretation are also the ones that would yield the most realistic synthesis, thus suggesting an approach to analysis guided 3-D representation. For concreteness, the paper focuses on flight image sequences of a planar, textured surface. The integration of information in these diverse cues is carried out using optimization. For reliable estimation, a sequential batch method is used to compute motion and structure. Synthesis is done by using (i) image attributes extracted from the image sequence, and (ii) simple, artificial image attributes which are not present in the original images. For display, real and/or artificial attributes are shown as a monocular or a binocular sequence. Performance evaluation is done through experiments with one synthetic sequence, and two real image sequences digitized from a commercially available video tape and a laserdisc. The attribute based representation of these sequences compressed their sizes by 502 and 367. The visualization sequence appears very similar to the original sequence in informal, monocular as well as stereo viewing on a workstation monitor     

Estimating the principal curvatures and the Darboux frame from real 3-D range data

Principal curvatures and the local Darboux frame are natural tools to be used during processes which involve extraction of geometric properties from three-dimensional (3-D) range data. As second-order features their estimations are highly sensitive to noise and therefore, until recent years, it was almost impractical to extract reliable results from real 3-D data. Since the use of more accurate 3-D range imaging equipment has become more popular, as well as the use of polyhedral meshes to approximate surfaces, evaluation of existing algorithms for curvature estimation is again relevant. The work presented here, makes some subtle but very important modifications to two such algorithms, originally suggested by Taubin (1995) and Chen and Schmitt (1992). The algorithms have been adjusted to deal with real discrete noisy range data, given as a cloud of sampled points, lying on surfaces of free-form objects. The results of this linear time (and space) complexity implementation were evaluated in a series of tests on synthetic and real input. We also present one of many possible uses for these extracted features in an efficient and robust application for the recovery of 3-D geometric primitives from range data of complex scenes. The application combines the segmentation, classification and fitting processes in a single process which advances monotonously through the recovery procedure. It is also very robust and does not use any least-squares fittings. The conclusion of this study is that with current scanning technology and the algorithms presented here, reliable estimates of the principal curvatures and Darboux frame can be extracted from real data and used in a large variety of tasks.     

Fusion of range camera and photogrammetry: a systematic procedure for improving 3-D models metric accuracy

The generation of three-dimensional (3-D) digital models produced by optical technologies in some cases involves metric errors. This happens when small high-resolution 3-D images are assembled together in order to model a large object. In some applications, as for example 3-D modeling of Cultural Heritage, the problem of metric accuracy is a major issue and no methods are currently available for enhancing it. The authors present a procedure by which the metric reliability of the 3-D model, obtained through iterative alignments of many range maps, can be guaranteed to a known acceptable level. The goal is the integration of the 3-D range camera system with a close range digital photogrammetry technique. The basic idea is to generate a global coordinate system determined by the digital photogrammetric procedure, measuring the spatial coordinates of optical targets placed around the object to be modeled. Such coordinates, set as reference points, allow the proper rigid motion of few key range maps, including a portion of the targets, in the global reference system defined by photogrammetry. The other 3-D images are normally aligned around these locked images with usual iterative algorithms. Experimental results on an anthropomorphic test object, comparing the conventional and the proposed alignment method, are finally reported.