点状特征柔性物体三维运动捕获方法

针对具有点状特征的柔性物体,提出了一种三维运动捕获方法.首先,该方法利用两个标定的高速摄像机拍摄柔性物体的运动视频,并对图像进行立体校正;然后,采用DOG （Difference Of Gaussian）算法获取点状特征的位置,并提取特征点极值;其次,在一定范围的窗口上搜索匹配对,匹配左右图像的特征点;再次,通过三角测量法进行三维重建;最后,利用搜索策略进行时间序列上的匹配,实现动态柔性物体的三维运动捕获,并计算空间坐标、速度、加速度参数.实验结果表明,相比于采用sift算法匹配特征点捕获柔性运动物体的方法,本方法精度更高.     

Pose estimation from multiple cameras based on Sylvester’s equation

In this paper, we introduce a method to estimate the object’s pose from multiple cameras. We focus on direct estimation of the 3D object pose from 2D image sequences. Scale-Invariant Feature Transform (SIFT) is used to extract corresponding feature points from adjacent images in the video sequence. We first demonstrate that centralized pose estimation from the collection of corresponding feature points in the 2D images from all cameras can be obtained as a solution to a generalized Sylvester’s equation. We subsequently derive a distributed solution to pose estimation from multiple cameras and show that it is equivalent to the solution of the centralized pose estimation based on Sylvester’s equation. Specifically, we rely on collaboration among the multiple cameras to provide an iterative refinement of the independent solution to pose estimation obtained for each camera based on Sylvester’s equation. The proposed approach to pose estimation from multiple cameras relies on all of the information available from all cameras to obtain an estimate at each camera even when the image features are not visible to some of the cameras. The resulting pose estimation technique is therefore robust to occlusion and sensor errors from specific camera views. Moreover, the proposed approach does not require matching feature points among images from different camera views nor does it demand reconstruction of 3D points. Furthermore, the computational complexity of the proposed solution grows linearly with the number of cameras. Finally, computer simulation experiments demonstrate the accuracy and speed of our approach to pose estimation from multiple cameras.     

Circular camera array system for 3‐D displays based on the reconstruction of parallax rays

Abstract— A circular camera system employing an image‐based rendering technique that captures light‐ray data needed for reconstructing three‐dimensional (3‐D) images by using reconstruction of parallax rays from multiple images captured from multiple viewpoints around a real object in order to display a 3‐D image of a real object that can be observed from multiple surrounding viewing points on a 3‐D display is proposed. An interpolation algorithm that is effective in reducing the number of component cameras in the system is also proposed. The interpolation and experimental results which were performed on our previously proposed 3‐D display system based on the reconstruction of parallax rays will be described. When the radius of the proposed circular camera array was 1100 mm, the central angle of the camera array was 40°, and the radius of a real 3‐D object was between 60 and 100 mm, the proposed camera system, consisting of 14 cameras, could obtain sufficient 3‐D light‐ray data to reconstruct 3‐D images on the 3‐D display.     

基于参考点配准法的全景泊车系统研究

为了提高驾驶行车的安全性,研究并改进了一种全景泊车系统,由鱼眼摄像头,图像处理单元和显示器三大部分组成.鱼眼摄像头采集汽车四周的超广角图像;在OpenCV环境下通过鱼眼摄像头标定,桶型矫正和图像的透视变换技术形成汽车前后左右的4幅平面俯视图;提出了一种参考点配准的方法,可以根据平面俯视图得到汽车的360°全景图像,并在显示器上显示.与特征点匹配法相比,参考点配准方法受环境的影响很小且对硬件性能要求较低,因此,更适用于嵌入式系统实现.     

Uncalibrated Motion Capture Exploiting Articulated Structure Constraints

We present an algorithm for 3D reconstruction of dynamic articulated structures, such as humans, from uncalibrated multiple views. The reconstruction exploits constraints associated with a dynamic articulated structure, specifically the conservation over time of length between rotational joints. These constraints admit reconstruction of metric structure from at least two different images in each of two uncalibrated parallel projection cameras. As a by product, the calibration of the cameras can also be computed. The algorithm is based on a stratified approach, starting with affine reconstruction from factorization, followed by rectification to metric structure using the articulated structure constraints. The exploitation of these specific constraints admits reconstruction and self-calibration with fewer feature points and views compared to standard self-calibration. The method is extended to pairs of cameras that are zooming, where calibration of the cameras allows compensation for the changing scale factor in a scaled orthographic camera. Results are presented in the form of stick figures and animated 3D reconstructions using pairs of sequences from broadcast television. The technique shows promise as a means of creating 3D animations of dynamic activities such as sports events.     

RGBD融合明暗恢复形状的全视角三维重建技术研究

为了高效、高精度、低成本地实现对物体的全视角三维重建, 提出一种使用深度相机融合光照约束实现全视角三维重建的方法。该重建方法中,在进行单帧重建时采用RGBD深度图像融合明暗恢复形状(Shape from shading,SFS)的重建方法, 即在原有的深度数据上加上额外的光照约束来优化深度值; 在相邻两帧配准时, 采用快速点特征直方图（Fast point feature histograms, FPFH）特征进行匹配并通过随机采样一致性（Random sample consensus, RANSAC）滤除错误的匹配点对求解粗配准矩阵, 然后通过迭代最近点（Iterative closest point, ICP）算法进行精配准得出两帧间的配准矩阵; 在进行全视角的三维重建时, 采用光束平差法优化相机位姿, 从而消除累积误差使首尾帧完全重合, 最后融合生成一个完整的模型。该方法融入了物体表面的光照信息,因此生成的三维模型更为光顺,也包含了更多物体表面的细节信息,提高了重建精度;同时该方法仅通过单张照片就能在自然光环境下完成对多反射率三维物体的重建,适用范围更广。本文方法的整个实验过程通过手持深度相机就能完成,不需要借助转台,操作更加方便。     

一种基于特征匹配的目标入侵检测方法

视频监控中常用云台摄像机监控视场较大的区域.对于云台摄像机跟随拍摄的情况,提出了一种基于特征匹配的目标入侵检测方法.通过提取的尺度不变特征变换(SIFT)特征点对,将当前图像和全景图像进行匹配,从而得到当前图像和全景图像投影关系,再将当前图像的坐标系变换到全景图像下,最后运用差分法,找到入侵目标.实验结果表明,即使当前图像与全景图像存在尺度、缩放、形变等差异,通过本方法也可正确地检测出入侵目标.     

Geometric techniques for 3D tracking of ultrasound sensor,tumor segmentation in ultrasound images,and 3D reconstruction

This paper presents different methods, some based on geometric algebra, for ultrasound probe tracking in endoscopic images, 3D allocation of the ultrasound probe, ultrasound image segmentation (to extract objects like tumors), and 3D reconstruction of the surface defined by a set of points. The tracking of the ultrasound probe in endoscopic images is done with a particle filter and an auxiliary method based on thresholding in the HSV space. The 3D pose of the ultrasound probe is calculated using conformal geometric algebra (to locate each slide in 3D space). Each slide (ultrasound image) is segmented using two methods: the level-set method and the morphological operators approach in order to obtain the object we are interested in. The points on the object of interest are obtained from the segmented ultrasound images, and then a 3D object is obtained by refining the convex hull. To do that, a peeling process with an adaptive radius is applied, all of this in the geometric algebra framework. Results for points from ultrasound images, as well as for points from objects from the AimatShape Project, are presented (A.I.M.A.T.S.H.A.P.E. – Advanced an Innovative Models And Tools for the development of Semantic-based systems for Handling, Acquiring, and Processing knowledge Embedded in multidimensional digital objects).     

Estimation of the fundamental matrix from uncalibrated stereo hand images for 3D hand gesture recognition

All 3D hand models employed for hand gesture recognition so far use kinematic models of the hand. We propose to use computer vision models of the hand, and recover hand gestures using 3D reconstruction techniques. In this paper, we present a new method to estimate the epipolar geometry between two uncalibrated cameras from stereo hand images. We first segmented hand images using the RCE neural network based color segmentation algorithm and extracted edge points of fingers as points of interest, then match them based on the topological features of the hand. The fundamental matrix is estimated using a combination of techniques such as input data normalization, rank-2 constraint, linear criterion, nonlinear criterion as well as M-estimator. This method has been tested with real calibrated and uncalibrated images. The experimental comparison demonstrates the effectiveness and robustness of the method.     

3D Scene Reconstruction from Multiple Spherical Stereo Pairs

We propose a 3D environment modelling method using multiple pairs of high-resolution spherical images. Spherical images of a scene are captured using a rotating line scan camera. Reconstruction is based on stereo image pairs with a vertical displacement between camera views. A 3D mesh model for each pair of spherical images is reconstructed by stereo matching. For accurate surface reconstruction, we propose a PDE-based disparity estimation method which produces continuous depth fields with sharp depth discontinuities even in occluded and highly textured regions. A full environment model is constructed by fusion of partial reconstruction from spherical stereo pairs at multiple widely spaced locations. To avoid camera calibration steps for all camera locations, we calculate 3D rigid transforms between capture points using feature matching and register all meshes into a unified coordinate system. Finally a complete 3D model of the environment is generated by selecting the most reliable observations among overlapped surface measurements considering surface visibility, orientation and distance from the camera. We analyse the characteristics and behaviour of errors for spherical stereo imaging. Performance of the proposed algorithm is evaluated against ground-truth from the Middlebury stereo test bed and LIDAR scans. Results are also compared with conventional structure-from-motion algorithms. The final composite model is rendered from a wide range of viewpoints with high quality textures.     

由单幅图像重构三轴试样三维变形

针对岩土三轴试验的土样变形测量需要进行三维测量这个难题,提出了新的由单幅图像进行三维重构方法--结构标记法,即利用三维试样标记的各点相对于摄像机的物距不同而造成成像位置改变的原理,从二维图像中提取出了三维信息,实现了三维重构.利用双目或多目图像进行三维重构是比较常见的方法.但是这种方法受到光照条件、噪声干扰、景物几何形状和畸变、表面物理特性以及摄像机特性等因素的严重干扰,而且所需的试验设备也非常复杂.为了克服这些困难,可以采用单幅图像来代替多目图像进行三维重构.给出了计算结果,并说明了这些结果的合理性.     

Calibration of Central Catadioptric Cameras Using a DLT-Like Approach

In this study, we present a calibration technique that is valid for all single-viewpoint catadioptric cameras. We are able to represent the projection of 3D points on a catadioptric image linearly with a 6×10 projection matrix, which uses lifted coordinates for image and 3D points. This projection matrix can be computed from 3D–2D correspondences (minimum 20 points distributed in three different planes). We show how to decompose it to obtain intrinsic and extrinsic parameters. Moreover, we use this parameter estimation followed by a non-linear optimization to calibrate various types of cameras. Our results are based on the sphere camera model which considers that every central catadioptric system can be modeled using two projections, one from 3D points to a unitary sphere and then a perspective projection from the sphere to the image plane. We test our method both with simulations and real images, and we analyze the results performing a 3D reconstruction from two omnidirectional images.     

Robust active stereo vision using Kullback-Leibler divergence

Active stereo vision is a method of 3D surface scanning involving the projecting and capturing of a series of light patterns where depth is derived from correspondences between the observed and projected patterns. In contrast, passive stereo vision reveals depth through correspondences between textured images from two or more cameras. By employing a projector, active stereo vision systems find correspondences between two or more cameras, without ambiguity, independent of object texture. In this paper, we present a hybrid 3D reconstruction framework that supplements projected pattern correspondence matching with texture information. The proposed scheme consists of using projected pattern data to derive initial correspondences across cameras and then using texture data to eliminate ambiguities. Pattern modulation data are then used to estimate error models from which Kullback-Leibler divergence refinement is applied to reduce misregistration errors. Using only a small number of patterns, the presented approach reduces measurement errors versus traditional structured light and phase matching methodologies while being insensitive to gamma distortion, projector flickering, and secondary reflections. Experimental results demonstrate these advantages in terms of enhanced 3D reconstruction performance in the presence of noise, deterministic distortions, and conditions of texture and depth contrast.     

基于哈希特征的大规模图像快速匹配算法

针对基于图像进行三维重建技术在使用大规模图像集合进行重建时,需要对图像集合中图像进行两两匹配耗时问题,提出了基于哈希技术对图像构建全局哈希特征的方法,通过过滤掉无效的图像关系对来减少计算时间,极大地提高了大规模图像集合三维重建的匹配计算效率。提出的大规模图像快速哈希匹配算法包括构建图像哈希特征、构建初始匹配图、挑选候选匹配对、哈希匹配几个步骤。实验结果表明该方法能显著地提高三维重建中图像匹配的速度。     

Registration based on projective reconstruction technique for augmented reality systems

In AR systems, registration is one of the most difficult problems currently limiting their application. In this paper, we propose a simple registration method using projective reconstruction. This method consists of two steps: embedding and tracking. Embedding involves specifying four points to build the world coordinate system on which a virtual object will be superimposed. In tracking, a projective reconstruction technique is used to track these four specified points to compute the model view transformation for augmentation. This method is simple, as only four points need to be specified at the embedding stage and the virtual object can then be easily augmented onto a real scene from a video sequence. In addition, it can be extended to a scenario using the projective matrix that has been obtained from previous registration results using the same AR system. The proposed method has three advantages: 1) it is fast because the linear least square method can be used to estimate the related matrix in the algorithm and it is not necessary to calculate the fundamental matrix in the extended case. 2) A virtual object can still be superimposed on a related area even if some parts of the specified area are occluded during the whole process. 3) This method is robust because it remains effective even when not all the reference points are detected during the whole process, as long as at least six pairs of related reference points correspondences can be found. Some experiments have been conducted to validate the performance of the proposed method.     

伪点云修正增强激光雷达数据

目的 激光雷达在自动驾驶中具有重要意义,但其价格昂贵,且产生的激光线束数量仍然较少,造成采集的点云密度较稀疏。为了更好地感知周围环境,本文提出一种激光雷达数据增强算法,由双目图像生成伪点云并对伪点云进行坐标修正,进而实现激光雷达点云的稠密化处理,提高3D目标检测精度。此算法不针对特定的3D目标检测网络结构,是一种通用的点云稠密化方法。方法 首先利用双目RGB图像生成深度图像,根据先验的相机参数和深度信息计算出每个像素点在雷达坐标系下的粗略3维坐标,即伪点云。为了更好地分割地面,本文提出了循环RANSAC （random sample consensus）算法,引入了一个分离平面型非地面点云的暂存器,改进复杂场景下的地面分割效果。然后将原始点云进行地面分割后插入KDTree （k-dimensional tree）,以伪点云中的每个点为中心在KDTree中搜索若干近邻点,基于这些近邻点进行曲面重建。根据曲面重建结果,设计一种计算几何方法导出伪点云修正后的精确坐标。最后,将修正后的伪点云与原始激光雷达点云融合得到稠密化点云。结果 实验结果表明,稠密化的点云在视觉上具有较好的质量,物体具有更加完整的形状和轮廓,并且在KITTI （Karlsruhe Institute of Technology and Toyota Technological Institute）数据集上提升了3D目标检测精度。在使用该数据增强方法后,KITTI数据集下AVOD （aggregate view object detection）检测方法的AP_3D-Easy （average precision of 3D object detection on easy setting）提升了8.25%,AVOD-FPN （aggregate view object detection with feature pyramid network）检测方法的AP_BEV-Hard （average precision of bird’s eye view on hard setting）提升了7.14%。结论 本文提出的激光雷达数据增强算法,实现了点云的稠密化处理,并使3D目标检测结果更加精确。     

基于多粒度匹配的高超速飞行体三维重构方法

针对高超速飞行体在飞行过程中能获取到的图像信息较少,无法完全复现其轮廓信息,提出一种基于多粒度匹配的三维重构优化方法。首先通过两台高分辨率CCD相机正交的阴影照相站系统采集高超速飞行体多幅图像信息,利用图像分割技术提取物体的二维轮廓;然后采用改进的SFS算法求解出物体表面各点的相对高度和表面法向量,恢复其三维形貌;最后使用图像拼接优化技术实现高超速飞行体表面的三维重构。并经过实体模型的三维重构验证所提方法的可行性及有效性。     

一种用于未标定图像三维重建的立体匹配算法*

提出了一种适用于未标定图像三维重建的立体匹配算法。该算法首先引入限制因子来消除Harris角点聚簇的现象,使用高斯曲面拟合内插使Harris角点达到亚像素级;接着采用特征点的Sift特征描述符进行初始匹配,利用随机抽样算法估计基础矩阵的同时剔除误匹配点对;最后在估计的基础矩阵的引导下进行双向匹配。实验证明,该算法能够很好地恢复物体的结构,是一种有效的用于未标定图像三维重建的立体匹配算法。     

Horizon matching for localizing unordered panoramic images

There is currently an abundance of vision algorithms which, provided with a sequence of images that have been acquired from sufficiently close successive 3D locations, are capable of determining the relative positions of the viewpoints from which the images have been captured. However, very few of these algorithms can cope with unordered image sets. This paper presents an efficient method for recovering the position and orientation parameters corresponding to the viewpoints of a set of panoramic images for which no a priori order information is available, along with certain structure information regarding the imaged environment. The proposed approach assumes that all images have been acquired from a constant height above a planar ground and operates sequentially, employing the Levenshtein distance to deduce the spatial proximity of image viewpoints and thus determine the order in which images should be processed. The Levenshtein distance also provides matches between imaged points, from which their corresponding environment points can be reconstructed. Image matching with the aid of the Levenshtein distance forms the crux of an iterative process that alternates between image localization from multiple reconstructed points and point reconstruction from multiple image projections, until all views have been localized. Periodic refinement of the reconstruction with the aid of bundle adjustment, distributes the reconstruction error among images. The approach is demonstrated on several unordered sets of panoramic images obtained in indoor environments.     

一种基于TOF相机与CCD相机的联合标定算法研究

针对基于Time-of-Flight(TOF)相机的彩色目标三维重建需标定CCD相机与TOF相机联合系统的几何参数,在研究现有的基于彩色图像和TOF深度图像标定算法的基础上,提出了一种基于平面棋盘模板的标定方法。拍摄了固定在平面标定模板上的彩色棋盘图案在不同角度下的彩色图像和振幅图像,改进了Harris角点提取,根据棋盘格上角点与虚拟像点的共轭关系,建立了相机标定系统模型,利用Levenberg-Marquardt算法求解,进行了标定实验。获取了TOF与CCD相机内参数,并利用像平面之间的位姿关系估计两相机坐标系的相对姿态,最后进行联合优化,获取了相机之间的旋转矩阵与平移向量。实验结果表明,提出的算法优化了求解过程,提高了标定效率,能够获得较高的精度。