Stereo vision calibration based on GMDH neural network

In order to improve the accuracy and stability of stereo vision calibration, a novel stereo vision calibration approach based on the group method of data handling (GMDH) neural network is presented. Three GMDH neural networks are utilized to build a spatial mapping relationship adaptively in individual dimension. In the process of modeling, the Levenberg-Marquardt optimization algorithm is introduced as an interior criterion to train each partial model, and the corrected Akaike's information criterion is introduced as an exterior criterion to evaluate these models. Experiments demonstrate that the proposed approach is stable and able to calibrate three-dimensional (3D) locations more accurately and learn the stereo mapping models adaptively. It is a convenient way to calibrate the stereo vision without specialized knowledge of stereo vision.     

基于多视图立体视觉的沙堆三维尺寸测量研究

利用从运动中恢复结构方法(SFM),提出了一种基于多视图立体视觉的沙堆三维重建及三维尺寸测量方法。首先根据SFM方法的求解不稳定特点,结合光束平差法对SFM求解过程进行分析及优化;其次针对SFM重建结果为稀疏点云的问题,利用基于面片的稠密重建算法重新生成稠密的三维点云,再利用泊松算法对密集点云进行三维曲面重建;最后获得模型的三维尺寸信息。对某建筑工地的沙堆进行了三维尺寸的测量实验,实验结果验证了该方法的有效及可行性,提高了重建能力及精度,同时考虑了目标实际测量误差与重建误差,能够满足实际智能测量的应用需求。     

Descent image based landing area terrain reconstruction technology for lunar landing mission

In this paper, we present a descent image based extraterrestrial terrain reconstruction technology, which has been successfully applied in China's first soft landing Chang'E-3 mission on moon. When scouting on the moon surface, the lunar rover (Jade Rabbit rover) requires reliable terrain information for navigation and positioning tasks. The descent image sequence acts as a link between the orbital images and the surface images. The proposed approach collects multilevel descent images, as well as inertial information from an inertial measurement unit and surface stereo vision information, to generalise a consistent surface model. A data fusion strategy is proposed to track the camera trajectory, and refined plane sweep stereo matching and 3D alignment technologies are analysed in this article. The onboard result, as well as on-earth simulation experiments, has shown that the discussed techniques are reliable to reconstruct the terrain information for the lunar rover's localisation and navigation.     

Estimating three-dimensional vehicle motion in an outdoor scene using stereo image sequences

We propose a motion estimation system that uses stereo image pairs as the input data. To perform experimental work, we also obtain a sequence of outdoor stereo images taken by two metric cameras. The system consists of four main stages, which are (1) determination of point correspondences on the stereo images, (2) correction of distortions in image coordinates, (3) derivation of 3D point coordinates from 2D correspondences, and (4) estimation of motion parameters based on 3D point correspondences. For the first stage of the system, we use a four-way matching algorithm to obtain matched point on two stereo image pairs at two consecutive time instants (t_i and t_{i + 1}). Since the input data are stereo images taken by cameras, it has two types of distortions, which are (i) film distortion and (ii) lens distortion. These two distortions must be corrected before any process can be applied on the matched points. To accomplish this goal, we use (i) bilinear transform for film distortion correction and (ii) lens formulas for lens distortion correction. After correcting the distortions, the results are 2D coordinates of each matched point that can be used to derive 3D coordinates. However, due to data noise, the calculated 3D coordinates to not usually represent a consistent rigid structure that is suitable for motion estimation; therefore, we suggest a procedure to select good 3D point sets as the input for motion estimation. The procedure exploits two constraints, rigidity between different time instants and uniform point distribution across the object on the image. For the last stage, we use an algorithm to estimate the motion parameters. We also wish to know what is the effect of quantization error on the estimated results; therefore an error analysis based on quantization error is performed on the estimated motion parameters. In order to test our system, eight sets of stereo image pairs are extracted from an outdoor stereo image sequence and used as the input data. The experimental results indicate that the proposed system does provide reasonable estimated motion parameters.     

基于曲率自适应的航空零件法矢量测量研究

现有的法矢量测量技术无法满足测量准确度和实时性的双重要求,法矢量计算方法不能适用于不同曲率的曲面。利用双目立体视觉,本文提出了一种基于曲率自适应的法矢量测量方法。首先,在双目立体视觉的基础上,基于变曲率曲面特征建立制孔区域曲面模型,提出投影点的布局方法;然后,基于三维重建的投影点数据,提出了基于曲面曲率自适应识别的法矢量计算方法;最后,针对小曲率曲面样件的测量结果,与三坐标测量仪测得的法矢量进行对比,用以验证本双目视觉测量方法的精度。实验结果表明:该方法测量法矢量误差为1.6°。该方法可有效提高法矢量测量的准确度,满足大型航空零件现场测量的工程要求。     

Real-time pedestrian classification exploiting 2D and 3D information

A new approach for standing and walking pedestrian detection using pattern matching and exploiting both 2D image information and 3D dense stereo information is proposed. Because 3D information accuracy does not allow the direct classification of the 3D shape, a combined 3D-2D method is proposed. The 3D data are used in an innovative way for pedestrian hypotheses generation, scale and depth estimation and 2D models selection. Also the 3D hypotheses allow the corresponding 2D image region of interest selection and the 2D hypothesis generation. The 2D hypothesis consists of the object-s external edges obtained by an edge extraction and a depth coherency-based filtering out process. The scaled models are matched against the selected hypothesis using an elastic high-speed matching based on the Chamfer distance. The method has been tested on synthetic and real-world scenarios.     

Research on the application of single camera stereo vision sensor in three-dimensional point measurement

A single camera stereo vision sensor model based on planar mirror imaging is proposed for measuring a three-dimensional point. The model consists of a CCD camera and a planar mirror. Using planar mirror reflections of a scene, a picture with parallax is obtained by shooting the target object and its virtual image. This is equivalent to shooting the target object from different angles with the camera and the virtual camera in the planar mirror, so it has the function of binocular stereo vision. In addition, the measurement theory of the three-dimensional point is discussed. The mathematical model of a single camera stereo vision sensor is established, the intrinsic and extrinsic parameters are calibrated, and the corresponding experiment has been done. The experimental results show that the measuring method is convenient and effective; it also has the advantages of simple structure, convenient adjustment, and is especially suitable for short-distance measurement with high precision.     

Face tracking and pose estimation with automatic three-dimensional model construction

A method for robustly tracking and estimating the face pose of a person using stereo vision is presented. The method is invariant to identity and does not require previous training. A face model is automatically initialised and constructed online: a fixed point distribution is superposed over the face when it is frontal to the cameras, and several appropriate points close to those locations are chosen for tracking. Using the stereo correspondence of the cameras, the three-dimensional (3D) coordinates of these points are extracted, and the 3D model is created. The 2D projections of the model points are tracked separately on the left and right images using SMAT. RANSAC and POSIT are used for 3D pose estimation. Head rotations up to plusmn45deg are correctly estimated. The approach runs in real time. The purpose of this method is to serve as the basis of a driver monitoring system, and has been tested on sequences recorded in a moving car.     

基于LS-SVM的立体视觉摄像机标定

利用最小二乘支持向量机来直接学习图像信息与三维信息之间的关系,不需确定摄像机具体的内部参数和外部参数.在双目视觉的情况下,两摄像机的位置关系不需具体求出,而是隐含在映射关系中.根据最小二乘支持向量机与摄像机标定的特点,提出了基于最小二乘支持向量机的双目立体摄像机标定方法.将摄像头采集到的图像的像素坐标作为输入,将世界坐标作为输出,用最小二乘支持向量机使网络实现给定的输入输出映射关系.该方法同BP神经网络预测结果对比表明:基于最小二乘支持向量机的双目视觉标定方法速度快,实时性好,能有效提高标定精度.     

Bond line thickness and die tilt inspection in die bonding process

This study focuses upon building an automated inspection system for the in-line measurement of bond-line thickness (BLT) and die tilt in die attachments in the semiconductor packaging process. A prototype of a visual system utilizing the line scan stereo vision technique via two linear CCD cameras has been developed for the die bonding quality inspection. A novel algorithm is proposed for the determination of the 3D coordinates of certain points on an object. The BLT and die tilt of a die attached chip of substrate height 800 μm and cross section 15 × 15 mm² were measured to be approximately 76.8 μm and 0.00094, respectively. The measurement can be completed within one second. Thus the in-line inspection of die bonding quality can be conducted efficiently using the proposed technique.     

双摄像机光笔式三维坐标测量系统研究

基于双目立体视觉传感器三维测量模型和空间坐标变换原理,采用最小二乘冗余算法,讨论了双摄像机测量空间三维点坐标的基本原理,建立了视觉测量系统的非线性测量方程.通过实验验证了双摄像机光笔式三维坐标视觉测量系统建立的可行性.     

Image rectification for stereoscopic visualization

This paper proposes an approach to rectifying two images of the same scene captured by cameras at general positions so that the results form a stereo pair that satisfies the constraints of the stereoscopic visualization platforms. This is unlike conventional image rectification research that primarily focuses on making stereo matching easier but pays little attention to 3D viewing. The novel derivation of the rectification algorithm also has an intuitive physical meaning that is not available from conventional approaches. Practical issues related to wide-baseline rectification and operation range of the proposed method are analyzed. Both simulated and real data experiments are used to assess the performance of the proposed algorithm.     

双目视觉摄像机神经网络标定方法

摄像机标定是精密视觉测量的基础。为了描述双目视觉中三维空间物点坐标和两个摄像机像面像点坐标间的非线性关系,传统的标定方法需要建立复杂的数学模型。而神经网络可以有效地处理非线性映射问题,笔者介绍了一种BP(ErrorBackPropagation)神经网络,并且为了提高网络的学习能力引入了动态因子。用相同的参考数据,将神经网络标定方法与线性标定方法比较,实验结果表明基于神经网络的双目视觉标定方法能获得较高的标定精度。     

双目立体视觉中特征点三维坐标重构校准研究

双目立体视觉是一种商业化较成熟的三维测量技术,左右摄像机内外参数的精确标定是实现三维重构的基础和关键。针对棋盘格和圆点两种标定板图案研究了相应的图像处理技术,实现了标定点的亚像素精度定位及其有效排序。采用基于单映性约束和非线性优化的多视角平面标定算法实现了摄像机光学及空间位置参数求解。用极线约束残差法衡量标定结果的准确度。基于桥式三坐标标准实现标定点三维重构平面度以及多平面空间夹角测量结果的校准,基于光学三坐标标准实现了标定点三维重构空间距离的校准,并分析了校准结果的不确定度。圆标志点三维重构空间距离示值误差为0.029 mm,不确定度U=24 μm（k=2）     

Performance Analysis and Evaluation of Geometric Parameters in Stereo Deflectometry

This paper presents a novel geometric parameters analysis to improve the measurement accuracy of stereo deflectometry. Stereo deflectometry can be used to obtain form information for freeform specular surfaces. A measurement system based on stereo deflectometry typically consists of a fringe-displaying screen, a main camera, and a reference camera. The arrangement of the components of a stereo deflectometry system is important for achieving high-accuracy measurements. In this paper, four geometric parameters of a stereo deflectometry system are analyzed and evaluated: the distance between the main camera and the measured object surface, the angle between the main camera ray and the surface normal, the distance between the fringe-displaying screen and the object, and the angle between the main camera and the reference camera. The influence of the geometric parameters on the measurement accuracy is evaluated. Experiments are performed using simulated and experimental data. The experimental results confirm the impact of these parameters on the measurement accuracy. A measurement system based on the proposed analysis has been set up to measure a stock concave mirror. Through a comparison of the given surface parameters of the concave mirror, a global measurement accuracy of 154.2 nm was achieved.     

基于视觉辅助定位的机械臂运动学参数辨识研究

标定机器臂的运动学参数可以有效提高机械臂的绝对定位精度。针对一般平面约束标定方法往往通过手动示教获取测量数据,效率低,提出一种基于视觉辅助定位约束平面的机械臂运动学参数辨识方法。为了弥补双目视觉视场范围狭小的弊端,在约束平面上粘贴3个靶点,以此将对平面的定位等效成对靶点的定位。应用双目视觉系统提取靶点中心并进行立体匹配,得到靶点在机械臂基坐标系下的三维位置信息;同时构建靶点坐标系,以此规划出按一定规律分布的约束点;为了进一步提高标定精度,建立双平面约束误差模型,通过两垂直平面上任意非共线的3个点得到一系列法向量,每一对法向量的数量积为0,即增加了约束方程;利用机械臂对相互垂直的两约束平面自动进行接触式测量,通过改进的最小二乘法辨识出真实的运动学参数误差。实验结果表明,基于双平面约束误差模型,修正运动学参数后,机械臂绝对位置精度由1.234 mm提高到了0.453 mm。该方法实现了数据的自动化测量,大大提高了标定效率,为机械臂批量标定提供了参考,具有工程意义。     

基于2D-3D双目运动估计的立体视觉定位算法

运动估计算法是影响立体视觉定位精度的重要因素,传统的3D-3D运动估计算法受噪声影响很大,计算精度不高.本文提出了一种基于2D-3D双目运动估计的立体视觉定位算法.算法不使用运动后的特征点3D坐标,而直接利用其2D图像投影坐标.首先,利用EPnP运动估计算法确定匹配内点和初始运动参数.接着,利用双目相机之间的2D投影几...     

Probabilistic 3D object recognition and pose estimation using multiple interpretations generation

This paper presents a probabilistic object recognition and pose estimation method using multiple interpretation generation in cluttered indoor environments. How to handle pose ambiguity and uncertainty is the main challenge in most recognition systems. In order to solve this problem, we approach it in a probabilistic manner. First, given a three-dimensional (3D) polyhedral object model, the parallel and perpendicular line pairs, which are detected from stereo images and 3D point clouds, generate pose hypotheses as multiple interpretations, with ambiguity from partial occlusion and fragmentation of 3D lines especially taken into account. Different from the previous methods, each pose interpretation is represented as a region instead of a point in pose space reflecting the measurement uncertainty. Then, for each pose interpretation, more features around the estimated pose are further utilized as additional evidence for computing the probability using the Bayesian principle in terms of likelihood and unlikelihood. Finally, fusion strategy is applied to the top ranked interpretations with high probabilities, which are further verified and refined to give a more accurate pose estimation in real time. The experimental results show the performance and potential of the proposed approach in real cluttered domestic environments.     

Single‐camera special stereo‐digital image correlation applied for accurate virtual fields method identification

A single‐camera special stereo‐DIC (SS‐DIC) is proposed for accurate virtual fields method (VFM) identification. The single‐camera SS‐DIC allows accurate surface 3D deformation measurements using a single colour camera and a specially designed colour separation device. It not only effectively eliminates the unavoidable out‐of‐plane movement/rotation due to unideal in‐plane loading but also delivers uniformly distributed measurement points that brings great simplicity and convenience for internal virtual work calculation in VFM. In addition, since only a single camera is used for stereovision, the proposed SS‐DIC system requires no complicated synchronisation devices. The effectiveness and practicality of the proposed method are evaluated by heterogeneous deformation experiments of a holed aluminium alloy and 304 stainless steel plate. Combined with a high‐speed colour camera, the proposed method is expected to be a simple and practical method for the calibration of material constitutive model under intermediate and high strain rate conditions using VFM.     

Optimized stereo reconstruction of free-form space curves based on a nonuniform rational B-spline model

Analytical reconstruction of 3D curves from their stereo images is an important issue in computer vision. We present an optimization framework for such a problem based on a nonuniform rational B-spline (NURBS) curve model that converts reconstruction of a 3D curve into reconstruction of control points and weights of a NURBS representation of the curve, accordingly bypassing the error-prone point-to-point correspondence matching. Perspective invariance of NURBS curves and constraints deduced on stereo NURBS curves are employed to formulate the 3D curve reconstruction problem into a constrained nonlinear optimization. A parallel rectification technique is then adopted to simplify the constraints, and the Levenberg-Marquardt algorithm is applied to search for the optimal solution of the simplified problem. The results from our experiments show that the proposed framework works stably in the presence of different data samplings, randomly posed noise, and partial loss of data and is potentially suitable for real scenes.