双目视觉摄像机神经网络标定方法

摄像机标定是精密视觉测量的基础。为了描述双目视觉中三维空间物点坐标和两个摄像机像面像点坐标间的非线性关系,传统的标定方法需要建立复杂的数学模型。而神经网络可以有效地处理非线性映射问题,笔者介绍了一种BP(ErrorBackPropagation)神经网络,并且为了提高网络的学习能力引入了动态因子。用相同的参考数据,将神经网络标定方法与线性标定方法比较,实验结果表明基于神经网络的双目视觉标定方法能获得较高的标定精度。     

The effects of a camera's intrinsic parameters on the determination of a coordinate frame

Abstract

In this paper, two methods, the Individual Calibration Method (ICM) and the Simultaneous Calibration Method (SCM), are used to calibrate a vision system with two cameras. The first approach calibrates a camera individually. For the second approach, two or more sets of camera intrinsic and extrinsic parameters are determined simultaneously such that the position errors for the 3‐D points are minimized. We show, theoretically and experimentally, the following results: (a) the offsets of the image centers do not affect significantly the position and orientation of a coordinate frame with respect to a stereo vision system with two cameras; (b) the lens distortion will not change dramatically the position and orientation of a coordinate frame with respect to a stereo vision system. We also find that (a) there is a difference in the determination of an object's location for the simultaneous and individual calibration methods; (b) the approach of calibrating two cameras simultaneously has the advantage of better accuracy over the method of individual calibration.     

基于LS-SVM的立体视觉摄像机标定

利用最小二乘支持向量机来直接学习图像信息与三维信息之间的关系,不需确定摄像机具体的内部参数和外部参数.在双目视觉的情况下,两摄像机的位置关系不需具体求出,而是隐含在映射关系中.根据最小二乘支持向量机与摄像机标定的特点,提出了基于最小二乘支持向量机的双目立体摄像机标定方法.将摄像头采集到的图像的像素坐标作为输入,将世界坐标作为输出,用最小二乘支持向量机使网络实现给定的输入输出映射关系.该方法同BP神经网络预测结果对比表明:基于最小二乘支持向量机的双目视觉标定方法速度快,实时性好,能有效提高标定精度.     

Stereo calibration of binocular ultra-wide angle long-wave infrared camera based on an equivalent small field of view camera

ABSTRACT

In this paper, a stereo calibration method for binocular ultra-wide angle long-wave infrared camera is proposed on the basis of an equivalent small field of view camera. Extrinsic parameters are calibrated through the corrected images from the left and right cameras. They can be viewed as images taken by a small field of view camera. The calibration procedure consists of three steps: monocular calibration, distortion correction and extrinsic parameters calibration. In order to evaluate the accuracy of the method, stereo vision of the camera is modelled and a 3D reconstruction approach is presented. A series of experiments, including intrinsic parameters, extrinsic parameters and 3D reconstruction, are conducted to validate the proposed method. The results show that the baseline length error decreases to 0.67%, and the relative error for the 3D reconstruction of corners is smaller than 8.11%. In contrast to the common stereo calibration method, it improves calibration accuracy.     

Using neural networks for 3D measurement in stereo vision inspection systems

This paper presents a stereo vision inspection process which derives precise 3D measurements. Two artificial neural networks are used to facilitate the whole measurement process. At first, a simple camera calibration process is developed to derive the focal lengths and the relative information. A Hopfield neural network is used to solve the stereo matching problem, which has been constructed as an energy function. By means of a recursive process, the disparities of extracted feature points are obtained. In addition, a backpropagation neural network-based measurement error correction model for 3D measurement is proposed. It reduces the errors of 3D measurement associated with a part's orientation, position, magnitude and distance between the object and cameras. Four procedural processes are designed to implement this model. Our laboratory experiments demonstrate that the proposed measurement process has a satisfactory measurement result.     

Binocular self-calibration performed via adaptive genetic algorithm based on laser line imaging

Abstract

An accurate technique to perform binocular self-calibration by means of an adaptive genetic algorithm based on a laser line is presented. In this calibration, the genetic algorithm computes the vision parameters through simulated binary crossover (SBX). To carry it out, the genetic algorithm constructs an objective function from the binocular geometry of the laser line projection. Then, the SBX minimizes the objective function via chromosomes recombination. In this algorithm, the adaptive procedure determines the search space via line position to obtain the minimum convergence. Thus, the chromosomes of vision parameters provide the minimization. The approach of the proposed adaptive genetic algorithm is to calibrate and recalibrate the binocular setup without references and physical measurements. This procedure leads to improve the traditional genetic algorithms, which calibrate the vision parameters by means of references and an unknown search space. It is because the proposed adaptive algorithm avoids errors produced by the missing of references. Additionally, the three-dimensional vision is carried out based on the laser line position and vision parameters. The contribution of the proposed algorithm is corroborated by an evaluation of accuracy of binocular calibration, which is performed via traditional genetic algorithms.     

一种双目立体视觉系统的校准方法

基于双目立体视觉系统的图像分析以及人工神经网络的三维空间建模算法,设计了一种针对双目立体视觉相机的校准方法,并可应用于运动目标点的轨迹追踪。将均匀分布目标点的校准平面放置在有效视野内的不同位置,通过双目立体视觉系统来捕获处于不同位置的校准平面图像。在图像处理之后,使用校准点中心的二维坐标作为人工神经网络训练的输入样本集,通过建立人工神经网络模型结构,实现目标点二维平面坐标到三维空间坐标的映射关系。采用这种具有通用性的方法,可以有效修正系统中存在的失真因子,获得目标三维位置信息,而无需进行复杂的相机校准操作。实验表明,提出的方案具有良好的可行性和鲁棒性。     

Vision ray calibration for the quantitative geometric description of general imaging and projection optics in metrology

Exact geometric calibration of optical devices like projectors or cameras is the basis for utilizing them in quantitative metrological applications. The common state-of-the-art photogrammetric pinhole-imaging-based models with supplemental polynomial corrections fail in the presence of nonsymmetric or high-spatial-frequency distortions and in describing caustics efficiently. These problems are solved by our vision ray calibration (VRC), which is proposed in this paper. The VRC takes an optical mapping system modeled as a black box and directly delivers corresponding vision rays for each mapped pixel. The underlying model, the calibration process, and examples are visualized and reviewed, demonstrating the potential of the VRC.     

A CNN-based passive optical range finder for real-time roboticapplications

The paper presents a new cellular neural network (CNN) for real-time stereo vision, useful as a passive optical range finder for autonomous robots and vehicles. The stereo matching as energy minimization is discussed, and former neural approaches to the problem are analyzed. Experimental results with the new CNN both with synthetic and real images are reported, demonstrating the performance of the system     

立体视觉曲面重建与系统误差分析

采用立体视觉空间曲面重建技术对三维曲面表面成像进行边缘提取、图像匹配、匹配点空间位置计算等步骤,得到三维曲面表面点的空间位置,利用空间点信息对三维曲面形状进行重建,恢复曲面三维形状;并讨论了立体视觉系统的摄像机分辨率、测量范围和摄像机间距等参数之间的关系.利用该方法对堆积物表面形状及体积进行测量实验结果表明,该方法能准确、快速、方便地给出三维曲面的形状.     

Novel method of calibration with restrictive constraints for stereo-vision system

Regarding the calibration of a stereo vision measurement system, this paper puts forward a new bundle adjustment algorithm based on the stereo vision camera calibration method. Multiple-view geometric constraints and a bundle adjustment algorithm are used to optimize the inner and outer parameters of the camera accurately. A fixed relative constraint relationship between cameras is introduced. We have improved the normal equation construction process of the traditional bundle adjustment method, so that each iteration process occurs just outside the parameters of two images that are taken by a camera that has been optimized to better integrate two cameras bound together as one camera. The relationship between the fixed relative constraints can effectively increase the number of superfluous observations of the adjustment system and optimize higher accuracy while reducing the dimension of the normal matrix; it means that each iteration will reduce the time required. Simulation and actual experimental results show the superior performance of the proposed approach in terms of robustness and accuracy, and our approach also can be extended to stereo-vision system with more than two cameras.     

无人飞艇低空数码影像多视摄影测量

针对大比例尺测图的精度要求,研究低空数码影像多视摄影测量的方法.根据计算机视觉的多视几何原理建立匹配的约束条件,实现多视匹配;利用低空数码影像的重叠度大的特点,建立多基线立体,进行多片空间前方交会.不同重叠度的立体像对空间前方交会和多片空间前方交会的实验结果证明,多视摄影测量能够提高有效低空数码影像的摄影测量精度.     

Path synthesis of four-bar mechanisms using synergy of polynomial neural network and Stackelberg game theory

In this article, a novel approach based on game theory is presented for multi-objective optimal synthesis of four-bar mechanisms. The multi-objective optimization problem is modelled as a Stackelberg game. The more important objective function, tracking error, is considered as the leader, and the other objective function, deviation of the transmission angle from 90° (TA), is considered as the follower. In a new approach, a group method of data handling (GMDH)-type neural network is also utilized to construct an approximate model for the rational reaction set (RRS) of the follower. Using the proposed game-theoretic approach, the multi-objective optimal synthesis of a four-bar mechanism is then cast into a single-objective optimal synthesis using the leader variables and the obtained RRS of the follower. The superiority of using the synergy game-theoretic method of Stackelberg with a GMDH-type neural network is demonstrated for two case studies on the synthesis of four-bar mechanisms.     

Intensity-based computer binocular stereo vision: Signal models and algorithms

This article summarizes briefly the results obtained within the symmetrized approach to intensity-based computer binocular stereo vision, considering it as a problem of optimal statistical estimation of digital terrain model (DTM) from a given stereo pair of images. This approach was proposed by the author in 1972–1979 and then extended to take into account main geometric and photometric characteristics of the binocular stereo vision: symmetry of both channels, nonuniform intensity distortions over the images, limited possibilities for binocular perception of the terrain, and inherent ambiguity of the DTM estimation from the stereo pair.     

The contribution of stereo vision to the control of braking

In this study the contribution of stereo vision to the control of braking in front of a stationary target vehicle was investigated. Participants with normal (StereoN) and weak (StereoW) stereo vision drove a go-cart along a linear track towards a stationary vehicle. They could start braking from a distance of 4, 7, or 10m from the vehicle. Deceleration patterns were measured by means of a laser. A lack of stereo vision was associated with an earlier onset of braking, but the duration of the braking manoeuvre was similar. During the deceleration, the time of peak deceleration occurred earlier in drivers with weak stereo vision. Stopping distance was greater in those lacking in stereo vision. A lack of stereo vision was associated with a more prudent brake behaviour, in which the driver took into account a larger safety margin. This compensation might be caused either by an unconscious adaptation of the human perceptuo-motor system, or by a systematic underestimation of distance remaining due to the lack of stereo vision. In general, a lack of stereo vision did not seem to increase the risk of rear-end collisions.     

Calibrating constitutive models with full-field data via physics informed neural networks

The calibration of solid constitutive models with full-field experimental data is a long-standing challenge, especially in materials that undergo large deformations. In this paper, we propose a physics-informed deep-learning framework for the discovery of hyperelastic constitutive model parameterizations given full-field surface displacement data and global force-displacement data. Contrary to the majority of recent literature in this field, we work with the weak form of the governing equations rather than the strong form to impose physical constraints upon the neural network predictions. The approach presented in this paper is computationally efficient, suitable for irregular geometric domains, and readily ingests displacement data without the need for interpolation onto a computational grid. A selection of canonical hyperelastic material models suitable for different material classes is considered including the Neo–Hookean, Gent, and Blatz–Ko constitutive models as exemplars for general non-linear elastic behaviour, elastomer behaviour with finite strain lock-up, and compressible foam behaviour, respectively. We demonstrate that physics informed machine learning is an enabling technology and may shift the paradigm of how full-field experimental data are utilized to calibrate constitutive models under finite deformations.     

Development of a robust calibration model for nonlinear in-line process data

A comparative study involving a global linear method (partial least squares), a local linear method (locally weighted regression), and a nonlinear method (neural networks) has been performed in order to implement a calibration model on an industrial process. The models were designed to predict the water content in a reactor during a distillation process, using in-line measurements from a near-infrared analyzer. Curved effects due to changes in temperature and variations between the different batches make the problem particularly challenging. The influence of spectral range selection and data preprocessing has been studied. With each calibration method, specific procedures have been applied to promote model robustness. In particular, the use of a monitoring set with neural networks does not always prevent overfitting. Therefore, we developed a model selection criterion based on the determination of the median of monitoring error over replicate trials. The back-propagation neural network models selected were found to outperform the other methods on independent test data.     

Adapting second‐order response surface designs to specific needs

Experimental design strategies most often involve an initial choice of a classic factorial or response surface design and adapt that design to meet restrictions or unique requirements of the system under study. One such experience is described here, in which the objective was to develop an efficient experimental design strategy that would facilitate building second‐order response models with excellent prediction capabilities. In development, careful consideration was paid to the desirable properties of response surface designs. Once developed, the proposed design was evaluated using Monte Carlo simulation to prove the concept, a pilot implementation of the design carried out to evaluate the accuracy of the response models, and a set of validation runs enacted to look for potential weaknesses in the approach. The purpose of the exercise was to develop a procedure to efficiently and effectively calibrate strain‐gauge balances to be used in wind tunnel testing. The current calibration testing procedure is based on a time‐intensive one‐factor‐at‐a‐time method. In this study, response surface methods were used to reduce the number of calibration runs required during the labor‐intensive heavy load calibration, to leverage the prediction capabilities of response surface designs, and to provide an estimate of uncertainty for the calibration models. Results of the three‐phased approach for design evaluation are presented. The new calibration process will require significantly fewer tests to achieve the same or improved levels of precision in balance calibration. Copyright © 2008 John Wiley & Sons, Ltd.     

一种用于外科手术导航的双目视觉简化系统

针对外科手术导航光学定位仪参数标定过程繁琐和对应点匹配歧异性大的问题,进行了双目立体视觉简化系统的研究.运用坐标变换方法一次性标定系统24个参数;选用黑白棋盘格标记物和运用相似性度量原理,简化立体匹配过程、提高立体匹配准确度.利用标定后的系统对标记物之间的距离进行测量,在距双目视觉传感单元1.5 m范围内误差小于1 mm.交叉韧带重建手术导舷模拟实验表明:该系统具有稳定,可靠,标定方法简单,匹配准确,定位精度高的特点,能够满足外科手术导航的要求.