Recursive estimation of motion and a scene model with a two-camera system of divergent view

This paper deals with recursive reconstruction of a scene model from unknown motion of a two-camera system capturing the images of the scene. Single camera systems with a relatively small field of view have limited accuracy because of the inherent confusion between translation and rotation. Estimation results from the stereo camera systems are also compromised due to this confusion if the systems require the fields of view to intersect for stereo correspondence. The cameras constituting the two-camera system considered in this paper are arranged so that there is a small intersection of the fields of view. This configuration of divergent view improves the accuracy of the structure and motion estimation because the ambiguity mentioned above decreases due to a large field of view. In this paper, a recursive algorithm is proposed for fast scene model reconstruction using a two-camera system of divergent view. Using inversely inferred stereo correspondences in the intersection of the fields of view is also proposed to remove degeneracy of scale factor determination and to acquire more accurate results from the information redundancy. The results of the experiments with long term real image sequences are presented to demonstrate the feasibility of the proposed system.     

各向异性扩散平滑滤波的改进算法

图像的噪声过滤和增强是数字图像处理中非常重要的组成部分.在图像处理过程中,为了既有效地去除噪声,又能够较好地保持图像的边缘和重要的细节信息,在Perona-Malik各向异性扩散模型（PM模型）的基础上,通过对变分方法的扩散方程中扩散系数的改进,提出了一个对噪声图像更有效更具有适应性的去噪扩散模型.该模型针对不同的梯度大小采用了不同的扩散系数.在实际处理过程中该模型不仅能够有效地保持图像的边缘,而且还能够克服PM模型对小尺度噪声敏感和部分边缘和细节失真的问题.实验结果表明,改进的扩散模型的性能优于PM模型,是一种较为理想的保边缘平滑模型.     

Robust factorization

Factorization algorithms for recovering structure and motion from an image stream have many advantages, but they usually require a set of well-tracked features. Such a set is in generally not available in practical applications. There is thus a need for making factorization algorithms deal effectively with errors in the tracked features. We propose a new and computationally efficient algorithm for applying an arbitrary error function in the factorization scheme. This algorithm enables the use of robust statistical techniques and arbitrary noise models for the individual features. These techniques and models enable the factorization scheme to deal effectively with mismatched features, missing features, and noise on the individual features. The proposed approach further includes a new method for Euclidean reconstruction that significantly improves convergence of the factorization algorithms. The proposed algorithm has been implemented as a modification of the Christy-Horaud factorization scheme, which yields a perspective reconstruction. Based on this implementation, a considerable increase in error tolerance is demonstrated on real and synthetic data. The proposed scheme can, however, be applied to most other factorization algorithms     

基于4目阵列的计算机视觉三维重建算法

目前计算机视觉三维重建方法因需布置和标定摄像机环形拍摄场或者需要结构光而存在应用局限性问题,且算法不稳定。为此提出一种将摄像机阵列和图像配准有机结合的4目阵列重建算法,该算法不需要结构光和现场标定摄像机。经过基于包含光照和阴影的复杂室内仿真图像的实验表明,该方法能稳定有效地进行密集点云重建,且能克服现有重建方法的应用局限性与不稳定等缺陷。     

RGB-D结构相似性度量下的多边自适应深度图像超分辨率重建

目的 深度相机能够对场景的深度信息进行实时动态捕捉,但捕获的深度图像分辨率低且容易形成空洞。利用高分辨率彩色图像作为引导,是深度图超分辨率重建的重要方式。现有方法对彩色边缘与深度不连续区域的不一致性问题难以有效解决,在深度图超分辨率重建中引入了纹理复制伪影。针对这一问题,本文提出了一种鲁棒的彩色图像引导的深度图超分辨率重建算法。方法 首先,利用彩色图像边缘与深度图像边缘的结构相关性,提出RGB-D结构相似性度量,检测彩色图像与深度图像共有的边缘不连续区域,并利用RGB-D结构相似性度量自适应选取估计像素点邻域的最优图像块。接着,通过提出的定向非局部均值权重,在图像块区域内建立多边引导下的深度估计,解决彩色边缘和深度不连续区域的结构不一致性。最后,利用RGB-D结构相似性度量与图像平滑性之间的对应关系,对多边引导权重的参数进行自适应调节,实现鲁棒的深度图超分辨率重建。结果 在Middlebury合成数据集、ToF和Kinect数据集以及本文自建数据集上的实验结果表明,相比其他先进方法,本文方法能够有效抑制纹理复制伪影。在Middlebury、ToF和Kinect数据集上,本文方法相较于次优算法,平均绝对偏差平均降低约63.51%、39.47 %和7.04 %。结论 对于合成数据集以及真实场景的深度数据集,本文方法均能有效处理存在于彩色边缘和深度不连续区域的不一致性问题,更好地保留深度边缘的不连续性。     

A Robust Approach for Structure from Planar Motion by Stereo Image Sequences

This paper proposes a robust method for recovery of motion and structure from two image sequences taken by stereo cameras undergoing a planar motion. The feature correspondences between images are extracted and refined automatically by the relation of the stereo cameras and the property of the motion. To improve the robustness, an auto-scale random sample consensus (RANSAC) algorithm is adopted in the motion and structure estimation. Unlike other work recovering epipolar geometry, here we use a random sampling algorithm to recover the 2D motion and to exclude the outliers which lie both on and out of the epipolar lines. Further more, the idea of RANSAC is used in structure estimation to exclude the outliers from the image sequence. The contribution of this work is the development of an approach to make structure and motion estimation more robust and efficient so as to be applicable to real applications. With the adoption of the auto-scale technique, the algorithm completely automates the estimation process without any prior information or user’s specification of parameters like thresholds. Indoor and outdoor experiments have been done to verify the performance of the algorithm. The results demonstrated that the proposed algorithm is robust and efficient for applications in planar motions.     

基于多尺度特征递归卷积的稠密点云重建网络

针对在三维重建任务中,由于弱纹理区域的光度一致性测量误差较大,使得传统的多视图立体算法难以处理的问题,提出了一种多尺度特征聚合的递归卷积网络(MARDC-MVSNet),用于弱纹理区域的稠密点云重建。为了使输入图像分辨率更高,该方法使用一个轻量级的多尺度聚合模块自适应地提取图像特征,以解决弱纹理甚至无纹理区域的问题。在代价体正则化方面,采用具有递归结构的分层处理网络代替传统的三维卷积神经网络(CNN),极大程度地降低了显存占用,同时实现高分辨率重建。在网络的末端添加一个深度残差网络模块,以原始图像为指导对正则化网络生成的初始深度图进行优化,使深度图表述更准确。实验结果表明,在DTU数据集上取得了优异的结果,该网络在拥有较高深度图估计精度的同时还节约了硬件资源,且能扩展到航拍影像的实际工程之中。     

基于分层压缩激励的ASPP网络单目深度估计

场景深度估计是场景理解的一项基本任务,其准确率反映了计算机对场景的理解程度.传统的深度估计利用金字塔池化(ASPP)模块可以在不改变图像分辨率的情况下处理不同像素特征,但该模块未考虑不同像素特征之间的关系,导致场景特征提取不准确.针对ASPP模块在深度估计中出现的弊端,提出了一种改进型的ASPP模块,解决了该模块在图像...     

Tracking 3-D motion from straight lines with trifocal tensors

We present a novel approach to track the position and orientation of a stereo camera using line features in the images. The method combines the strengths of trifocal tensors and Bayesian filtering. The trifocal tensor provides a geometric constraint to lock line features among every three frames. It eliminates the explicit reconstruction of the scene even if the 3-D scene structure is not known. Such a trifocal constraint thus makes the algorithm fast and robust. The twist motion model is applied to further improve its computation efficiency. Another major contribution is that our approach can obtain the 3-D camera motion using as little as 2 line correspondences instead of 13 in the traditional approaches. This makes the approach attractive for realistic applications. The performance of the proposed method has been evaluated using both synthetic and real data with encouraging results. Our algorithm is able to estimate 3-D camera motion in real scenarios accurately having little drifting from an image sequence longer than a 1,000 frames.     

基于压缩感知的SAR图像鲁棒编码传输

目的 尽管传统的联合信源信道编码方案可以获得高效的压缩性能,但当信道恶化超过信道编码的纠错能力时会导致解码端重构性能的急剧下降;为此利用压缩感知的民主性提出一种鲁棒的SAR图像编码传输方案,且采用了一系列方法提高该方案的率失真性能。方法 考虑到SAR图像丰富的边缘信息,采用具有更强方向表示能力的方向提升小波变换(DLWT)对SAR图像进行稀疏表示,且为消除压缩感知中恢复非稀疏信号时存在的混叠效应,采用了稀疏滤波方法保证大系数的精确恢复,在解码端采用了高效的Bayesian重建算法获得图像的高性能重建。结果 在同等码率下,与传统的联合信源信道编码方案CCSDS-RS相比,本文方案可以实现更加鲁棒的编码传输,当丢包率达到0.05时,本文方案DSFB-CS获得的重建性能明显要高于CCSDS-RS;与基于Bayesian重建算法TSW-CS的传统方案相比,本文方案可提高峰值信噪比(PSNR)3.9 dB。结论 本文方案DSFB-CS 实现了SAR图像的鲁棒传输,随着丢包率的上升,DSFB-CS获得的重建性能缓慢下降,保证了面对不稳定信道时,解码端可以获得相对稳定的重构图像。     

亮度—梯度联合约束的车牌图像超分辨率重建

目的 受成像距离、光照条件、动态模糊等因素影响,监控系统拍摄的车牌图像往往并不具备较高的可辨识度。为改善成像质量,提升对车牌的识别能力,提出一种基于亮度与梯度联合约束的车牌图像超分辨率重建方法。方法 首先充分结合亮度约束和梯度约束的优势,实现对运动位移和模糊函数的精确估计;为抑制重建图像中的噪声与伪影,基于车牌图像的文字化特征,进一步确定了亮度与梯度联合约束的图像先验模型。结果 为验证该方法的有效性,利用监控系统获得4组车牌图像,分别进行模拟和真实的超分辨率重建实验。在模拟实验中将联合约束图像先验重建结果与拉普拉斯、Huber-Markov（HMRF）以及总变分（TV）先验的处理结果进行对比,联合约束先验对车牌纹理信息的恢复效果优于其他3种常见图像先验;同时,在模拟和真实实验中,将本文算法与双三次插值、传统最大后验概率、非线性扩散正则化和自适应范数正则化方法的超分辨率重建结果进行比较,模拟实验的结果表明,在不添加噪声情况下,该算法峰值信噪比（PSNR）和结构相似性（SSIM）指标分别为35.326 dB和0.958,优于其他4种算法;该算法在真实实验中,能够有效增强车牌图像纹理信息,获得较优的视觉效果,通过对重建车牌图像的字符识别精度比较,本文算法重建结果的识别精度远高于其他3种算法,平均字符差距为1.3。结论 模拟和真实图像序列的实验结果证明,基于亮度—梯度联合约束的超分辨率重建方法,能够降低运动和模糊等参数的估计误差,有效减少图像中存在的模糊和噪声,提高车牌的识别精度。该算法广泛适用于因光照变化、相对运动等因素影响下的低质量车牌图像超分辨率重建。     

Robust adaptive control of nonlinearly parametrized multivariable systems with unmatched disturbances

A new robust adaptive control scheme is developed for nonlinearly parametrized multivariable systems in the presence of parameter uncertainties and unmatched disturbances. The developed control scheme employs a new integrated framework of a functional bounding technique for handling nonlinearly parametrized system dynamics, an adaptive parameter estimation algorithm for dealing with parameter uncertainties, a nonlinear feedback controller structure for stabilization of interconnected system states, and a robust adaptive control design for accommodating unmatched disturbances. It is proved that such a new robust adaptive control scheme is capable of ensuring the global boundedness and mean convergence of all closed‐loop system signals. A complete simulation study on an air vehicle system with nonlinear parametrization in the presence of an unmatched wind disturbance is conducted, and its results verify the effectiveness of the proposed robust adaptive control scheme.     

Stochastic Approximation and Rate-Distortion Analysis for Robust Structure and Motion Estimation

Recent research on structure and motion recovery has focused on issues related to sensitivity and robustness of existing techniques. One possible reason is that in practical applications, the underlying assumptions made by existing algorithms are often violated. In this paper, we propose a framework for 3D reconstruction from short monocular video sequences taking into account the statistical errors in reconstruction algorithms. Detailed error analysis is especially important for this problem because the motion between pairs of frames is small and slight perturbations in its estimates can lead to large errors in 3D reconstruction. We focus on the following issues: physical sources of errors, their experimental and theoretical analysis, robust estimation techniques and measures for characterizing the quality of the final reconstruction. We derive a precise relationship between the error in the reconstruction and the error in the image correspondences. The error analysis is used to design a robust, recursive multi-frame fusion algorithm using stochastic approximation as the framework since it is capable of dealing with incomplete information about errors in observations. Rate-distortion analysis is proposed for evaluating the quality of the final reconstruction as a function of the number of frames and the error in the image correspondences. Finally, to demonstrate the effectiveness of the algorithm, examples of depth reconstruction are shown for different video sequences.     

Recursive three-dimensional model reconstruction based on Kalman filtering.

A recursive two-step method to recover structure and motion from image sequences based on Kalman filtering is described in this paper. The algorithm consists of two major steps. The first step is an extended Kalman filter (EKF) for the estimation of the object's pose. The second step is a set of EKFs, one for each model point, for the refinement of the positions of the model features in the three-dimensional (3-D) space. These two steps alternate from frame to frame. The initial model converges to the final structure as the image sequence is scanned sequentially. The performance of the algorithm is demonstrated with both synthetic data and real-world objects. Analytical and empirical comparisons are made among our approach, the interleaved bundle adjustment method, and the Kalman filtering-based recursive algorithm by Azarbayejani and Pentland. Our approach outperformed the other two algorithms in terms of computation speed without loss in the quality of model reconstruction.     

基于生成对抗网络的多幅离焦图像超分辨率重建算法

为提高超分辨率算法重建出的图像质量,提出融合多幅离焦图像的超分辨率重建算法。以离焦图像作为切入点,利用自编码器提取离焦图像中的重要特征,根据空间特征变换层结构,将离焦特征与原始特征相结合,完成图像的超分辨率重建。在Celeb A人脸数据集上进行实验,结果表明,与传统插值算法及SRGAN算法相比,所提算法在大多数情况下能获得更高峰值信噪比及结构相似性数值,能生成质量更高的重建图像。     

一种基于场景图分割的混合式多视图三维重建方法

针对大范围三维重建, 重建效率较低和重建稳定性、精度差等问题, 提出了一种基于场景图分割的大范围混合式多视图三维重建方法.该方法首先使用多层次加权核K均值算法进行场景图分割; 然后,分别对每个子场景图进行混合式重建, 生成对应的子模型, 通过场景图分割、混合式重建和局部优化等方法提高重建效率、降低计算资源消耗, 并综合采用强化的最佳影像选择标准、稳健的三角测量方法和迭代优化等策略, 提高重建精度和稳健性; 最后, 对所有子模型进行合并, 完成大范围三维重建.分别使用互联网收集数据和无人机航拍数据进行了验证, 并与1DSFM、HSFM算法在计算精度和计算效率等方面进行了比较.实验结果表明, 本文算法大大提高了计算效率、计算精度, 能充分保证重建模型的完整性, 并具备单机大范围场景三维重建能力.     

DSP平台的图像特征点快速索引结构

针对DSP(digital signal processor,数字信号处理器)平台上的图像特征点匹配问题,提出了一种高效的基于自聚类二分查找树的快速索引结构,并设计了适合于DSP结构特点的索引存储布局。通过在离线情况下将特征点参考数据集逐级地二分聚类,生成多级索引结构。以顺序数组的方式将树状索引结构存储到连续的内存空间中,便于导出为数据文件存储及进一步加载到DSP内存中使用。实验表明,该索引结构能够快速有效地在DSP平台完成特征点匹配工作。     

Models from image triplets using epipolar gradient features

In an application where sparse matching of feature points is used towards fast scene reconstruction, the choice of the type of features to be matched has an important impact on the quality of the resulting model. In this work, a method is presented for quickly and reliably selecting and matching points from three views of a scene. The selected points are based on epipolar gradients, and consist of stable image features relevant to reconstruction. Then, the selected points are matched using edge transfer, a measure of geometric consistency for point triplets and the edges on which they lie. This matching scheme is tolerant to image deformations due to changes in viewpoint. Models drawn from matches obtained by the proposed technique are shown to demonstrate its usefulness.