Variational method for super-resolution optical flow

The motion fields in an image sequence observed by a car-mounted imaging system depend on the positions in the imaging plane. Since the motion displacements in the regions close to the camera centre are small, for accurate optical flow computation in this region, we are required to use super-resolution of optical flow fields. We develop an algorithm for super-resolution optical flow computation. Super-resolution of images is a technique for recovering a high-resolution image from a low-resolution image and/or image sequence. Optical flow is the appearance motion of points on the image. Therefore, super-resolution optical flow computation yields the appearance motion of each point on the high-resolution image from a sequence of low-resolution images. We combine variational super-resolution and variational optical flow computation in super-resolution optical flow computation. Our method directly computes the gradient and spatial difference of high-resolution images from those of low-resolution images, without computing any high-resolution images used as intermediate data for the computation of optical flow vectors of the high-resolution image.     

基于光流场分析的红外图像自动配准方法研究

提出了一种基于光流场分析的准确的红外图像自动配准方法．该方法可分为两个过程：先是利用全局光流场完成两幅图像背景区域的配准；其次利用由粗到细的层级匹配算法提取两幅图像中运动目标的特征点集，根据两组特征点集由最小二乘法计算出运动目标的变换参数，完成运动目标的配准．对一定研究领域的红外图像自动配准的仿真实验表明：该方法准确且对场景的运动有很好的鲁棒性．     

基于改进ORB的图像特征匹配算法研究

针对ORB算法特征匹配精度低的缺陷,结合金字塔光流特性,提出一种优化ORB特征匹配的方法.首先,采用区域分块法对待匹配图像进行处理,挑选出最佳匹配子块,缩小无效匹配区域;接着,对子块提取ORB关键字并计算匹配描述子得到粗匹配点对,采用金字塔光流法追踪ORB特征点,求解特征点的运动位移矢量,以此剔除粗匹配部分错误的匹配对...     

Robust optical flow estimation based on a sparse motion trajectory set

This paper presents an approach to the problem of estimating a dense optical flow field. The approach is based on a multiframe, irregularly spaced motion trajectory set, where each trajectory describes the motion of a given point as a function of time. From this motion trajectory set a dense flow field is estimated using a process of interpolation. A set of localized motion models are estimated, with each pixel labeled as belonging to one of the motion models. A Markov random field framework is adopted, allowing the incorporation of contextual constraints to encourage region-like structures. The approach is compared with a number of conventional optical flow estimation algorithms taken over a number of real and synthetic sequences. Results indicate that the method produces more accurate results for sequences with known ground truth flow. Also, applying the method to real sequences with unknown flow results in lower DFD, for all of the sequences tested.     

增量式SFM中特征点检测与匹配方法的改进

图像特征点的提取与匹配是增量式SFM重建系统中至关重要的一步。为了提高匹配的准确率以及有效匹配点对的数量,提出了一种改进方法:首先在多尺度空间中利用自适应阈值的FAST角点检测算法获取特征点;然后计算特征点与其多个环形邻域之间的灰度对比信息,再与采样区域的局部梯度信息融合得到特征点描述子;接着利用曼哈顿距离与切比雪夫距离的线性组合代替欧氏距离完成特征点的稀疏匹配;最后利用稀疏匹配结果作为种子点对进行同步生长,在多约束条件下得到最终的稠密匹配结果。在Oxford数据集上的实验证明了改进的稀疏匹配算法的准确率与有效匹配点对数量都高于SIFT算法,在增量式SFM系统中的实验证明了稀疏匹配与稠密匹配的组合算法可以获得更好的重建效果。     

基于邻域一致性的极化SAR图像仿射配准

极化SAR图像的配准是极化SAR图像处理的基础,需要具备较高的精度与速度.基于深度学习的极化SAR图像配准大多数是结合图像块特征的匹配与基于随机抽样一致性的参数迭代估计来实现的.目前尚未实现端到端的基于深度卷积神经网络的一步仿射配准.该文提出了一种基于弱监督学习的端到端极化SAR图像配准框架,无需图像切块处理或迭代参数...     

Real-time video super resolution network using recurrent multi-branch dilated convolutions

Recent developments of video super-resolution reconstruction often exploit spatial and temporal contexts from input frame sequence by making use of explicit motion estimation, e.g., optical flow, which may introduce accumulated errors and requires huge computations to obtain an accurate estimation. In this paper, we propose a novel multi-branch dilated convolution module for real-time frame alignment without explicit motion estimation, which is incorporated with the depthwise separable up-sampling module to formulate a sophisticated real-time video super-resolution network. Specifically, the proposed video super-resolution framework can efficiently acquire a larger receptive field and learn spatial–temporal features of multiple scales with minimal computational operations and memory requirements. Extensive experiments show that the proposed super-resolution network outperforms current state-of-the-art real-time video super-resolution networks, e.g., VESPCN and 3DVSRnet, in terms of PSNR values (0.49 dB and 0.17 dB) on average in various datasets, but requires less multiplication operations.     

基于深度空时域特征融合的高动态空中多形态目标检测方法(特邀)

针对复杂背景下,依靠高超声速飞行器搭载的红外探测器对高动态空中目标的可靠探测和精确识别问题,提出了一种基于深度空时域特征融合的空中多形态目标检测方法。设计了加权双向循环特征金字塔结构提取多形态目标静态特征,并引入可切换空洞卷积,增大感受野的同时减少空域信息损失。对于时序运动特征的提取,为了抑制复杂背景噪声的同时将角点信息集中到运动区域中,通过特征点匹配法生成掩膜图,之后进行光流计算,根据计算结果设计稀疏光流特征图,利用3D卷积提取多个连续帧图像中包含的时序特征,生成三维时序运动特征图。最后,通过对图像静态特征与时序运动特征进行通道维度的拼接,实现深度空时域特征融合。大量的对比实验表明,文中方法可明显减少复杂背景下的虚假识别概率,具备高实时性的同时目标识别准确率达89.87%,满足高动态下的红外目标智能检测识别需求。     

非刚性稠密匹配大位移运动光流估计

光流场是目标检测,无人机定位等众多计算机视觉任务的重要基础.本文针对非刚性大位移运动等困难运动类型图像序列光流计算的准确性与鲁棒性问题,提出一种基于非刚性稠密匹配的TV-L1（Total Variational with L1 norm,TV-L1）大位移光流计算方法.首先,使用非刚性稠密块匹配计算图像序列初始最近邻域场,其次根据图像相邻块区域的相似性消除初始最近邻域场中的非一致性区域以得到准确的图像最近邻域场.然后,在图像金字塔分层计算框架下,将图像最近邻域场引入基于非局部约束的TV-L1光流估计模型,通过Quadratic Pseudo-Boolean Optimization（QPBO）融合算法在金字塔分层图像光流计算时对TV-L1模型光流估计进行大位移运动补偿.最后,采用标准测试图像序列对本文方法和当前代表性的变分方法LDOF（Large Displacement Optical Flow,LDOF）、Classic+NL、NNF（Nearest Neighbor Fields,NNF）以及深度学习方法FlowNet2.0进行对比分析.实验结果表明,本文方法能有效提高非刚性运动、大位移运动以及运动遮挡等困难运动类型光流估计的精度与鲁棒性.     

3D modeling and rendering from multiple wide-baseline images by match propagation

In this paper, we present a method for modeling a complex scene from a small set of input images taken from widely separated viewpoints and then synthesizing novel views. First, we find sparse correspondences across multiple input images and calibrate these input images taken with unknown cameras. Then one of the input images is chosen as the reference image for modeling by match propagation. A sparse set of reliably matched pixels in the reference image is initially selected and then propagated to neighboring pixels based on both the clustering-based light invariant photoconsistency constraint and the data-driven depth smoothness constraint, which are integrated into a pixel matching quality function to efficiently deal with occlusions, light changes and depth discontinuity. Finally, a novel view rendering algorithm is developed to fast synthesize a novel view by match propagation again. Experimental results show that the proposed method can produce good scene models from a small set of widely separated images and synthesize novel views in good quality.     

Robust weighted graph transformation matching for rigid and nonrigid image registration

This paper presents an automatic point matching algorithm for establishing accurate match correspondences in two or more images. The proposed algorithm utilizes a group of feature points to explore their geometrical relationship in a graph arrangement. The algorithm starts with a set of matches (including outliers) between the two images. A set of nondirectional graphs is then generated for each feature and its K nearest matches (chosen from the initial set). Using the angular distances between edges that connect a feature point to its K nearest neighbors in the graph, the algorithm finds a graph in the second image that is similar to the first graph. In the case of a graph including outliers, the algorithm removes such outliers (one by one, according to their strength) from the graph and re-evaluates the angles until the two graphs are matched or discarded. This is a simple intuitive and robust algorithm that is inspired by a previous work. Experimental results demonstrate the superior performance of this algorithm under various conditions, such as rigid and nonrigid transformations, ambiguity due to partial occlusions or match correspondence multiplicity, scale, and larger view variation.     

Automatic construction of parts+geometry models for initializing groupwise registration

Groupwise nonrigid image registration is a powerful tool to automatically establish correspondences across sets of images. Such correspondences are widely used for constructing statistical models of shape and appearance. As existing techniques usually treat registration as an optimization problem, a good initialization is required. Although the standard initialization-affine transformation-generally works well, it is often inadequate when registering images of complex structures. In this paper we present a more sophisticated method that uses the sparse matches of a parts+geometry model as the initialization. We show that both the model and its matches can be automatically obtained, and that the matches are able to effectively initialize a groupwise nonrigid registration algorithm, leading to accurate dense correspondences. We also show that the dense mesh models constructed during the groupwise registration process can be used to accurately annotate new images. We demonstrate the efficacy of the approach on three datasets of increasing difficulty, and report on a detailed quantitative evaluation of its performance.     

基于先验信息与密集连接网络的人脸超分辨率重建方法

图像超分辨率在医疗和安防等领域应用广泛,本文针对传统超分辨率重建(super-resolution reconstruction, SR)方法无法重建出边缘特征图像的不足,提出了一种先验信息与密集连接网络模型的重建方案,利用考虑输入统计信息的残差特征的不同组合,引入了多注意力模块,通过与主干网络结构协作,在不增加额外模块的情况下提高了网络性能。新提出的模型与现有复杂结构的技术(state-of-the-art, SOTA)模型相比,具有更好的性能。为了避免输入的身份特征会急剧漂移的问题,提出了一种基于先验信息引入注意力机制网络模块来分辨真实低分辨率(low resolution, LR)对应物的模型,这种模型在捕获运动噪声等方面具有优势。经实验验证得出,本文方法相比其他主流方法,在评价指标和主观可视化分析方面更具优势。     

An efficient search strategy for block motion estimation usingimage features

Block motion estimation using the exhaustive full search is computationally intensive. Fast search algorithms offered in the past tend to reduce the amount of computation by limiting the number of locations to be searched. Nearly all of these algorithms rely on this assumption: the mean absolute difference (MAD) distortion function increases monotonically as the search location moves away from the global minimum. Essentially, this assumption requires that the MAD error surface be unimodal over the search window. Unfortunately, this is usually not true in real-world video signals. However, we can reasonably assume that it is monotonic in a small neighborhood around the global minimum. Consequently, one simple strategy, but perhaps the most efficient and reliable, is to place the checking point as close as possible to the global minimum. In this paper, some image features are suggested to locate the initial search points. Such a guided scheme is based on the location of certain feature points. After applying a feature detecting process to each frame to extract a set of feature points as matching primitives, we have extensively studied the statistical behavior of these matching primitives, and found that they are highly correlated with the MAD error surface of real-world motion vectors. These correlation characteristics are extremely useful for fast search algorithms. The results are robust and the implementation could be very efficient. A beautiful point of our approach is that the proposed search algorithm can work together with other block motion estimation algorithms. Results of our experiment on applying the present approach to the block-based gradient descent search algorithm (BBGDS), the diamond search algorithm (DS) and our previously proposed edge-oriented block motion estimation show that the proposed search strategy is able to strengthen these searching algorithms. As compared to the conventional approach, the new algorithm, through the extraction of image features, is more robust, produces smaller motion compensation errors, and has a simple computational complexity.     

基于光流预测的直线对应算法

对基于光流预测的直线对应算法进行了深入的研究,贡献在于:（1）提出了改进的光流估计算法,改善了图像低纹理区域的光流估计,优于Chiba光流估计算法;（2）以直线全局参数和基于灰度的直线方向作为相似度测度,能够得到可靠的直线对应,并区分平行靠近的直线;（3）提出投影相似度测度处理直线多段对应问题。      

基于双向稀疏光流融合的小目标检测方法

针对动态背景下的小目标检测问题,提出了基于双向稀疏光流融合的目标检测方法.首先采用FAST方法提取当前帧图像中的角点,然后在连续的三帧图像中进行前、后向稀疏光流跟踪,确定正确匹配的特征点对,利用匹配的特征点对计算用于背景补偿的帧间运动参数,最后在背景补偿的基础上进行三帧差分,以检测出图像中的运动小目标.实验结果显示,本算法能够很好地解决背景和目标同时快速运动的问题,为运动目标的跟踪奠定基础.     

融合特征光流与角点特征的图像特征匹配算法研究

图像特征匹配是视觉里程计的重要环节,针对视觉图像序列特征点匹配中存在的匹配精度低问题,提出一种融合金字塔特征光流与角点特征的精确快速图像特征匹配算法。算法首先利用ORB(二进制定向简单描述符)算法快速提取图像特征点,然后融合金字塔Lucas-Kanade特征光流的追踪特性,使用局部特征窗口计算图像特征点位移矢量。接着针对图像特征的匹配对齐问题以及特征丢失问题,算法采用K最近邻半径搜索作为特征滤波器移除混淆的匹配,最后使用RANSAC(Random Sample Consensus)算法剔除冗余误匹配点对,提高匹配率。通过多组实验数据对比,该算法的图像特征匹配率可达到98%。对比传统的ORB特征匹配算法,该算法在实时性和图像特征匹配精度上均有显著提高。     

Closed-form connectivity-preserving solutions for motioncompensation using 2-D meshes

Motion compensation using two-dimensional (2-D) mesh models requires computation of the parameters of a spatial transformation for each mesh element (patch). It is well known that the parameters of an affine (bilinear or perspective) mapping can be uniquely estimated from three (four) point correspondences (at the vertices of a triangular or quadrilateral mesh element). On the other hand, overdetermined solutions using more than the required minimum number of point correspondences provide increased robustness against correspondence-estimation errors, however, this necessitates special consideration to preserve mesh-connectivity. This paper presents closed-form, overdetermined solutions for least squares estimation of affine motion parameters for a triangular mesh, which preserve mesh-connectivity using patch-based or node-based connectivity constraints. In particular, four new algorithms are presented: patch-constrained methods using point correspondences or spatio-temporal intensity gradients, and node-constrained methods using point correspondences or spatio-temporal intensity gradients. The methods using point correspondences can be viewed as postprocessing of a dense motion field for best representation in terms of a set of irregularly spaced samples. The methods that are based on spatio-temporal intensity gradients offer closed-form solutions for direct estimation of the best node-point motion vectors (equivalently the best transformation parameters). We show that the performance of the proposed closed-form solutions are comparable to those of the alternative search-based solutions at a fraction of the computational cost.     

Iterative colour correction of multicamera systems using corresponding feature points

Multiview images captured by multicamera systems are generally not uniform in colour domain. In this paper, we propose a novel colour correction method of multicamera systems that can (i) be applied to not only dense multicamera system, but also sparse multicamera configuration and (ii) obtain an average colour pattern among all cameras. Our proposed colour correction method starts from any camera on the array sequentially, following a certain path, for pairs of cameras, until it reaches the starting point and triggers several iterations. The iteration stops when the correction applied to the images becomes small enough. We propose to calculate the colour correction transformation based on energy minimisation using a dynamic programming of a nonlinearly weighted Gaussian-based kernel density function of geometrically corresponding feature points, obtained by the modified scale invariant feature transformation (SIFT) method, from several time instances and their Gaussian-filtered images. This approach guarantees the convergence of the iteration procedure without any visible colour distortion. The colour correction is done for each colour channel independently. The process is entirely automatic, after estimation of the parameters through the algorithm. Experimental results show that the proposed iteration-based algorithm can colour-correct the dense/sparse multicamera system. The correction is always converged with average colour intensity among viewpoint, and out-performs the conventional method.     

基于卷积稀疏自编码的图像超分辨率重建

针对卷积稀疏编码算法中特征映射的准确性的问题,为了进一步提高图像超分辨率重建的的质量,文中提出一种基于卷积稀疏自编码的图像超分辨率重建算法。该算法首先在预训练阶段利用稀疏自编码器对输入高低分辨率图像分别进行训练,得到对应的图像稀疏特征表示;然后再由卷积神经网络根据得到的稀疏系数共同训练相应的滤波器及特征映射函数并更新到最优解;最后由高分辨率滤波器和对应的稀疏表示系数卷积求和,得到高分辨率重建图像估计。实验结果显示,改进算法的峰值信噪比（PSNR）结果较卷积稀疏编码算法提高了近0.1 dB,有效提高了重建图像的质量。