基于变分光流估计的肺部4D-CT图像超分辨率重建

由于受到扫描时间和照射剂量的限制,肺部4D-CT数据中纵向采样率远小于面内采样率.为了得到更高质量的肺部图像,从医学图像固有的自相似性出发,提出了一种基于局部和全局相结合的变分光流估计的图像序列超分辨率重建技术,用于提高4D-CT图像重建质量.首先,构建了一个用于求解肺部4D-CT不同相位图像之间的光流场的变分光流模型;然后,利用快速交替方向乘子法求解该模型,得到不同相位图像之间的光流场;最后,基于光流场,并利用非局部迭代反投影超分辨率重建算法,实现了高分辨率肺部图像的重建.实验结果表明：与已有算法相比,本方法在增强图像纹理结构的同时更好地保留了图像的轮廓.     

Over-Parameterized Variational Optical Flow

A novel optical flow estimation process based on a spatio-temporal model with varying coefficients multiplying a set of basis functions at each pixel is introduced. Previous optical flow estimation methodologies did not use such an over parameterized representation of the flow field as the problem is ill-posed even without introducing any additional parameters: Neighborhood based methods of the Lucas–Kanade type determine the flow at each pixel by constraining the flow to be described by a few parameters in small neighborhoods. Modern variational methods represent the optic flow directly via the flow field components at each pixel. The benefit of over-parametrization becomes evident in the smoothness term, which instead of directly penalizing for changes in the optic flow, accumulates a cost of deviating from the assumed optic flow model. Our proposed method is very general and the classical variational optical flow techniques are special cases of it, when used in conjunction with constant basis functions. Experimental results with the novel flow estimation process yield significant improvements with respect to the best results published so far.     

A Variational Framework for Retinex

Retinex theory addresses the problem of separating the illumination from the reflectance in a given image and thereby compensating for non-uniform lighting. This is in general an ill-posed problem. In this paper we propose a variational model for the Retinex problem that unifies previous methods. Similar to previous algorithms, it assumes spatial smoothness of the illumination field. In addition, knowledge of the limited dynamic range of the reflectance is used as a constraint in the recovery process. A penalty term is also included, exploiting a-priori knowledge of the nature of the reflectance image. The proposed formulation adopts a Bayesian view point of the estimation problem, which leads to an algebraic regularization term, that contributes to better conditioning of the reconstruction problem.Based on the proposed variational model, we show that the illumination estimation problem can be formulated as a Quadratic Programming optimization problem. An efficient multi-resolution algorithm is proposed. It exploits the spatial correlation in the reflectance and illumination images. Applications of the algorithm to various color images yield promising results.     

A New Parallel Differential Method for Optical Flow Estimation

Optical flow estimation is a recurrent problem in several disciplines and assumes a primary importance in a number of applicative fields such as medical imaging [12], computer vision [6], productive process control [4], etc. In this paper, a differential method for optical flow evaluation is being presented. It employs a new error formulation that ensures a more than satisfactory image reconstruction in those points which are free of motion discontinuity. A dynamic scheme of brightness-sample processing has been used to regularise the motion field. A technique based on the concurrent processing of sequences with multiple pairs of images has also been developed for improving detection and resolution of mobile objects on the scene, if they exist. This approach permits to detect motions ranging from a fraction of a pixel to a few pixels per frame. Good results, even on noisy sequences and without the need of a filtering pre-processing stage, can be achieved. The intrinsic method structure can be exploited for favourable implementation on multi-processor systems with a scalable degree of parallelism. Several sequences, some with noise and presenting various types of motions, have been used for evaluating the performances and the effectiveness of the method. Carmelo Lodato received his Dr. Ing. Degree in Civil Engineering from the University of Palermo, Italy, in 1987. He is Researcher at the High Performance Computing and Networking Institute (ICAR) of the Italian National Research Council (CNR). His current research interests include computer vision, image processing, motion analysis, optimization and stochastic algorithms. Salvatore Lopes received his Dr. Ing. Degree (summa com laude) in Nuclear Engineering from the University of Palermo, Italy, in 1988. He is Researcher at the High Performance Computing and Networking Institute (ICAR) of the Italian National Research Council (CNR). His current research interests include computer vision, image processing, motion analysis, optimization and stochastic algorithms.     

FALDOI: A New Minimization Strategy for Large Displacement Variational Optical Flow

We propose a large displacement optical flow method that introduces a new strategy to compute a good local minimum of any optical flow energy functional. The method requires a given set of discrete matches, which can be extremely sparse, and an energy functional which locally guides the interpolation from those matches. In particular, the matches are used to guide a structured coordinate descent of the energy functional around these keypoints. It results in a two-step minimization method at the finest scale which is very robust to the inevitable outliers of the sparse matcher and able to capture large displacements of small objects. Its benefits over other variational methods that also rely on a set of sparse matches are its robustness against very few matches, high levels of noise, and outliers. We validate our proposal using several optical flow variational models. The results consistently outperform the coarse-to-fine approaches and achieve good qualitative and quantitative performance on the standard optical flow benchmarks.     

Multi-view Scene Flow Estimation: A View Centered Variational Approach

We present a novel method for recovering the 3D structure and scene flow from calibrated multi-view sequences. We propose a 3D point cloud parametrization of the 3D structure and scene flow that allows us to directly estimate the desired unknowns. A unified global energy functional is proposed to incorporate the information from the available sequences and simultaneously recover both depth and scene flow. The functional enforces multi-view geometric consistency and imposes brightness constancy and piecewise smoothness assumptions directly on the 3D unknowns. It inherently handles the challenges of discontinuities, occlusions, and large displacements. The main contribution of this work is the fusion of a 3D representation and an advanced variational framework that directly uses the available multi-view information. This formulation allows us to advantageously bind the 3D unknowns in time and space. Different from optical flow and disparity, the proposed method results in a nonlinear mapping between the images’ coordinates, thus giving rise to additional challenges in the optimization process. Our experiments on real and synthetic data demonstrate that the proposed method successfully recovers the 3D structure and scene flow despite the complicated nonconvex optimization problem.     

运动细节估计的光流场方法

针对现有的光流方法在处理大位移和估计运动图像细节方面存在的问题,提出一种结合图像细节特征的变分光流场模型.首先通过增加特征点的对应,采用自适应的保持边缘的正则项以及引入occlusion检测函数对经典光流模型进行了改进;其次,采用基于变分框架下的高斯金字塔方法以及加权中值滤波的方法对所提出的模型进行求解.大量的实验结果...     

基于分层置信度传播的光流估计方法

置信度传播算法作为一种有效的寻找图像间对应点的方法,近年来被广泛应用于光流估计.但是在估计大位移高精度光流时,将置信度传播直接应用于原图像会导致标签空间过大和处理时间过长的问题.为了克服这个缺点,我们提出了一种基于分层置信度传播的算法来估计高精度大位移光流.本文方法将输入图像视作马尔科夫随机场,为了提高效率,在超像素和像素两个层面上执行置信度传播.我们将超像素层得到的基础位移结果作为粗略的位移参考值,可以有效地减小像素层置信度传播的标签空间,并在有限的标签空间内得到高精度的光流估计结果.MPI Sintel光流数据集上的实验结果显示本文提出的方法在精度和速度上都取得了较好的结果.     

A Variational Framework for Exemplar-Based Image Inpainting

Non-local methods for image denoising and inpainting have gained considerable attention in recent years. This is in part due to their superior performance in textured images, a known weakness of purely local methods. Local methods on the other hand have demonstrated to be very appropriate for the recovering of geometric structures such as image edges. The synthesis of both types of methods is a trend in current research. Variational analysis in particular is an appropriate tool for a unified treatment of local and non-local methods. In this work we propose a general variational framework for non-local image inpainting, from which important and representative previous inpainting schemes can be derived, in addition to leading to novel ones. We explicitly study some of these, relating them to previous work and showing results on synthetic and real images.     

基于事件相机的连续光流估计

事件相机对场景的亮度变化进行成像, 输出异步事件流, 具有极低的延时, 受运动模糊问题影响较少. 因此, 可以利用事件相机解决高速运动场景下的光流(Optical flow, OF)估计问题. 基于亮度恒定假设和事件产生模型, 利用事件相机输出事件流的低延时性质, 融合存在运动模糊的亮度图像帧, 提出基于事件相机的连续光流估计算法, 提升了高速运动场景下的光流估计精度. 实验结果表明, 相比于现有的基于事件相机的光流估计算法, 该算法在平均端点误差、平均角度误差和均方误差3个指标上, 分别提升11%、45% 和8%. 在高速运动场景下, 该算法能够准确重建出高速运动目标的连续光流, 保证了存在运动模糊情况时, 光流估计的精度.     

Discrete Orthogonal Decomposition and Variational Fluid Flow Estimation

We exploit the mimetic finite difference method introduced by Hyman and Shashkov to present a framework for estimating vector fields and related scalar fields (divergence, curl) of physical interest from image sequences. Our approach provides a basis for consistent definitions of higher-order differential operators, for the analysis and a novel stability result concerning second-order div-curl regularizers, for novel variational schemes to the estimation of solenoidal (divergence-free) image flows, and to convergent numerical methods in terms of subspace corrections.     

A Perceptually Inspired Variational Framework for Color Enhancement

Basic phenomenology of human color vision has been widely taken as an inspiration to devise explicit color correction algorithms. The behavior of these models in terms of significative image features (such as, e.g., contrast and dispersion) can be difficult to characterize. To cope with this, we propose to use a variational formulation of color contrast enhancement that is inspired by the basic phenomenology of color perception. In particular, we devise a set of basic requirements to be fulfilled by an energy to be considered as 'perceptually inspired', showing that there is an explicit class of functionals satisfying all of them. We single out three explicit functionals that we consider of basic interest, showing similarities and differences with existing models. The minima of such functionals is computed using a gradient descent approach. We also present a general methodology to reduce the computational cost of the algorithms under analysis from O(N2) to O(N logN), being N the number of pixels of the input image.     

A Fourier Domain Framework for Variational Image Registration

Vector fields arise in many problems of computer vision, particularly in non-rigid registration. In this paper, we develop coupled partial differential equations (PDEs) to estimate vector fields that define the deformation between objects, and the contour or surface that defines the segmentation of the objects as well. We also explore the utility of inequality constraints applied to variational problems in vision such as estimation of deformation fields in non-rigid registration and tracking. To solve inequality constrained vector field estimation problems, we apply tools from the Kuhn-Tucker theorem in optimization theory. Our technique differs from recently popular joint segmentation and registration algorithms, particularly in its coupled set of PDEs derived from the same set of energy terms for registration and segmentation. We present both the theory and results that demonstrate our approach.

Histogram-Based Optical Flow for Motion Estimation in Ultrasound Imaging

Motion estimation on ultrasound data is often referred to as ‘Speckle Tracking’ in clinical environments and plays an important role in diagnosis and monitoring of cardiovascular diseases and the identification of abnormal cardiac motion. The impact of physical effects in the process of data acquisition raises many problems for conventional image processing techniques. The most significant difference to other medical data is its high level of speckle noise, which has completely different characteristics from other noise models, e.g., additive Gaussian noise. In this paper we address the problem of multiplicative speckle noise for motion estimation techniques that are based on optical flow methods and prove that the influence of this noise leads to wrong correspondences between image regions if not taken into account. To overcome these problems we propose the use of local statistics and introduce an optical flow method which uses histograms as discrete representations of local statistics for motion analysis. We show that this approach is more robust under the presence of speckle noise than classical optical flow methods.     

用改进Census变换实现光照变化鲁棒光流计算

针对当前光流算法在野外光照变化条件下计算精度不高的问题,提出一种基于改进Census变换的变分光流算法.该算法根据图像轮廓信息自适应选择变换窗口形状,以提高深度不连续区域图像子块信息描述的准确性;在变换窗口内构建完整的梯度流向量描述子,克服传统算法由于信息量不够而导致的不同图像子块区分度有限的问题;在构建二进制串时,与基准元素越近的点排在二进制串高位,减弱2帧图像因投影变形对光流求解的影响.在TV_L1变分光流计算框架下,用改进的Census变换描述子构建光流模型中的数据项.对图像进行高斯金字塔分解,并结合加权中值滤波进行分层光流估计.最后以Middlebury和KITTI数据库为测试平台,证明了文中算法的有效性.     

A Geometric Framework and a New Criterion in Optical Flow Modeling

We evaluate the dense optical flow between two frames via variational approach. In this paper, a new framework for deriving the regularization term is introduced giving a geometric insight into the action of a smoothing term. The framework is based on the Beltrami paradigm in image denoising. It includes a general formulation that unifies several previous methods. Using the proposed framework we also derive two novel anisotropic regularizers incorporating a new criterion that requires co-linearity between the gradients of optical flow components and possibly the intensity gradient. We call this criterion “alignment” and reveal its existence also in the celebrated Nagel and Enkelmann’s formulation. It is shown that the physical model of rotational motion of a rigid body, pure divergent/convergent flow and irrotational fluid flow, satisfy the alignment criterion in the flow field. Experimental tests in comparison to a recently published method show the capability of the new criterion in improving the optical flow estimations.

Improved Accuracy in Gradient-Based Optical Flow Estimation

Optical flow estimation by means of first derivatives can produce surprisingly accurate and dense optical flow fields. In particular, recent empirical evidence suggests that the method that is based on local optimization of first-order constancy constraints is among the most accurate and reliable methods available. Nevertheless, a systematic investigation of the effects of the various parameters for this algorithm is still lacking. This paper reports such an investigation. Performance is assessed in terms of flow-field accuracy, density, and resolution. The investigation yields new information regarding pre-filter, differentiator, least-squares neighborhood, and reliability test selection. Several changes to previously-employed parameter settings result in significant overall performance improvements, while they simultaneously reduce the computational cost of the estimator.     

Optical Flow Estimation Using Discontinuity Conforming Filters

Discontinuities and large image displacements pose some of the hardest problems in flow estimation. This paper proposes a set of filters that change shape to avoid blending of the constraints across discontinuity boundaries according to an incompatibility measure of the constraints of neighboring pixels. The algorithm is embedded in a coarse to fine multigrid scheme to address the problem of large displacements. We report results on real and synthetic images which show that the algorithm works very well.     

基于小波滤波器组的光流估计

为了解决光流估计时存在的孔径问题和时域混叠，在Bernard的理论基础上，给出了基于快速滤波器组的方法：小波函数在频域可以表示为滤波器组的乘积．所以时域的小波系数由傅里叶变换转换到频域。通过滤波器组快速求解。由于解析小波具有实小波所没有的优点．采用解析小波计算光流。文中给出该方法的结果。还给出了Horn-Schunck的正则化方法的结果。比较后可知，本文的方法能较好地对运动物体进行光流估计。     

深度光流估计方法研究进展

结合深度学习模型实现光流端到端的计算是当前计算机视觉领域的一个研究热点.文中对基于深度学习的光流估计方法进行总结和梳理.首先,介绍了光流的起源与定义;其次,总结了现有的数据集合和评价指标;最重要的是,着重从3个方面回顾了深度光流估计方法,包括有监督的深度光流估计方法、无监督的深度光流估计方法以及对现有光流估计方法的性能...