Computing optical flow across multiple scales: An adaptive coarse-to-fine strategy

Single-scale approaches to the determination of the optical flow field from the time-varying brightness pattern assume that spatio-temporal discretization is adequate for representing the patterns and motions in a scene. However, the choice of an appropriate spatial resolution is subject to conflicting, scene-dependent, constraints. In intensity-base methods for recovering optical flow, derivative estimation is more accurate for long wavelengths and slow velocities (with respect to the spatial and temporal discretization steps). On the contrary, short wavelengths and fast motions are required in order to reduce the errors caused by noise in the image acquisition and quantization process.Estimating motion across different spatial scales should ameliorate this problem. However, homogeneous multiscale approaches, such as the standard multigrid algorithm, do not improve this situation, because an optimal velocity estimate at a given spatial scale is likely to be corrupted at a finer scale. We propose an adaptive multiscale method, where the discretization scale is chosen locally according to an estimate of the relative error in the velocity estimation, based on image properties.Results for synthetic and video-acquired images show that our coarse-to-fine method, fully parallel at each scale, provides substantially better estimates of optical flow than do conventional algorithms, while adding little computational cost.     

Motion detail preserving optical flow estimation

A common problem of optical flow estimation in the multiscale variational framework is that fine motion structures cannot always be correctly estimated, especially for regions with significant and abrupt displacement variation. A novel extended coarse-to-fine (EC2F) refinement framework is introduced in this paper to address this issue, which reduces the reliance of flow estimates on their initial values propagated from the coarse level and enables recovering many motion details in each scale. The contribution of this paper also includes adaptation of the objective function to handle outliers and development of a new optimization procedure. The effectiveness of our algorithm is demonstrated by Middlebury optical flow benchmarkmarking and by experiments on challenging examples that involve large-displacement motion.     

Coarse-to-fine stereo vision with accurate 3D boundaries

This paper presents methods for efficient recovery of accurate binocular disparity estimates in the vicinity of 3D surface discontinuities. Of particular concern are methods that impact coarse-to-fine, local block-based matching as it forms the basis of the fastest and the most resource efficient stereo computation procedures. A novel coarse-to-fine refinement procedure that adapts match window support across scale to ameliorate corruption of disparity estimates near boundaries is presented. Extensions are included to account for half-occlusions and colour uniformity. Empirical results show that incorporation of these advances in the standard coarse-to-fine, block matching framework reduces disparity errors by more than a factor of two, while performing little extra computation, preserving low complexity and the parallel/pipeline nature of the framework. Moreover, the proposed advances prove to be beneficial for CTF global matchers as well.     

Semi-dense and robust image registration by shift adapted weighted aggregation and variational completion

This work presents a novel approach for both stereo and optical flow that deals with large displacements, depth/motion discontinuities and occlusions. The proposed method comprises two main steps. First, a novel local stereo matching algorithm is presented, whose main novelty relies in the block-matching aggregation step. We adopt an adaptive support weights approach in which the weight distribution favors pixels that share the same displacement with the reference one. State-of-the-art methods make the weight function depend only on image features. On the contrary, the proposed weight function depends additionally on the tested shift, by giving more importance to those pixels in the block-matching with smaller cost, as these are supposed to have the tested displacement. Moreover, the method is embedded into a pyramidal procedure to locally limit the search range, which helps to reduce ambiguities in the matching process and saves computational time. Second, the non-dense local estimation is filtered and interpolated by means of a new variational formulation making use of intermediate scale estimates of the local procedure. This permits to keep the fine details estimated at full resolution while being robust to noise and untextured areas using estimates at coarser scales. The introduced variational formulation as well as the block-matching algorithm are robust to illumination changes. We test our algorithm for both stereo and optical flow public datasets showing competitive results.     

面向运动分割优化的变分光流计算方法研究

面向运动分割的需求,围绕变分光流计算中的运动边缘保留问题,对反应-扩散式光流计算模型中的扩散张量设计与改进方法进行系统深入研究。 在分析已有设计方法的基础上,提出了一种融合图像和流场信息驱动的扩散张量设计方法,并利用该扩散张量建立光流计算模型,然后在多尺度计算框架下给出了相应的偏微分方程数值计算方法。 理论分析与对比实验结果表明,这种设计方法能有效克服现有方法提取运动边缘不精确的缺点,更准确地刻画出运动边缘,提高光流的计算精度。     

Clustering factor estimation for totally clustered attributes

Cost models based on the clustering factor (CF) of the attributes have been proposed and shown to be attractive for block access estimation in databases, thanks to their accuracy and economy of use. While query optimizers can use the actual CFs, measured from the data, physical design methods and tools must rely on estimates before the data are stored.

In this paper we present a CF estimation procedure which can be applied to totally clustered attributes (e.g. ordered attributes). Simple and accurate approximations of the derived formulas are also introduced.

Simulations show the accuracy of the proposed CF estimates and the improvment in their behaviour compared to previously published estimates. Reliability for physical design of cost models based on the CF in the presence of a skewed data distribution is also discussed.     

A coarse-to-fine disparity energy model with both phase-shift and position-shift receptive field mechanisms

Numerous studies suggest that the visual system uses both phase- and position-shift receptive field (RF) mechanisms for the processing of binocular disparity. Although the difference between these two mechanisms has been analyzed before, previous work mainly focused on disparity tuning curves instead of population responses. However, tuning curve and population response can exhibit different characteristics, and it is the latter that determines disparity estimation. Here we demonstrate, in the framework of the disparity energy model, that for relatively small disparities, the population response generated by the phase-shift mechanism is more reliable than that generated by the position-shift mechanism. This is true over a wide range of parameters, including the RF orientation. Since the phase model has its own drawbacks of underestimating large stimulus disparity and covering only a restricted range of disparity at a given scale, we propose a coarse-to-fine algorithm for disparity computation with a hybrid of phase-shift and position-shift components. In this algorithm, disparity at each scale is always estimated by the phase-shift mechanism to take advantage of its higher reliability. Since the phase-based estimation is most accurate at the smallest scale when the disparity is correspondingly small, the algorithm iteratively reduces the input disparity from coarse to fine scales by introducing a constant position-shift component to all cells for a given location in order to offset the stimulus disparity at that location. The model also incorporates orientation pooling and spatial pooling to further enhance reliability. We have tested the algorithm on both synthetic and natural stereo images and found that it often performs better than a simple scale-averaging procedure.     

Multiple representations for cartographic objects in a multi-scale tree—An intelligent graphical zoom

In Geographic Information Systems, a function to draw cartographic sketches quickly and in arbitrary scales is needed. This calls for cartographic generalization, a notoriously difficult problem. Efforts to achieve automatic cartographic generalization were successful for specific aspects, but no complete solution is known, nor are there any expected within the immediate future. In practical applications, a base map is stored and its scale is changed. Without major distortions, only changes to twice or half the original scale are feasible by simple numeric scale change. Everything beyond this requires adaptation of symbols, selection of objects, placements of labels, etc.

Extending ideas of hierarchies or pyramids, where representations of the same objects at different scales are stored, a multi-scale, hierarchial spatial model is proposed. Objects with increasing detail are stored in levels and can be used to compose a map at a particular scale. Applied to the particular problem of cartographic mapping, this results in a multi-scale cartographic tree. The same concept can be used equally well for other applications, which require rendering of objects at different levels of detail.

The structure of the multi-scale tree is explained. It is based on a trade-off between storage and computation, replacing all steps which are difficult to automate by essentially redundant storage. The dominant operation is ‘zoom,’ which moves towards a more detailed level, intelligently replacing the current graphical representation with the more detailed one, appropriate for the selected new scale. Methods to select objects for rendering are based on the principle of equal information density. Principles of possible implementations are presented.     

Scale selection for anisotropic diffusion filter by Markov random field model

The selection of stopping time (i.e., scale) significantly affects the performance of anisotropic diffusion filter for image denoising. This paper designs a Markov random field (MRF) scale selection model, which selects scales for image segments, then the denoised image is the composition of segments at their optimal scales in the scale space. Firstly, statistics-based scale selection criteria are proposed for image segments. Then we design a scale selection energy function in the MRF framework by considering the scale coherence between neighboring segments. A segment-based noise estimation algorithm is also developed to estimate the noise statistics efficiently. Experiments show that the performance of MRF scale selection model is much better than the previous global scale selection schemes. Combined with this scale selection model, the anisotropic diffusion filter is comparable to or even outperform the state-of-the-art denoising methods in performance.     

Estimating uncertainty in spatial microsimulation approaches to small area estimation: A new approach to solving an old problem

A wide range of user groups from policy makers to media commentators demand ever more spatially detailed information yet the desired data are often not available at fine spatial scales. Increasingly, small area estimation (SAE) techniques are called upon to fill in these informational gaps by downscaling survey outcome variables of interest based on the relationships seen with key covariate data. In the process SAE techniques both rely extensively on small area Census data to enable their estimation and offer potential future substitute data sources in the event of Census data becoming unavailable. Whilst statistical approaches to SAE routinely incorporate intervals of uncertainty around central point estimates in order to indicate their likely accuracy, the continued absence of such intervals from spatial microsimulation SAE approaches severely limits their utility and arguably represents their key methodological weakness. The present article presents an innovative approach to resolving this key methodological gap based on the estimation of variance of the between-area error term from a multilevel regression specification of the constraint selection for iterative proportional fitting (IPF). The performance of the estimated credible intervals are validated against known Census data at the target small area and show an extremely high level of performance. As well as offering an innovative solution to this long-standing methodological problem, it is hoped more broadly that the research will stimulate the spatial microsimulation community to adopt and build on these foundations so that we can collectively move to a position where intervals of uncertainty are delivered routinely around spatial microsimulation small area point estimates.     

On optimal robust disturbance attenuation

In this article, we give an explicit solution to the nonstandard H^∞ problem which includes the optimal robust disturbance attenuation problem and the two-disc problem. The solution depends on an operator which is analogous to the Sarason operator for the standard H^∞ problem, it allows to quantify the optimal performance of feedback systems and provides an explicit formula for the optimal controller. The nonstandard H^∞ problem is perhaps the simplest control problem which deals with significant plant uncertainty. Surprisingly, relatively little work has been done on it despite the large number of papers in the H^∞ control literature.     

基于边缘保护尺度空间的形变配准方法及在自适应放疗中的应用

计划CT图像与锥形束CT (Cone beam CT, CBCT)图像的配准是基于CBCT图像引导放射治疗(Image guided radiation therapy, IGRT)系统中实现自适应放疗(Adaptive radiation therapy, ART)的关键部分.边缘保护多尺度空间基于非线性扩散模型,可以为基于互信息的配准提供丰富的空间位置信息.为了提高系统中配准算法性能,本文提出了一种基于边缘保护尺度空间与自由形变模型(Free form deformation, FFD)相结合的多尺度形变配准方法.我们采用了在不同的尺度上根据精细程度选择相应的自由形变控制点数,由粗及精地恢复形变.同时, 提出了自动获取非线性扩散模型中平滑参数λ的方法来实现全自动配准. 实验结果表明,本文提出的方法用于基于CBCT的图像引导放射系统时,可实现日常放疗时的CBCT图像和计划CT图像准确且快速的配准.通过获得的形变域,可实现CBCT图像肿瘤靶区、危及器官(Organ at risk, OR)和等剂量线的自动勾画,从而实现剂量体积直方图(Dose volume histograms, DVH)分析.最终实现了放疗计划从CT到CBCT的自适应转移.     

Motion estimation and segmentation

In the general structure-from-motion (SFM) problem involving several moving objects in a scene, the essential first step is to segment moving objects independently. We attempt to deal with the problem of optical flow estimation and motion segmentation over a pair of images. We apply a mean field technique to determine optical flow and motion boundaries and present a deterministic algorithm. Since motion discontinuities represented by line process are embedded in the estimation of the optical flow, our algorithm provides accurate estimates of optical flow especially along motion boundaries and handles occlusion and multiple motions. We show that the proposed algorithm outperforms other well-known algorithms in terms of estimation accuracy and timing.     

一种基于光流和能量的图像匹配算法

结合光流与图像信息,提出一种获取稠密视差的图像匹配算法.首先对于基线较大的左右图像,在多分辨率框架下采用由粗到精的策略计算光流,从而实现大偏移量时的光流获取.其次为了避免光流在图像边界上的不可靠性,通过光流计算所得的光流场作为初始视差图,采用基于能量的方法依据对应的图像梯度场对光流场内部进行平滑并保持边缘的不连续性,最终得到精准稠密的视差图.实验验证,该方法是一种行之有效的图像匹配算法.     

A parameterless feature ranking algorithm based on MI

A parameterless feature ranking approach is presented for feature selection in the pattern classification task. Compared with Battiti's mutual information feature selection (MIFS) and Kwak and Choi's MIFS-U methods, the proposed method derives an estimation of the conditional MI between the candidate feature f_i and the output class C given the subset of selected features S, i.e. I(C;f_iS), without any parameters like β in MIFS and MIFS-U methods to be preset. Thus, the intractable problem can be avoided completely, which is how to choose an appropriate value for β to achieve the tradeoff between the relevance to the output classes and the redundancy with the already-selected features. Furthermore, a modified greedy feature selection algorithm called the second order MI feature selection approach (SOMIFS) is proposed. Experimental results demonstrate the superiority of SOMIFS in terms of both synthetic and benchmark data sets.     

Temporal Multi-Scale Models for Flow and Acceleration

A model for computing image flow in image sequences containing a very wide range of instantaneous flows is proposed. This model integrates the spatio-temporal image derivatives from multiple temporal scales to provide both reliable and accurate instantaneous flow estimates. The integration employs robust regression and automatic scale weighting in a generalized brightness constancy framework. In addition to instantaneous flow estimation the model supports recovery of dense estimates of image acceleration and can be readily combined with parameterized flow and acceleration models. A demonstration of performance on image sequences of typical human actions taken with a high frame-rate camera is given.     

Local scale control for edge detection and blur estimation

We show that knowledge of sensor properties and operator norms can be exploited to define a unique, locally computable minimum reliable scale for local estimation at each point in the image. This method for local scale control is applied to the problem of detecting and localizing edges in images with shallow depth of field and shadows. We show that edges spanning a broad range of blur scales and contrasts can be recovered accurately by a single system with no input parameters other than the second moment of the sensor noise. A natural dividend of this approach is a measure of the thickness of contours which can be used to estimate focal and penumbral blur. Local scale control is shown to be important for the estimation of blur in complex images, where the potential for interference between nearby edges of very different blur scale requires that estimates be made at the minimum reliable scale     

Direct detection of flow discontinuities by 3D curvature operators

We present an approach for the direct detection of flow discontinuities which avoids explicit computation of a dense optic flow field. It is based on regarding the time varying image as a hypersurface in four-dimensional space and on using the Gaussian curvature properties of this hypersurface as a direct indicator for the presence of motion discontinuities. An easy to implement, nonlinear operator is suggested and possible extensions of the basic scheme are discussed.