Optical flow approximation based motion object extraction for MPEG-2 video stream

This paper presents a compressed-domain motion object extraction algorithm based on optical flow approximation for MPEG-2 video stream. The discrete cosine transform (DCT) coefficients of P and B frames are estimated to reconstruct DC + 2AC image using their motion vectors and the DCT coefficients in I frames, which can be directly extracted from MPEG-2 compressed domain. Initial optical flow is estimated with Black’s optical flow estimation framework, in which DC image is substituted by DC + 2AC image to provide more intensity information. A high confidence measure is exploited to generate dense and accurate motion vector field by removing noisy and false motion vectors. Global motion estimation and iterative rejection are further utilized to separate foreground and background motion vectors. Region growing with automatic seed selection is performed to extract accurate object boundary by motion consistency model. The object boundary is further refined by partially decoding the boundary blocks to improve the accuracy. Experimental results on several test sequences demonstrate that the proposed approach can achieve compressed-domain video object extraction for MPEG-2 video stream in CIF format with real-time performance.     

Vector field analysis for multi-object behavior modeling

This paper proposes an end-to-end system to recognize multi-person behaviors in video, unifying different tasks like segmentation, modeling and recognition within a single optical flow based motion analysis framework. We show how optical flow can be used for analyzing activities of individual actors, as opposed to dense crowds, which is what the existing literature has concentrated on mostly. The algorithm consists of two steps — identification of motion patterns and modeling of motion patterns. Activities are analyzed using the underlying motion patterns which are formed by the optical flow field over a period of time. Streaklines are used to capture these motion patterns via integration of the flow field. To recognize the regions of interest, we utilize the Helmholtz decomposition to compute the divergence potential. The extrema or critical points of this potential indicates regions of high activity in the video, which are then represented as motion patterns by clustering the streaklines. We then present a method to compare two videos by measuring the similarity between their motion patterns using a combination of shape theory and subspace analysis. Such an analysis allows us to represent, compare and recognize a wide range of activities. We perform experiments on state-of-the-art datasets and show that the proposed method is suitable for natural videos in the presence of noise, background clutter and high intra class variations. Our method has two significant advantages over recent related approaches — it provides a single framework that takes care of both low-level and high-level visual analysis tasks, and is computationally efficient.     

基于图像的动目标检测技术

本文研究了光流估计的基本算法，从运动分割的角度的提出了使用光流分割方案来解决复杂背景下的动目标检测问题。文中分析了标准光流估计的缺陷及其原因，并使用鲁棒性估计技术提高了光流估计算法的实用性。最后给出了动目标检测算法的仿真结果，表明利用图像的运动特征来实现目标检测是一种很有效的方法。     

融合IMU去除运动模糊的改进光流匹配算法

为进一步提高视觉SLAM中的光流匹配精度和速度,提出一种融合惯性测量单元（inertial measurement unit,IMU）去除运动模糊的改进光流匹配算法。该算法首先利用IMU运动信息计算的点扩散函数去除运动模糊,提高特征点匹配率;其次在LK（Lucas-Kanade）光流的基础上引入梯度误差,并使用图像梯度L1范数作为正则项模拟稀疏噪声,构建代价函数;然后利用IMU预测特征点位置作为该算法初始值,并加入BB（Barzilar-Borwein）步长改进原有的高斯牛顿算法,提高计算速度。实验表明,通过两帧之间比较,该算法的效率和精度均优于LK光流法;然后将该算法集成到VINS-Mono框架,在数据集EuRoC上结果显示,该算法提高了原有框架的定位精度和鲁棒性。     

On 3-D scene flow and structure recovery from multiview image sequences

Two novel systems computing dense three-dimensional (3-D) scene flow and structure from multiview image sequences are described in this paper. We do not assume rigidity of the scene motion, thus allowing for nonrigid motion in the scene. The first system, integrated model-based system (IMS), assumes that each small local image region is undergoing 3-D affine motion. Non-linear motion model fitting based on both optical flow constraints and stereo constraints is then carried out on each local region in order to simultaneously estimate 3-D motion correspondences and structure. The second system is based on extended gradient-based system (EGS), a natural extension of two-dimensional (2-D) optical flow computation. In this method, a new hierarchical rule-based stereo matching algorithm is first developed to estimate the initial disparity map. Different available constraints under a multiview camera setup are further investigated and utilized in the proposed motion estimation. We use image segmentation information to adopt and maintain the motion and depth discontinuities. Within the framework for EGS, we present two different formulations for 3-D scene flow and structure computation. One formulation assumes that initial disparity map is accurate, while the other does not. Experimental results on both synthetic and real imagery demonstrate the effectiveness of our 3-D motion and structure recovery schemes. Empirical comparison between IMS and EGS is also reported.     

Non Uniform Multiresolution Method for Optical Flow and Phase Portrait Models: Environmental Applications

In this paper we define a complete framework for processing large image sequences for a global monitoring of short range oceanographic and atmospheric processes. This framework is based on the use of a non quadratic regularization technique for optical flow computation that preserves flow discontinuities. We also show that using an appropriate tessellation of the image according to an estimate of the motion field can improve optical flow accuracy and yields more reliable flows. This method defines a non uniform multiresolution approach for coarse to fine grid generation. It allows to locally increase the resolution of the grid according to the studied problem. Each added node refines the grid in a region of interest and increases the numerical accuracy of the solution in this region. We make use of such a method for solving the optical flow equation with a non quadratic regularization scheme allowing the computation of optical flow field while preserving its discontinuities. The second part of the paper deals with the interpretation of the obtained displacement field. For this purpose a phase portrait model used along with a new formulation of the approximation of an oriented flow field allowing to consider arbitrary polynomial phase portrait models for characterizing salient flow features. This new framework is used for processing oceanographic and atmospheric image sequences and presents an alternative to complex physical modeling techniques.     

Accurate optical flow computation under non-uniform brightness variations

In this paper, we present a very accurate algorithm for computing optical flow with non-uniform brightness variations. The proposed algorithm is based on a generalized dynamic image model (GDIM) in conjunction with a regularization framework to cope with the problem of non-uniform brightness variations. To alleviate flow constraint errors due to image aliasing and noise, we employ a reweighted least-squares method to suppress unreliable flow constraints, thus leading to robust estimation of optical flow. In addition, a dynamic smoothness adjustment scheme is proposed to efficiently suppress the smoothness constraint in the vicinity of the motion and brightness variation discontinuities, thereby preserving motion boundaries. We also employ a constraint refinement scheme, which aims at reducing the approximation errors in the first-order differential flow equation, to refine the optical flow estimation especially for large image motions. To efficiently minimize the resulting energy function for optical flow computation, we utilize an incomplete Cholesky preconditioned conjugate gradient algorithm to solve the large linear system. Experimental results on some synthetic and real image sequences show that the proposed algorithm compares favorably to most existing techniques reported in literature in terms of accuracy in optical flow computation with 100% density.     

Determination of Horizontal Motion through Optical Flow Computations

For intelligent/autonomous subsea vehicles,reliable short-range horizontal positioning is difficult to achieve,particularly over flat bottom topography.A potential solution proposed in this paper utilized a passive optical sensing method to estimate the vehicle displacement using the bottom surface texture.The suggested optical flow method does not require any feature correspondences in images and it is robust in allowing brightness changes between image frames.Fundamentally,this method is similar to correlation methods attempting to match images and compute the motion disparity.However,in correlation methods,searching a neighbor region blindly for best match is lengthy.Main contributions of this paper come from the analysis showing that optical flow computation based on the general model cannot avoid errors except for null motion although the sign of optical flow keeps correct,and from the development of an iterative shifting method based on the error characteristics to accurately determine motions.Advantages of the proposed method are verified by real image experiments.     

Combining deblurring and denoising for handheld HDR imaging in low light conditions

This paper proposes a probability formulation that unifies both single-image deblurring and multi-image denoising using variational inference. The proposed formulation is based on a theoretical analysis that compares denoising and deblurring in the same probabilistic framework, and supported by a practical approach that deal with general motion that creates HDR images in the presence of spatially varying motion. Based on this formulation, a new algorithm for deblurring a noisy and blurry image pair is presented. Besides, we provide also an approach that combines existing optical flow and image denoising techniques for High Dynamic Range imaging.     

Abnormal event detection and localization in crowded scenes based on PCANet

Detecting and localizing abnormal events in crowded scenes still remains a challenging task among computer vision community. An unsupervised framework is proposed in this paper to address the problem. Low-level features and optical flows (OF) of video sequences are extracted to represent motion information in the temporal domain. Moreover, abnormal events usually occur in local regions and are closely linked to their surrounding areas in the spatial domain. To extract high-level information from local regions and model the relationship in spatial domain, the first step is to calculate optical flow maps and divide them into a set of non-overlapping sub-maps. Next, corresponding PCANet models are trained using the sub-maps at same spatial location in the optical flow maps. Based on the block-wise histograms extracted by PCANet models, a set of one-class classifiers are trained to predict the anomaly scores of test frames. The framework is completely unsupervised because it utilizes only normal videos. Experiments were carried out on UCSD Ped2 and UMN datasets, and the results show competitive performance of this framework when compared with other state-of-the-art methods.     

Compression of Probabilistic Volumetric Models using multi-resolution scene flow

This paper presents a novel method to estimate dense scene flow using volumetric and probabilistic 3-d models. The method first reconstructs 3-d models at each time step using images synchronously captured from multiple views. Then, the 3-d motion between two consecutive 3-d models is estimated using a formulation that is the analog of Horn and Schunck's optical flow method. This particular choice of 3-d model representation allows estimating highly dense scene flow results, tracking of surfaces undergoing topological change and reliably recovering large motion displacements. The benefits of the method and the accuracy of 3-d flow results are demonstrated on recent multi-view datasets. The second goal of this work is to compress and reconstruct 3-d scenes at various time points using the estimated flow. A new method of scene warping is proposed that involves partitioning the optical flow field in regions of coherent motion which are subsequently parametrized by affine transformations. The compression objective of this work is achieved by the low storage requirements of the affine parameters that describe the optical flow field and the efficient reconstruction method through warping.     

Three-dimensional scene flow

Just as optical flow is the two-dimensional motion of points in an image, scene flow is the three-dimensional motion of points in the world. The fundamental difficulty with optical flow is that only the normal flow can be computed directly from the image measurements, without some form of smoothing or regularization. In this paper, we begin by showing that the same fundamental limitation applies to scene flow; however, many cameras are used to image the scene. There are then two choices when computing scene flow: 1) perform the regularization in the images or 2) perform the regularization on the surface of the object in the scene. In this paper, we choose to compute scene flow using regularization in the images. We describe three algorithms, the first two for computing scene flow from optical flows and the third for constraining scene structure from the inconsistencies in multiple optical flows.     

Adjusting shape parameters using model-based optical flow residuals

We present a method for estimating the shape of a deformable model using the least-squares residuals from a model-based optical flow computation. This method is built on top of an estimation framework using optical flow and image features, where optical flow affects only the motion parameters of the model. Using the results of this computation, our new method adjusts all of the parameters so that the residuals from the flow computation are minimized. We present face tracking experiments that demonstrate that this method obtains a better estimate of shape compared to related frameworks     

A Stochastic Approach for Blurred Image Restoration and Optical Flow Computation on Field Image Sequence

The blur in target images caused by camera vibration due to robot motion or hand shaking and by object(s) moving in the background scene is different to deal with in the computer vision system.In this paper,the authors study the relation model between motion and blur in the case of object motion existing in video image sequence,and work on a practical computation algorithm for both motion analysis and blut image restoration.Combining the general optical flow and stochastic process,the paper presents and approach by which the motion velocity can be calculated from blurred images.On the other hand,the blurred image can also be restored using the obtained motion information.For solving a problem with small motion limitation on the general optical flow computation,a multiresolution optical flow algoritm based on MAP estimation is proposed. For restoring the blurred image ,an iteration algorithm and the obtained motion velocity are used.The experiment shows that the proposed approach for both motion velocity computation and blurred image restoration works well.     

一种基于线性亮度变化模型的鲁棒的光流算法

提出了一种鲁棒光流算法,用于计算光照强度、帧间运动速度及运动速度变化较大情况下的光流场。在梯度约束方程中嵌入了线性亮度变化模型,以提高大的光照强度变化下算法稳健性;将各向异性扩散方程引入空间方向平滑约束,以改善运动不连续处的流速计算精度,并依此建立了多尺度空间微分光流算法。参数的均衡化得到了线性尺度变化下的恒定能量函数。迭代运算引入运动补偿的概念,使亮度误差减小。实验结果表明,在光照强度和运动速度及速度变化较大时,本文算法具有很好的计算精度,并产生密度100%的光流场。     

图象光流场计算技术研究进展

时变图象光流场计算技术是计算机视觉中的重要研究内容，也是当今研究的热点问题。为了使人们对该技术有一个较全面的了解，因而对时变图象光流场计算技术的研究和进展做了较系统的论述，首先分别列举了灰度时变图象和彩色时变图象的光流场计算方法，并对这些方法进行了分类，然后总结了出目前图象光流场计算中存在的几个问题，最后对光流场计算技术的研究发展及其应用前景指出了一些可能的方向。     

一种基于PSO的改进光流算法

针对变分光流法无法有效检测由间断、遮挡等因素造成的错误光流分量的缺陷,提出一种基于PSO(Particle Swarm Optimization)的光流算法。该方法在Classic+NL算法模型的基础上计算出光流后,引入前向光流和后向光流的运动一致性理论来判断遮挡区域,并通过基于PSO的修补法来实现对遮挡区域错误光流的有效修补,同时,利用邻域光流修补法实现了再次修补。实验结果表明,该方法能有效克服由间断、遮挡等因素造成的错误光流分量的缺陷,更准确地刻画出光流,提高光流的计算精度。     

Piecewise-Smooth Dense Optical Flow via Level Sets

We propose a new algorithm for dense optical flow computation. Dense optical flow schemes are challenged by the presence of motion discontinuities. In state of the art optical flow methods, over-smoothing of flow discontinuities accounts for most of the error. A breakthrough in the performance of optical flow computation has recently been achieved by Brox et~al. Our algorithm embeds their functional within a two phase active contour segmentation framework. Piecewise-smooth flow fields are accommodated and flow boundaries are crisp. Experimental results show the superiority of our algorithm with respect to alternative techniques. We also study a special case of optical flow computation, in which the camera is static. In this case we utilize a known background image to separate the moving elements in the sequence from the static elements. Tests with challenging real world sequences demonstrate the performance gains made possible by incorporating the static camera assumption in our algorithm.     

Dust particle detection in traffic surveillance video using motion singularity analysis

Dust particle detection in video aims to automatically determine whether the video is degraded by dust particle or not. Dust particles are usually stuck on the camera lends and typically temporally static in the images of a video sequence captured from a dynamic scene. The moving objects in the scene can be occluded by the dusts; consequently, the motion information of moving objects tends to yield singularity. Motivated by this, a dust detection approach is proposed in this paper by exploiting motion singularity analysis in the video. First, the optical model of dust particle is theoretically studied in by simulating optical density of artifacts produced by dust particles. Then, the optical flow is exploited to perform motion singularity analysis for blind dust detection in the video without the need for ground truth dust-free video. More specifically, a singularity model of optical flow is proposed in this paper using the direction of the motion flow field, instead of the amplitude of the motion flow field. The proposed motion singularity model is further incorporated into a temporal voting mechanism to develop an automatic dust particle detection in the video. Experiments are conducted using both artificially-simulated dust-degraded video and real-world dust-degraded video to demonstrate that the proposed approach outperforms conventional approaches to achieve more accurate dust detection.     

利用平稳光流估计的海上视频去抖

目的 海上拍摄的视频存在大面积的无纹理区域,传统基于特征点检测和跟踪的视频去抖方法处理这类视频时往往效果较差。为此提出一种基于平稳光流估计的海上视频去抖算法。方法 该算法以层次化块匹配作为基础,引入平滑性约束计算基于层次块的光流,能够快速计算海上视频的近似光流场;然后利用基于平稳光流的能量函数优化,实现海上视频的高效去抖动。结果 分别进行了光流估计运行时间对比、视频稳定运行时间对比和用户体验比较共3组实验。相比于能处理海上视频去抖的SteadyFlow算法,本文的光流估计算法较SteadFlow算法的运动估计方法快10倍左右,整个视频去抖算法在处理速度上能提升70%以上。本文算法能够有效地实现海上视频去抖,获得稳定的输出视频。结论 提出了一种基于平稳光流估计的海上视频去抖算法,相对于传统方法,本文方法更适合处理海上视频的去抖。