Image sequence stabilisation based on Kalman filtering of framepositions

An image sequence stabilisation system based on Kalman filtering of absolute frame displacements is reported. The stabiliser compensates for undesired fluctuations in the sequence by shifting image frames into smoothed positions obtained through the Kalman filter. The stabiliser operates in real-time, removing jitter while preserving desired gross camera displacements     

Variational method for super-resolution optical flow

The motion fields in an image sequence observed by a car-mounted imaging system depend on the positions in the imaging plane. Since the motion displacements in the regions close to the camera centre are small, for accurate optical flow computation in this region, we are required to use super-resolution of optical flow fields. We develop an algorithm for super-resolution optical flow computation. Super-resolution of images is a technique for recovering a high-resolution image from a low-resolution image and/or image sequence. Optical flow is the appearance motion of points on the image. Therefore, super-resolution optical flow computation yields the appearance motion of each point on the high-resolution image from a sequence of low-resolution images. We combine variational super-resolution and variational optical flow computation in super-resolution optical flow computation. Our method directly computes the gradient and spatial difference of high-resolution images from those of low-resolution images, without computing any high-resolution images used as intermediate data for the computation of optical flow vectors of the high-resolution image.     

Automated rig removal with Bayesian motion interpolation

Some of the most convincing film and video effects are created in digital post-production by removing apparatus that supports or manipulates actors and objects. Wires, cranes and other objects can be removed by digitally painting them out of the scene provided that some 'clean plate' image is available for pasting in the missing regions. The problem of when no such plate is available is addressed. Provided that the undesired object (the rig) is moving, it is possible to use the motion throughout the sequence to reconstruct automatically the image material that was obscured. The paper takes a novel approach that allows the estimation of the motion of the material beneath the rig and then reconstruction of the missing image material. A Bayesian framework is used to solve the motion reconstruction problem, and a unique tool is developed for automated rig removal. This tool holds great potential for speeding up one of the major tasks performed in the effects industry and is currently being tested in that environment.     

Spatially invariant image sequences

The authors define linearly additive spatially invariant image sequences and present an explicit mathematical model for describing them. In such a sequence, all objects are positionally invariant in each image of the sequence but have varying gray-scale contributions to the successive images of the sequence. The various components (features or processes) of the scene or object contribute additively to each image of the sequence, but each component has a characteristic variation (signature) from image to image due to the variation of the function, parameter or spectral band over the sequence. Objects with different spectral characteristics will have different image sequence signatures which can be used to distinguish them. Also presented are the general formulation, derivation, and explicit expression for the linear filter, called the simultaneous diagonalization (SD) filter, that calculates a single new image from the sequence such that a desired process is emphasized and any number of undesired processes is suppressed in the filtered image.     

基于光流法的多分辨率电子稳像算法

针对动载体摄像系统中视频序列受载体姿态运动及抖动的干扰而出现的不稳定现象,提出一种基于光流算法的多分辨率电子稳像算法。首先,通过划定有效的运动估算区域取代对整帧图像的计算以降低计算量;然后,利用基于光流算法的多分辨率分层运动估计快速并精确地计算出包含平移、旋转以及缩放运动的相邻帧间仿射变换参数;最后,采用固定帧补偿算法,利用求得的仿射变换参数,对图像进行运动补偿,消除或减轻图像序列帧间的随机抖动,达到稳像的目的。实验结果表明,针对包含运动目标的动态场景,该算法可以精确地检测出视频序列帧间平移、旋转以及缩放等复杂的抖动,水平和垂直方向的稳像精确度小于1pixel,保证视频序列的稳定输出,可应用于目标跟踪系统中。     

Concurrent 3-D motion segmentation and 3-D interpretation of temporal sequences of monocular images.

The purpose of this study is to investigate a variational method for joint multiregion three-dimensional (3-D) motion segmentation and 3-D interpretation of temporal sequences of monocular images. Interpretation consists of dense recovery of 3-D structure and motion from the image sequence spatiotemporal variations due to short-range image motion. The method is direct insomuch as it does not require prior computation of image motion. It allows movement of both viewing system and multiple independently moving objects. The problem is formulated following a variational statement with a functional containing three terms. One term measures the conformity of the interpretation within each region of 3-D motion segmentation to the image sequence spatiotemporal variations. The second term is of regularization of depth. The assumption that environmental objects are rigid accounts automatically for the regularity of 3-D motion within each region of segmentation. The third and last term is for the regularity of segmentation boundaries. Minimization of the functional follows the corresponding Euler-Lagrange equations. This results in iterated concurrent computation of 3-D motion segmentation by curve evolution, depth by gradient descent, and 3-D motion by least squares within each region of segmentation. Curve evolution is implemented via level sets for topology independence and numerical stability. This algorithm and its implementation are verified on synthetic and real image sequences. Viewers presented with anaglyphs of stereoscopic images constructed from the algorithm's output reported a strong perception of depth.     

应用于校正扩展视场偏差的红外成像测量方法

介绍了一种可扩展视场红外光学系统的工作原理,针对系统初始化后由于楔形镜装配误差引起的视场偏差对成像的影响,设计了一种校正该光学系统视场偏差的红外成像测量方法。通过计算机视觉技术在目标运动轨迹测量中的应用,把该技术与红外成像系统进行结合;用红外成像系统对运动目标成像,将目标的运动轨迹视为序列离散点,利用计算机视觉技术对这些离散点进行轨迹检测,并采集这些目标连续运动过程中的离散点图像。后期通过对采集的目标运动轨迹原始图像进行像图像处理,运用编程仿真构造目标的实际运动轨迹,最终完成了可扩展视场红外光学系统视场偏差的校正。结果表明,基于计算机视觉技术的红外成像测量方法能够很好地实现运动目标的轨迹测量。     

Automatic extraction and measurement of leukocyte motion inmicrovessels using spatiotemporal image analysis

This paper describes a computer vision system for the automatic extraction and velocity measurement of moving leukocytes that adhere to microvessel walls from a sequence of images. The motion of these leukocytes can be visualized as motion along the wall contours. The authors use the constraint that the leukocytes move along the vessel wall contours to generate a spatiotemporal image, and the leukocyte motion is then extracted using the methods of spatiotemporal image analysis. The generated spatiotemporal image is processed by a special-purpose orientation-selective filter and a subsequent grouping process newly developed for this application. The orientation-selective filter is designed by considering the particular properties of the spatiotemporal image in this application in order to enhance only the traces of leukocytes. In the subsequent grouping process, leukocyte trace segments are selected and grouped among all the segments obtained by simple thresholding and skeletonizing operations. The authors show experimentally that the proposed method can stably extract leukocyte motion     

Optical Flow Estimation for a Periodic Image Sequence

We propose a temporal modeling approach for determining image motion from a sequence of images wherein the inherent motion is periodic over time. To exploit the periodic nature of the motion, we use a Fourier harmonic representation to model the temporal evolution of the motion field for the entire sequence. We then determine the motion field simultaneously for the different image frames by estimating the parameters of this representation model, where the model order in the Fourier representation serves as a regularization parameter on the temporal coherence of the motion field. This approach can take advantage of the statistics of all the available data in the image sequence. In our experiments, we tested the proposed approach on several motion types at different noise levels, including translational motion, convergent/divergent motion, and cardiac motion. Our results demonstrate that this approach could lead to more robust estimation of the motion field in the presence of strong imaging noise compared to a frame-by-frame estimation approach.     

Contour image sequence compression through motion analysis and hybrid coding method

This paper presents a motion analysis algorithm (MAA) and a hybrid coding method for contour image sequence compression. The contour image sequence consists of objects moving and rotating in a 3-D world with occlusion, shape, and size variations from frame to frame. The MAA separates the moving image sequence into several object-oriented subsequences (OOSs). In each OOS, the component is either stationary or moves smoothly, and the motion parameters can be easily estimated. The first and last frames of OOS are key frames, and the others are in-between frames. The key frames are unpredictable, and the entire frames need to be encoded. The in-between frames are compensable, and they are encoded by the motion parameter coding. The hybrid coder uses vectorgraph coding to remove spatial redundancy of the key frames and motion parameter coding to reduce the temporal redundancy of the OOSs. The motion parameters are encoded as combinations of 2-D translation, 2-D rotation, and scaling. There are many applications for contour image sequence compression. The cartoon image sequence (a sequence of line drawing sketches) and the high-frame-rate videophone for sign language transmission are good examples. Experiments show that our method encodes the contour image sequence at a very high compression ratio without losing intelligibility.     

Optimal filtering of radiographic image sequences using simultaneous diagonalization

A method is presented for the processing of temporal image sequences to enhance a desired process and suppress an undesired (interfering) process and random noise. Furthermore, the processed information is contained in a single frame which is easily interpreted. The method consists of collecting information about the desired and interfering processes from the frames of the given image sequence. The information is in the form of vectors that characterize the temporal properties of the processes. Matrices are formed by performing outer product expansions on these vectors and an eigenvector matrix is found which will simultaneously diagonalize these matrices. By calculating the inner product of a selected eigenvector from this matrix with the image sequence, an enhanced image of the desired process is obtained. A parameter can be adjusted which will increase the amount of suppression for either random noise or the interfering process. At one limit setting of this parameter, a matched filter for the desired process results, while at the other extreme, very high attenuation of the interfering process will occur. Simulations which demonstrate the effectiveness of this technique are presented along with results obtained by processing a radiographic temporal image sequence.     

分块投影匹配的运动目标检测方法

针对动态场景下图像序列中运动目标检测问题,首先采用基于分块投影匹配的全局运动参数估计方法,然后利用估计得到的运动参数补偿背景的全局运动以稳定图像序列,最后在稳定后的图像序列上采用背景减除法提取运动目标。实验表明,分块投影匹配的运动目标检测能有效地提取动态背景下的运动目标。     

Object-based estimation of dense motion fields

Motion estimation belongs to key techniques in image sequence processing. Segmentation of the motion fields such that, ideally, each independently moving object uniquely corresponds to one region, is one of the essential elements in object-based image processing. This paper is concerned with unsupervised simultaneous estimation of dense motion fields and their segmentations. It is based on a stochastic model relating image intensities to motion information. Based on the analysis of natural images, a region-based model of motion-compensated prediction error is proposed. In each region the error is modeled by a white stationary generalized Gaussian random process. The motion field and its segmentation are themselves modeled by a compound Gibbs/Markov random field accounting for statistical bindings in spatial direction and along the direction of motion trajectories. The a posteriori distribution of the motion field for a given image sequence is formulated as an objective function, such that its maximization results in the MAP estimate. A deterministic multiscale relaxation technique with regular structure is employed for optimization of the objective function. Simulation results are in a good agreement with human perception for both the motion fields and their segmentations.     

确定三维运动参数的张量分析方法

利用中心投影图象序列估计三维运动参数,需要知道空间运动物体的特征与图象平面上投影坐标之间的对应关系。为了避免利用特征对应关系,本文给出在仿射变换系统中的张量分析方法,也给出模拟实验结果数据。     

A TENSOR ANALYSIS METHOD FOR ESTIMATING 3-D MOTION PARAMETERS

By using the center projection image sequence to estimate 3-D motion parameters,one needs to know the corresponding relationship between the feature of motion object in spaceand the projection coordinate on image plane.In order to avoid using the relationship of featurecorrespondence,the tensor analysis method in the affine transformation system is presented,andthe simulation data of experimental results are given.     

基于视频序列的数字图像拼接技术研究进展

基于视频序列的数字图像拼接是指将具有重叠区的多帧视频通过数字配准和融合获得单幅宽视场静态全景图或动态全景图.基于视频序列的数字图像拼接技术主要包括全局快速配准算法、运动目标分割算法和无缝融合算法.首先分析理想数字图像拼接系统的特性,然后介绍近年来基于视频序列的数字图像拼接技术的研究进展,最后分析其研究动向.     

红外视频帧运动模糊复原技术研究

当前红外空域监测探测系统常用视频的形式存储和传输图像信号,但是视频图像在形成、传输和记录过程中,易受运动模糊和噪声的污染,为了使该系统适用于当前空域形势,提出基于该系统的视频运动模糊复原算法。首先构建基于视频流运动模糊复原模型,综合序列图像各帧之间的互信息,估计有效的点扩散函数,然后描述运动模糊复原流程,提出相应算法,构建各功能模块。操作中视频以降频采样的方式减少计算复杂度,提高图像质量,获取较高复原效果。最后,通过引入主、客观两套评价体系对使用的算法以及其他经典算法作对照,评估复原结果。实验结果表明:复原视频各帧的峰值信噪比达到37,均方误差在9以下,均优于对照算法。基本满足监测系统发现目标,监测空域的要求。     

基于2D-TDI及运动补偿的红外图像增强算法

针对红外图像的特点,提出了一种基于2D-TDI及运动补偿的新方法。将序列红外图像根据运动特性进行多分辨运动估计,然后根据得到的运动矢量进行2D-TDI。多分辨运动估计的引入,提高了运动估计对噪声的抗干扰能力,并有效地减小了运算量。该算法克服了传统图像直接叠加引入的图像边缘模糊的缺点,并能明显提高图像的信噪比。用真实的红外图像序列进行了处理,实验证明了算法的优越性,并取得了满意的效果。     

单目图像序列中基于正则化的局部三维非刚体运动估计

通过局部的三维非刚体运动估计进而达到全局估计结果是三维非刚体运动估计中的重要方法。本文提出了在单目图像序列中利用正则化的手段解决局部的三维非刚体运动估计。首先，在帧间特征点匹配已确立的前提下，利用仿射运动模型并结合中心投影方式提出了运动估计的最小二乘模型；然后针对三维运动估计的不适定性提出了正则化的运动估计方法，以正则化的形式融入运动的先验知识，使运动估计的结果更具鲁棒性；最后利用Levenberg-Marquart方法实现运动参数的求解。仿真图像序列的实验反映了本文方法的有效性。