Mining of Moving Objects from Time-Series Images and its Application to Satellite Weather Imagery

The framework of mining of moving objects from image data sequence is presented. Scenes are first clustered and labeled by using two-stage SOM that is modified to recognize images including similar moving objects as the same cluster, and that well recognizes scenes including prominent objects. After extraction of images which include prominent objects based on clustering result, the position and the shape of objects are approximated by using mixture gaussian model via EM algorithm, providing the adequate or larger number of components. By adopting the average of the data points in the smaller blocks as the initial parameters, the solutions are stabilized and the identification of components among time-series images and the tracking of a specific object become easier.This framework is applied to a four-year (ranging from 1997 to 2000) dataset of cloud images taken by Japanese weather satellite GMS-5 to evaluate its performance. Modified SOM method well classifies scenes which include prominent moving object, and seasonal variation tendency is detected in the cluster ID sequence. The result of object detection via EM algorithm for summer-type images including clear cloud masses such as typhoons shows that this approach well approximate the adequate distribution of cloud masses in many cases. Objects in the very irregular shapes are also well represented as the mixtures of gaussians.The extracted object information, together with the scene clustering result, is expected to offer us a rich source for knowledge discovery of video datasets. This approach is one of the effective ways of mining video images whose characteristics are unknown in advance, and thus applicable to the various type of applications.     

Temporal probabilistic object bases

There are numerous applications where we have to deal with temporal uncertainty associated with objects. The ability to automatically store and manipulate time, probabilities, and objects is important. We propose a data model and algebra for temporal probabilistic object bases (TPOBs), which allows us to specify the probability with which an event occurs at a given time point. In explicit TPOB-instances, the sets of time points along with their probability intervals are explicitly enumerated. In implicit TPOB-instances, sets of time points are expressed by constraints and their probability intervals by probability distribution functions. Thus, implicit object base instances are succinct representations of explicit ones; they allow for an efficient implementation of algebraic operations, while their explicit counterparts make defining algebraic operations easy. We extend the relational algebra to both explicit and implicit instances and prove that the operations on implicit instances correctly implement their counterpart on explicit instances.     

融合时域信息的修正分水岭视频分割新方法

提出一种将时域信息融入分水岭的视频分割新方法,以帧间变化检测为基础,通过运动边缘信息得到对象的初始模型,利用时域信息得到前景和背景的标识,结合提出的彩色多尺度形态学梯度算子进行分水岭分割,得到具有精确边界的视频对象,对慢变和快变的目标均有良好的效果,能够检测新出现的运动对象和现有对象的消失,能够定位和跟踪运动目标.继承了变化检测和分水岭算法速度快的优点,克服了两者易受噪声影响的缺点.     

A dynamic conditional random field model for foreground and shadow segmentation

This paper proposes a dynamic conditional random field (DCRF) model for foreground object and moving shadow segmentation in indoor video scenes. Given an image sequence, temporal dependencies of consecutive segmentation fields and spatial dependencies within each segmentation field are unified by a dynamic probabilistic framework based on the conditional random field (CRF). An efficient approximate filtering algorithm is derived for the DCRF model to recursively estimate the segmentation field from the history of observed images. The foreground and shadow segmentation method integrates both intensity and gradient features. Moreover, models of background, shadow, and gradient information are updated adaptively for nonstationary background processes. Experimental results show that the proposed approach can accurately detect moving objects and their cast shadows even in monocular grayscale video sequences.     

Composition and Retrieval of Visual Information for Video Databases

This paper presents a new visual aggregation model for representing visual information about moving objects in video data. Based on available automatic scene segmentation and object tracking algorithms, the proposed model provides eight operations to calculate object motions at various levels of semantic granularity. It represents trajectory, color and dimensions of a single moving object and the directional and topological relations among multiple objects over a time interval. Each representation of a motion can be normalized to improve computational cost and storage utilization. To facilitate query processing, there are two optimal approximate matching algorithms designed to match time-series visual features of moving objects. Experimental results indicate that the proposed algorithms outperform the conventional subsequence matching methods substantially in the similarity between the two trajectories. Finally, the visual aggregation model is integrated into a relational database system and a prototype content-based video retrieval system has been implemented as well.     

Temporal relational data model

This paper incorporates a temporal dimension to nested relations. It combines research in temporal databases and nested relations for managing the temporal data in nontraditional database applications. A temporal data value is represented as a temporal atom; a temporal atom consists of two parts: a temporal set and a value. The temporal atom asserts that the value is valid over the time duration represented by its temporal set. The data model allows relations with arbitrary levels of nesting and can represent the histories of objects and their relationships. Temporal relational algebra and calculus languages are formulated and their equivalence is proved. Temporal relational algebra includes operations to manipulate temporal data and to restructure nested temporal relations. Additionally, we define operations to generate a power set of a relation, a set membership test, and a set inclusion test, which are all derived from the other operations of temporal relational algebra. To obtain a concise representation of temporal data (temporal reduction), collapsed versions of the set-theoretic operations are defined. Procedures to express collapsed operations by the regular operations of temporal relational algebra are included. The paper also develops procedures to completely flatten a nested temporal relation into an equivalent 1 NF relation and back to its original form, thus providing a basis for the semantics of the collapsed operations by the traditional operations on 1 NF relations     

Collaborative Multimedia Presentations in Mobile Environments

Distributed multimedia documents systems, distributed video servers are examples of multimedia presentations involving collaboration among multiple information sources. In such applications, objects have to be retrieved from their sources and presented to users according to specified temporal relationships. Objects retrieval in these collaborative applications is influenced by their presentation times, durations, and network throughput available to their sources. Replication of objects amongst the set of collaborating systems gives a choice for object retrieval. Client going through a multimedia presentation can be in a mobile environment. Here, object retrievals from collaborating servers are carried out by base stations to which the client is attached. Mobile client then downloads objects from the base station.In this paper, we present a graph-search based algorithm for computing and negotiating throughput requirements of collaborating multimedia presentations with replicated objects in a mobile environment. This algorithm maximizes the number of cached objects (that have already been played) for handling operations such as reverse presentation.     

基于运动显著性的移动镜头下的运动目标检测

针对移动镜头下的运动目标检测中的背景建模复杂、计算量大等问题,提出一种基于运动显著性的移动镜头下的运动目标检测方法,在避免复杂的背景建模的同时实现准确的运动目标检测。该方法通过模拟人类视觉系统的注意机制,分析相机平动时场景中背景和前景的运动特点,计算视频场景的显著性,实现动态场景中运动目标检测。首先,采用光流法提取目标的运动特征,用二维高斯卷积方法抑制背景的运动纹理;然后采用直方图统计衡量运动特征的全局显著性,根据得到的运动显著图提取前景与背景的颜色信息;最后,结合贝叶斯方法对运动显著图进行处理,得到显著运动目标。通用数据库视频上的实验结果表明,所提方法能够在抑制背景运动噪声的同时,突出并准确地检测出场景中的运动目标。     

复杂条件下运动物体的视频检测

提出了一种适合于复杂条件下运动物体视频检测的新方法。首先采用改进的点状相关图法检测出可能存在运动物体的区域,然后对可能存在的运动区域二值化并采用形态学算子去除环境噪音和光线的干扰,精确地确定运动物体的轮廓。大量实验验证了该方法的有效性和实用性。     

自动分割及跟踪视频运动对象的一种实现方法

随着MPEG-4压缩标准的制定,分割及跟踪视频运动对象的研究显得极其重要。在MPEG-4视频编码标准中,为了实现基于视频内容的交互功能,其视频序列的每一帧由视频对象面（VOP）来表示。为了生成视频对象面,需要对视频序列中的运动对象进行有效的分割;并跟踪运动对象随时间的变化,为此提出并实现了一种用于分割及跟踪视频运动对象的时空联合方法。该方法首先采用连续帧间差的4次统计量假设检验,确定运动对象的位置,自动地分离出运动区域与背景区域;在运动区域内,采用数学形态学的分水线算法来精确地提取运动对象的轮廓;最后,将提取到的运动对象作为模板,对后续的视频序列,用Hausdorff距离度量,来跟踪并提取后续帧中运动对象。实验结果表明,该方法能有效地分割和跟踪视频运动对象,且能有效减少计算复杂度,其调整参数也较少。     

Spatio-temporal composition of video objects: representation andquerying in video database systems

A key characteristic of video data is the associated spatial and temporal semantics. It is important that a video model models the characteristics of objects and their relationships in time and space. J.F. Allen's (1983) 13 temporal relationships are often used in formulating queries that contain the temporal relationships among video frames. For the spatial relationships, most of the approaches are based on projecting objects on a two or three-dimensional coordinate system. However, very few attempts have been made formally to represent the spatio-temporal relationships of objects contained in the video data and to formulate queries with spatio-temporal constraints. The purpose of the work is to design a model representation for the specification of the spatio-temporal relationships among objects in video sequences. The model describes the spatial relationships among objects for each frame in a given video scene and the temporal relationships (for this frame) of the temporal intervals measuring the duration of these spatial relationships. It also models the temporal composition of an object, which reflects the evolution of object's spatial relationships over the subsequent frames in the video scene and in the entire video sequence. Our model representation also provides an effective and expressive way for the complete and precise specification of distances among objects in digital video. This model is a basis for the annotation of raw video     

运动目标三维轨迹可视化与关联分析方法

随着治安监控系统的普及,越来越多的监控摄像头被安装在各个交通道路和公共场所中,每天都产生大量的监控视频.如今,监控视频分析工作主要是采用人工观看的方式来排查异常,以这种方式来分析视频内容耗费大量的人力和时间.目前,关于视频分析方面的研究大多是针对目标个体的异常行为检测和追踪,缺乏针对对象之间的关联关系的分析,对视频中的一些对象和场景之间的关联关系等还没有较为有效的表示和分析方法.针对这一现状,提出一种基于运动目标三维轨迹的关联视频可视分析方法来辅助人工分析视频,首先对视频资料进行预处理,获取各个目标对象的运动轨迹信息,由于二维轨迹难以处理轨迹的自相交、循环运动和停留等现象,并且没有时间信息就难以对同一空间内多个对象轨迹进行的关联性分析,于是结合时间维度对轨迹进行三维化扩展.该方法支持草图交互方式来操作,在分析过程中进行添加草图注释来辅助分析.可结合场景和对象的时空关系对轨迹进行关联性计算,得出对象及场景之间的关联模型,通过对对象在各个场景出现状况的统计,结合人工预先设定的规则,可实现对异常行为报警,辅助用户决策.     

构建面向对象时态数据模型的研究

本文通过对现存关系数据库和面向对象数据库的比较分析，总结其优点与不足，并在结合了二者优点的UniSQL/X模型基地上，对该模型进行时态上的扩展，使它能够记录和处理时态信息。在查询代数中，引入了一 些特殊的时态操作，通过聚集、概括和时间参考三种关联对对象进行访问，而且此代数同时反映了时态关系代数和对象代数的特点。     

海量监控视频快速回放与检索技术

为解决海量监控视频的快速浏览和检索,介绍了一种基于目标索引的视频摘要和检索方法。该方法在光流分析的基础上,在画面的静止区域更新背景,运动的区域利用差分法分割出运动目标图像。经过优化的快速特征匹配和建立运动跟踪模型后,根据目标运动轨迹,按照时空距离进行聚类。在目标图像数据和运动参数进行XML结构化存储为索引的基础上,最后在检索时将符合条件的所有目标图像,按照其原有时间顺序逐帧贴到同一个背景图像中,形成动态的摘要视频。由于该方法剔除了背景中大量的时空冗余信息,可在较短回放时间内浏览全部有用目标,显著提高海量监控视频的查阅效率。     

一种最大后验概率条件下的运动目标检测方法

提出一种最大后验概率条件下的运动目标检测方法.首先根据条件随机场模型和马尔可夫随机场模型建立了一个最大后验概率框架.在该框架内融入了连续标记场的时域信息、颜色信息和每个标记场的空域信息.考虑到传统方法融入的特征信息不够,提取目标的准确度不高,在目标模型中充分融入了颜色信息和边缘特征,以便获得更好的检测效果.实验结果表明提出的方法能正确检测到运动目标.     

Augmented reality with automatic illumination control incorporatingellipsoidal models

In applications of augmented reality like virtual studio TV production, multisite video conference applications using a virtual meeting room and synthetic/natural hybrid coding according to the new ISO/MPEG-4 standard, a synthetic scene is mixed into a natural scene to generate a synthetic/natural hybrid image sequence. For realism, the illumination in both scenes should be identical. In this paper, the illumination of the natural scene is estimated automatically and applied to the synthetic scene. The natural scenes are restricted to scenes with nonoccluding, simple, moving, mainly rigid objects. For illumination estimation, these natural objects are automatically segmented in the natural image sequence and three-dimensionally (3-D) modeled using ellipsoid-like models. The 3-D shape, 3-D motion, and the displaced frame difference between two succeeding images are evaluated to estimate three illumination parameters. The parameters describe a distant point light source and ambient light. Using the estimated illumination parameters, the synthetic scene is rendered and mixed to the natural image sequence. Experimental results with a moving virtual object mixed into real video telephone sequences show that the virtual object appears naturally having the same shading and shadows as the real objects. Further, shading and shadow allows the viewer to understand the motion trajectory of the objects much better     

基于OpenCV的视频目标检索

通过分析视频处理流程,设计并实现了一套基于OpenCV的视频目标检索方案。首先使用经过改良的背景差分法获取视频中的运动前景,接着根据前景的轮廓得到前景的位置,然后根据前景的颜色直方图特征对前景物体进行分类跟踪,最后将分好类别的目标进行排序,以便根据要求得到最迫切需要的目标及其运动过程。实验结果表明,该方法科学合理,思路清晰,功能完备,对视频目标检索的深入研究具有很高的研究价值。     

结合时空信息的视频对象平面自动提取算法

提出一种视频对象平面自动提取算法,首先以时域帧间运动信息为依据,利用高斯检验方法得到初始的二值运动模板,并建立双尺度邻域的MRF模型进一步检验,以获取平滑、完整的运动模板;然后提出了结合非线性变换的改进分水岭算法对运动区域进行帧内空域分割;最后对时域和空域分割结果进行比重运算,提取最终运动对象.实验结果说明了该算法的有效性.     

运动目标的自动空时分割算法

在视频应用中,运动目标的提取是一个重要的研究课题。为了对运动目标进行更有效的分割,提出了一种从视频序列中自动提取运动目标的空时分割算法。该算法在时域分割中采用基于齐异矢量消除的目标检测方法来获得运动目标的初始模板。通常,该初始模板具有不连续的边界和一些"孔"。为了得到较为完整的目标区域,用具有距离约束的区域生长算法来补偿初始模板。而在空域分割中,分水岭分割则通过考虑全局信息来增强其分割的精确性。然后,精确的运动目标即可通过空时融合模块提取出来。试验结果表明,该空时分割算法是有效的。     

Multilayer background modeling under occlusions

A multilayer background modeling technique is presented for video surveillance. Rather than simply classifying all features in a scene as either dynamically moving foreground or long-lasting, stationary background, a temporal model is used to place each scene object in time relative to each other. Foreground objects that become stationary are registered as layers on top of the background layer. In this process of layer formation, the algorithm deals with ”fake objects” created by moved background, and noise created by dynamic background and moving foreground objects. Objects that leave the scene are removed based on the occlusion reasoning among layers. The technique allows us to understand and visualize a scene with multiple objects entering, leaving, and occluding each other at different points in time. This scene understanding leads to a richer representation of temporal scene events than traditional foreground/background segmentation. The technique builds on a low-cost background modeling technique that makes it suitable for embedded, real-time platforms.