Region-based rate control and bit allocation for wireless video transmission

In this paper, we propose a joint source-channel region-based rate control algorithm for real-time video transmissions over wireless systems. During the video transmission, the channel throughput available to the video encoder in the wireless systems is inherently variable, due to the retransmission of the error packets using the automatic repeat request (ARQ) error control. The variable data rate of the wireless system is characterized by the packet-level Gilbert two-state Markov Model, the parameters of which are extracted from the statistical properties of the channel information obtained from the wireless channel simulator. The proposed algorithm adopts a fast but effective block-based segmentation method to extract the regions of interest. Unlike traditional bit allocation methods used in the region/content-based rate control, the algorithm exploits the most effective criteria "coding qualities" as quantitative factors to directly control bit allocation among different regions so as to achieve better visual quality in the regions of interest. The computational complexity of the algorithm is low making it suitable for real-time applications. Compared with the MPEG-4 rate control algorithm, our algorithm can effectively enhance the perceptual quality for the regions of interest and significantly reduce the number of frame skipping; thereby, improve the smoothness of the video.     

基于H．264压缩域的实时运动对象分割算法

在压缩域内直接分割运动对象对于有实时要求的应用而言是十分必要的，H．264以其优越的压缩效率已经在许多应用中逐渐取代了MPEG-2／4，但有关在H．264压缩域内进行运动对象分割的研究还很少。为此提出了一种从H.264压缩域实时分割运动对象的算法，该算法首先对从H．264视频中提取出的原始运动矢量场进行时域和空域的归一化，接着通过对连续多帧的运动矢量场进行累积来增强显著的运动信息；然后对累积运动矢量场进行全局运动补偿，同时利用快速的统计区域生长算法按照运动相似性将其分割成多个区域；最后利用运动矢量场的方向角直方图来判断出属于运动对象的分割区域，以组成运动对象。通过对多个MPEG-4测试序列的实验结果表明，该方法不仅能够从H．264压缩域中实时地分割出运动对象，且具有良好的分割质量。     

一种新的基于运动矢量场及弹性模板的自适应快速搜索算法

该文提出了一种新的基于运动矢量场及弹性模板自适应快速搜索算法，它是以视频运动具有高度时空相关性为基础，从运动矢量场的均匀性出发，综合采用了快速块匹配、自适应弹性模板选择策略、自适应阈值与搜索中止准则等一系列技术，是一种具有伸缩性结构的自适应快速运动估计算法．实验结果表明，该算法以极小的搜索代价得到了与全搜索相当的效果，并在搜索速度和搜索效果方面明显优于MPEG-4最新推荐的快速运动估计算法，特别是在中低码率压缩时有着更优良的性能，非常适合于低码率实时视频压缩领域的应用．     

一种适于实时应用的快速有效的自适应运动估计算法

运动估计是视频编码系统的关键部分,同时也是整个视频编码器中计算量最大的部分,运动估计性能的优劣直接影响到整个视频编码器的运行效率和整个视频序列的重构质量。该文提出了一种适用于实时应用的快速自适应运动估计算法,该算法首先利用运动矢量时空域的相关性预测初始运动矢量,然后对不同运动程度的图像块进行不同大小的菱形搜索,同时采用了高效的提前截止准则。实验结果表明,对于各种不同运动程度的视频序列,该文算法都可以获得和全搜索法基本相同的图像质量,同时可以节省超过99%的计算量,该文算法在搜索速度和搜索精度两方面比快速算法中性能突出的菱形搜索具有更大的优势,更适合实时应用。     

基于EM聚类的H.264压缩域视频对象实时分割算法

从压缩域直接分割视频对象比传统的像素域分割具有快速高效的特点,目前已有不少从MPEG域分割的方法,但从H.264压缩域分割的甚少。为此提出一种基于H.264域的实时分割运动对象方法,该算法先对当前视频帧进行全局运动估计和补偿,然后对4×4的运动矢量场进行分类处理,最后对非零运动矢量使用改进的EM聚类分割算法。本文算法对多个视频序列进行了实验,结果表明,该算法针对静止背景和运动背景的视频序列都能达到较精确的实时分割。     

Real-Time Camera Calibration for Virtual Studio

In this paper, we present an overall algorithm for real-time camera parameter extraction, which is one of the key elements in implementing virtual studio, and we also present a new method for calculating the lens distortion parameter in real time. In a virtual studio, the motion of a virtual camera generating a graphic studio must follow the motion of the real camera in order to generate a realistic video product. This requires the calculation of camera parameters in real-time by analyzing the positions of feature points in the input video. Towards this goal, we first design a special calibration pattern utilizing the concept of cross-ratio, which makes it easy to extract and identify feature points, so that we can calculate the camera parameters from the visible portion of the pattern in real-time. It is important to consider the lens distortion when zoom lenses are used because it causes nonnegligible errors in the computation of the camera parameters. However, the Tsai algorithm, adopted for camera calibration, calculates the lens distortion through nonlinear optimization in triple parameter space, which is inappropriate for our real-time system. Thus, we propose a new linear method by calculating the lens distortion parameter independently, which can be computed fast enough for our real-time application. We implement the whole algorithm using a Pentium PC and Matrox Genesis boards with five processing nodes in order to obtain the processing rate of 30 frames per second, which is the minimum requirement for TV broadcasting. Experimental results show this system can be used practically for realizing a virtual studio.     

一种适用于实时应用的快速运动估计算法

提出了一种适用于实时应用的快速运动估计搜索算法，该算法按照一定的顺序依次搜索候选运动矢量，并且尽可能多地采用小菱形搜索模式，同时采用了高效的提前截止准则。实验结果表明，在保持图像质量基本不变的前提下，该文算法的搜索速度是MPEG-4标准中的快速运动估计算法的两倍左右，该文提出的算法在搜索速度和搜索效果两方面具有很大的优势，更适合实时应用。     

基层层次光流的半自动时空视频分割技术

在新一代MPEG-4视频编码标准中，为了支持面向对象编码和实现基于内容的应用，视频的半自动分割成为关键技术之一，为此提出了一种基于层次光流的半自动时空视频分割算法。该算法由空域分割和时域分割组成。在空域分割中，提出的基于点的图形用户界面（PBGUI），在用户的协助下，能够精确地定义需要分割的视频对象（VO）。时域分割根据空域分割的结果采用层次光流算法对视频对象进行边界和整体跟踪。实验结果表明，利用该算法，能够精确地分割出视频对象。     

基于层次光流的半自动时空视频分割技术

在新一代 MPEG- 4视频编码标准中 ,为了支持面向对象编码和实现基于内容的应用 ,视频的半自动分割成为关键技术之一 ,为此提出了一种基于层次光流的半自动时空视频分割算法 .该算法由空域分割和时域分割组成 .在空域分割中 ,提出的基于点的图形用户界面 (PBGU I) ,在用户的协助下 ,能够精确地定义需要分割的视频对象 (VO) .时域分割根据空域分割的结果采用层次光流算法对视频对象进行边界和整体跟踪 .实验结果表明 ,利用该算法 ,能够较精确地分割出视频对象 .     

Real-time interactive MPEG-4 system encoder using a cluster ofworkstations

MPEG-4 currently being finalized by the Moving Pictures Experts Group of the ISO is a multimedia standard, MPEG-4 aims to support content-based coding of audio, text, image, and video (synthetic and natural) data, multiplexing of coded data, as well as composition and representation of audiovisual scenes. One of the most critical components of an MPEG-4 environment is the system encoder. An MPEG-4 scene may contain several audio and video objects, images, and text, each of which must be encoded individually and then multiplexed to form the system bitstream. Due to its flexible features, object-based nature, and provision for user interaction, MPEG-4 encoder is highly suitable for a software-based implementation. A full-scale software-based MPEG-J system encoder with real-time encoding speed is a nontrivial task and requires massive computation. We have built such an encoder using a cluster of workstations collectively working as a virtual parallel machine. Parallel processing of MPEG-4 encoder needs to be carried out carefully as objects may appear or disappear dynamically in a scene. In addition, objects may be synchronized with each other. User interactions may also prohibit a straightforward parallelization. We propose a modeling methodology that captures the spatio-temporal relationship between various objects and user interaction. We then propose a number of scheduling algorithms that periodically allocate MPEG-4 objects to multiple workstations ensuring load balancing and synchronization requirements among multiple objects. Each scheduling algorithm has its own performance and complexity characteristics. The experimental results, while showing real-time encoding rates, exhibit tradeoffs between load balancing, scheduling overhead cost, and global performance     

自动分割视频运动目标的一种实现方法

随着基于运动对象特征编码的MPEG-4压缩标准的制定及智能监控系统的广泛应用,从视频序列中分割出运动对象的算法成为当今研究的热点。为此,文章提出并实现了一种基于互帧差的视频运动对象分割方法。它先从互帧差图像中提取出运动物体的基本轮廓,通过膨胀运算将尽可能是前景的区域分离出来。再将前景区域边缘的象素作为种子队列的元素,从种子队列出发向前景区域内部收缩,最后搜索到运动物体。这种算法方法简单,运算量小,而且只使用了头两帧的信息,因而适合于实时应用。实验结果表明,这种方法能有效地分割出主要的运动目标。     

基于对象的菱表搜索运动估计方法

ＭＰＥＧ－４是一个基于对象的视频标准,其运动估计方法不同于其他视频标准,本文提出一种基于对象的菱形搜索运动估计算法,利用图像分割信息和运动矢量的相关性,减少了得到运动矢量所需要的平均搜索点数,实验结果表明,在降低计算复杂度的同时,本算法保持了较好的图像质量。     

灰度投影结合Fisher准则的对象分割

视频对象分割是基于内容的视频编码和视频检索中的的对象跟踪方法,提出了一种可以从复杂场景中分割出MPEG-4的视频对象新方法.首先采用灰度投影匹配进行全局运动估计和补偿,用以消除背景变化的影响;然后由二次差分抽取中间帧解决遮挡问题,通过Fisher评价函数结合数学形态学填充得到运动对象分割掩膜,同时消除残余噪声以及平滑边缘.实验结果表明,该方法在一定范围内较好地解决了遮挡问题,并能够高效快速地得到比较精确的视频对象.     

基于内容的视频分割技术研究

随着MPEG-4基于内容功能的提出以及MPEG-7标准的不断推广应用,视频对象分割技术已成为视频处理领域中的研究热点。视频对象分割就是从视频序列中分割出在语义上有意义的对象。目前对视频分割研究已从基于镜头的分割发展到了基于内容的视频对象分割。然而,基于内容的视频对象分割技术还不成熟。文中讨论了视频对象分割技术的发展和研究状况,从组成视频运动对象的分割系统出发,介绍了时域分割以及时空域联合分割等技术,并提出了一种基于多帧差的视频对象分割算法。最后对分割技术中需要深入研究的问题进行了探讨。     

基于最小匹配误差方向预测的快速半像素运动估计

为了减小半像素搜索的计算量,本文提出了一个基于最小匹配误差方向预测的快速半像素运动估计算法.本文提出的算法利用亚像素搜索窗内的匹配误差单峰曲面的特性来预测半像素搜索区域中最小匹配误差方向,从而避免了大量不必要的匹配运算量.实验结果表明,对于各种不同运动程度和空间细节的视频序列,本文提出的算法在保证和半像素全搜索法相同图像质量的同时,平均节省73%的计算量,很适合实时应用.     

Human segmentation by geometrically fusing visible-light and thermal imageries

From depth sensors to thermal cameras, the increased availability of camera sensors beyond the visible spectrum has created many exciting applications. Most of these applications require combining information from these hyperspectral cameras with a regular RGB camera. Information fusion from multiple heterogeneous cameras can be a very complex problem. They can be fused at different levels from pixel to voxel or even semantic objects, with large variations in accuracy, communication, and computation costs. In this paper, we propose a system for robust segmentation of human figures in video sequences by fusing visible-light and thermal imageries. Our system focuses on the geometric transformation between visual blobs corresponding to human figures observed at both cameras. This approach provides the most reliable fusion at the expense of high computation and communication costs. To reduce the computational complexity of the geometric fusion, an efficient calibration procedure is first applied to rectify the two camera views without the complex procedure of estimating the intrinsic parameters of the cameras. To geometrically register different blobs at the pixel level, a blob-to-blob homography in the rectified domain is then computed in real-time by estimating the disparity for each blob-pair. Precise segmentation is finally achieved using a two-tier tracking algorithm and a unified background model. Our experimental results show that our proposed system provides significant improvements over existing schemes under various conditions.     

一种MPEG-2到AVS帧内转码的快速模式选择算法

根据到在变换域中块系数分布能够表达空间方向上的频率分布特性,提出了一种基于变换域系数分布特性与空间方向频率特性之间的对应关系的快速帧内预测编转码算法.实验结果表明该方法能极大的减少帧内转码的计算量,转码视频质量几乎不变.     

一种基于运动的视频层次化分析框架

运动信息是视频中最重要的特征之一.本文提出了一种新颖的基于运动的视频层次化分析框架.为了揭示视频流中的运动关系和在拍摄过程摄像机的运动信息即拍摄者的兴趣所在,提出了视频层次化结构来对视频流进行描述和检索.该运动层次化结构引入一个新的概念:基本运动单元(Basic Motion Unit,BMU).BMU反映的是MPEG-7中定义的基本摄像机操作,这样就可以把视频分割成BMU的集合.在BMU的基础上发现并挖掘出视频流中潜在的运动语义模式,最后根据运动活力(Motion Activity)将BMU聚类为运动节奏快慢的运动活力单元(Motion Activity Unit,MAU).该基于运动的视频分析框架不仅能够直观地捕获视频流在拍摄过程中拍摄者的注意力,而且为视频流的快速浏览和检索提供了新的思路.     

一种语义视频对象分割改进算法

随着MPEG－4、MPEG－7等标准的提出，如何提取语义视频对象已成为当前视频压缩和检索领域共同的研究课题。针对Watershed方法不能很好处理MPEG－1视频数据和人机交互困难的缺陷，该文提出了改进算法，它可以在MPEG－1视频数据上分割出光滑轮廓的语义视频对象。