Fast multiframe motion estimation algorithms by motion vector composition for the MPEG-4/AVC/H.264 standard

The MPEG-4/AVC/H.264 video coding standard adopts various coding schemes such as multiple reference frames and variable block sizes for motion estimation. Hence, MPEG-4/AVC/H.264 provides gains in compression efficiency of up to 50% over a wide range of bit rates and video resolutions compared to previous standards. However, these features result in a considerable increase in encoder complexity, mainly regarding to mode decision and motion estimation. The proposed algorithms use the stored motion vectors to compose the motion vector without performing the full search in each reference frame. Therefore, the proposed algorithms can obtain an average speed up ratio of four for encoding, thus benefiting from the prediction of the motion vector for the reference frames in advance and maintaining good performance. Any fast search algorithm can be utilized to further largely reduce the computational load.     

Real-time interactive MPEG-4 system encoder using a cluster ofworkstations

MPEG-4 currently being finalized by the Moving Pictures Experts Group of the ISO is a multimedia standard, MPEG-4 aims to support content-based coding of audio, text, image, and video (synthetic and natural) data, multiplexing of coded data, as well as composition and representation of audiovisual scenes. One of the most critical components of an MPEG-4 environment is the system encoder. An MPEG-4 scene may contain several audio and video objects, images, and text, each of which must be encoded individually and then multiplexed to form the system bitstream. Due to its flexible features, object-based nature, and provision for user interaction, MPEG-4 encoder is highly suitable for a software-based implementation. A full-scale software-based MPEG-J system encoder with real-time encoding speed is a nontrivial task and requires massive computation. We have built such an encoder using a cluster of workstations collectively working as a virtual parallel machine. Parallel processing of MPEG-4 encoder needs to be carried out carefully as objects may appear or disappear dynamically in a scene. In addition, objects may be synchronized with each other. User interactions may also prohibit a straightforward parallelization. We propose a modeling methodology that captures the spatio-temporal relationship between various objects and user interaction. We then propose a number of scheduling algorithms that periodically allocate MPEG-4 objects to multiple workstations ensuring load balancing and synchronization requirements among multiple objects. Each scheduling algorithm has its own performance and complexity characteristics. The experimental results, while showing real-time encoding rates, exhibit tradeoffs between load balancing, scheduling overhead cost, and global performance     

一种基于帧间和帧内宏块级的X264并行编码算法

结合H.264编码标准对X264编码器进行了分析与研究,目的在于提高编码速度,增强X264的实时性。在重点分析了宏块间数据依赖关系的情况下,针对帧间宏块级多线程并行编码的特点,本文提出了一种基于帧间和帧内宏块级的多线程并行编码算法。该算法在原有的帧间宏块级多线程并行编码的基础上,遵循宏块之间的空间相关性,为I帧内每行宏块创建单独的线程,实现了帧间和帧内宏块级并行编码,达到了多粒度并行的效果。实验结果表明,该算法在视频序列能够有效地编码和保持峰值信噪比变化不大的情况下,提高了编码的加速比,从而加强了视频编码的实时性。     

基于并行和预测的方向菱形运动估计算法

对MPEG-4视频编码中的块运动估计算法进行分析,得出算法中存在的并行性。将多核并行计算的思想应用于方向菱形模板的运动估计算法中,并引入一种预测机制来充分提高并行性,得到基于多核并行预测的方向菱形运动估计算法(PPDDME)。在PC机和Omap5910 两个平台上的验证表明,该算法相比串行算法在保证了压缩质量的前提下,有效提高了编码速度。     

基于MPEG-4 FGS的视频流量模型

目前MPEG-4精细颗粒度可伸缩性(Fine Graruldty Scalability，FGS)编码视频正成为视频流服务的一种主要的业务流，因此，针对MPEG-4FGS视频流量进行建模对于网络性能仿真和通信网络设计具有十分重要的意义。该文首先介绍了MPEG-4FGS编码原理，然后对MPEG-4FGS视频流量的统计特性进行了分析，在此基础上，提出了基于MPEG-4FGS的视频流量模型。实验结果表明，该模型能较好地拟合原视频帧序列大小，且能根据网络带宽的动态变化进行适应的码率分配。     

SuperENC: MPEG-2 video encoder chip

Thanks to increased market acceptance of applications such as digital versatile disks (DVDs), HDTV, and digital satellite broadcasting, the MPEG-2 (Moving Picture Experts Group-2) standard is becoming widely used. The MPEG-2 video standard, established in 1934, provides for a high-quality video compression format that, through high bit rates and frame rates, yields high-resolution video images. Emerging multimedia applications, such as digital versatile disk and high-definition television, demand higher quality video than ever before. In response, our MPEG-2 video encoder chip supports multiple profiles and levels     

MPEG-4在无线视频通信中的应用研究

通过对无线视频通信和MPEG-4的特点分析,提出适合无线视频通信的MPEG-4SP级编解码框架,对影响MPEG-4编码速度的两项关键技术：运动估计和DCT变换进行编码优化,以提高编码速度,对优化结果进行了验证,为MPEG-4编码器在移动通信设备上的实现建立一个研究的基础平台。     

A portable fault-tolerant parallel software MPEG-1 encoder

MPEG video compression is quite difficult to achieve in real time, and hardware solutions for this problem are expensive. We present a portable, fault-tolerant, parallel, MPEG-1 encoder implemented in software. We detail the implementation strategy for the encoder and give performance results on a network of workstations and a massively parallel processor. We also show that our encoder can efficiently adapt to fluctuating processing power typical in workstation networks.     

MPEG-4编码技术应用及FPGA实现

为满足视频通信、智能化监控等的需求，研究了MPEG-4编码技术的应用特点，设计了基于FPGA的MPEG-4编码器IP。通过分类比较的方法，研究分析了MPEG-4编码技术在不同类型中的应用方案，总结归纳了各类应用的特点，为进一步提高编码器的效率、满足低功耗的要求，在FPGA上设计并实现了MPEG-4编码器。经测试表明：该设计方案编码高效，符合实时视频通信的需求，可广泛应用在移动视频通信、远程无线监控等领域。     

Fine grain pipeline architecture for high performance phase-based optical flow computation

Accurate motion analysis of real life sequences is a very active research field due to its multiple potential applications. Currently, new technologies offer us very fast and accurate sensors that provide a huge quantity of data per second. Processing these data streams is very expensive (in terms of computing power) for general purpose processors and therefore, is beyond processing capabilities of most current embedded devices. In this work, we present a specific hardware architecture that implements a robust optical flow algorithm able to process input video sequences at a high frame rate and high resolution, up to 160 fps for VGA images. We describe a superpipelined datapath of more than 85 stages (some of them configured with superscalar units able to process several data in parallel). Therefore, we have designed an intensive parallel processing engine. System speed (frames per second) produces fine optical flow estimations (by constraining the actual motion ranges between consecutive frames) and the phase-based method confers the system robustness to image noise or illumination changes. In this work, we analyze the architecture of different frame rates and input image noise levels. We compare the results with other approaches in the state of the art and validate our implementation using several hardware platforms.     

MPEG-4编码技术应用及FPGA实现

为满足视频通信、智能化监控等的需求,研究了MPEG-4编码技术的应用特点,设计了基于FPGA的MPEG-4编码器IP。通过分类比较的方法,研究分析了MPEG-4编码技术在不同类型中的应用方案,总结归纳了各类应用的特点,为进一步提高编码器的效率、满足低功耗的要求,在FPGA上设计并实现了MPEG-4编码器。经测试表明:该设计方案编码高效,符合实时视频通信的需求,可广泛应用在移动视频通信、远程无线监控等领域。     

Performance Optimization of Video Coding Process on Multi-Core Platform Using Gop Level Parallelism

High definition video applications often require heavy computation, high bandwidth and high memory requirements which make their real-time implementation difficult. Multi-core architecture with parallelism provides new solutions to implementing complex multimedia applications in real-time. It is well-known that the speed of the H.264 encoder can be increased on a multi-core architecture using the parallelism concept. Most of the parallelization methods proposed earlier for these purposes suffer from the drawbacks of limited scalability and data dependency. In this paper, we present a result obtained using data-level parallelism at the Group-Of-Pictures (GOP) level for the video encoder. The proposed technique involves each GOP being encoded independently and implemented on JM 18.0 using advanced data structures and OpenMP programming techniques. The performance of the parallelized video encoder is evaluated for various resolutions based on the parameters such as encoding speed, bit rate, memory requirements and PSNR. The results show that with GOP level parallelism, very high speed up values can be achieved without much degradation in the video quality.     

嵌入式MPEG-4视频编码器的研究

论文介绍一种基于嵌入式ARM处理器的MPEG-4视频编码器方案。该方案采用嵌入式Linux操作系统,在FreescaleMXL硬件平台上实现MPEG-4simpleprofile视频编码子集。为提高编码效率,该方案从算法和结构两方面对编码器进行优化。首先利用处理器内嵌的多媒体加速器硬件实现DCT算法,并对运动估计、量化、半像素插值等关键部分进行汇编优化,然后提出一种基于运动矢量分布特征的半像素搜索算法,对半像素运动矢量搜索进行优化。对标准测试序列的试验表明该方案是可行的,对分辨率为QCIF的图像进行编码能够达到20fps以上的帧率。     

KAIST image computing system (KICS): A parallel architecture for real-time multimedia data processing

An efficient parallel architecture is proposed for high-performance multimedia data processing using multiple multimedia video processors (MVP; TMS320C80), which are fully programmable general digital signal processors (DSP). This paper describes several requirements for a multimedia data processing system and the system architecture of an image computing system called the KAIST Image Computing System (KICS). The performance of the KICS is evaluated in terms of its I/O bandwidth and the execution time for some image processing functions. An application of the KICS to the real-time Moving Picture Expert Group 2 (MPEG-2) encoder is introduced. The programmability and the high-speed data-access capability of the KICS are its most important features as a high-performance system for real-time multimedia data processing.     

MPEG-4中的运动图像分割算法

MPEG-4采纳了基于对象的编码技术,它要求对图像和视频作更多的分析,甚至是理解。基于对象的编码是MPEG-4的一个重要特点,但是对象的分割问题至今仍未得到满意的解决。为了应用MPEG-4视频标准,视频序列的每一帧应该根据视频对象平面来描述,编码前的首要工作是视频对象的识别,把每一帧分解为若干视频对象平面,每个视频对象平面代表不同的有语义的对象。该论文介绍了MPEG-4的标准、国际上图像分割的发展状况,在此基础上提出了利用空时域信息实现MPEG-4视频对象的自动分割的算法。     

低码率视频编码的优化方法

采用的低码率视频编码方法是以全局运动补偿和局部运动补偿为核心的混合编码方法.主要针对MPEG-4中原有算法计算量大、复杂度高的特点进行了编码系统优化,将这一编码方法实用化.该优化方法是针对占编码系统70%计算量的全局运动估计进行的,采用了基于运动特征的运动估计方法和鲁棒的目标函数,并在优化方法中引入了三级金字塔的分层计算.从对比实验结果来看,优化方法对不同运动类别视频的编码都是有效的,系统编码速度提高了3倍以上.目前,这项优化方法已经被MPEG-4视频编码优化小组采用.     

Real-Time MPEG-2 Video Codec System Using Multiple Digital Signal Processors

A real-time MPEG-2 video codec system, called the KAIST image computing system (KICS), was developed using multiple multimedia video processors (MVP, TMS320C80). The number of computations and optimum implementation of the MPEG-2 algorithm were analyzed for real-time processing. An MVP simulator was used to perform the timing analysis for the real-time MPEG-2 encoding and decoding, and it verified the design of the KICS. Task partitioning and mapping of the MPEG-2 algorithm to multiple MVPs were performed based on load-balancing, minimum data I/O, and the properties of the MVP. The KICS uses five MVPs for MPEG-2 real-time encoding of a main profile at the main level whereas it uses three MVPs for real-time decoding.     

基于同构多核处理器的H.264多粒度并行编码器

H.264码率低和视频质量高的优越性能以增加编码计算的复杂度为代价,如何开发适用于多核处理器平台的并行编码算法是提高其编码速度的重要研究内容,对于满足高清视频实时传输和大规模共享具有十分重要的意义.利用H.264开源编码器项目X264,在片级和数据级并行编码算法的基础上,通过分析图像帧之间的参考关系,提出并实现了B帧个数可变的帧级并行算法;根据宏块之间的参考关系,设计了一种类似流水线的宏块级并行方法;基于Intel同构多核平台,提出融合帧级、片级、宏块级和数据级4种不同粒度的并行编码方案,开发了H.264多粒度并行编码器.实验结果表明,在码率增加不大的情况下,H.264多粒度并行编码器可以很好地提升编码加速比,视频编码质量符合高质量的要求.     

Two-chip MPEG-2 video encoder

Our two-chip, real time, MPEG-2, simple-profile-at-main-level encoder supports NTSC 4:2:0 video signals with only three external memories. We have developed a compact encoder chip set. The key features of this chip set are a low encoding delay based on a simple profile at main level; wide-range motion estimation, which it performs using a hierarchical search; a flexible, macroblock level pipeline architecture based on RISC CPUs; and three small peripherals with no glue logic: a VRAM, synchronous DRAM, and FIFO DRAM     

Fast mode decision algorithm for H.264/SVC enhancement layer

Scalable video coding (SVC) has been standardized to extend the capabilities of the H.264 advanced video coding (AVC). The SVC can compress several video sequences of various resolutions as a single bit-stream. In the SVC enhancement layer, for Joint Scalable Video Model (JSVM) software implementation, an exhaustive mode decision process based on the base layer mode predictions is performed to obtain the best mode for each macroblock (MB). This technique may achieve a higher coding efficiency; however, it induces a significant computational complexity in the encoding engine. In order to speedup the SVC encoder, a fast mode decision algorithm was proposed in this paper. In other words, our aim was to decrease the number of candidate modes to reduce the computational complexity and maintain the same level of coding efficiency, this approach used the spatial and temporal correlation between MB situated at the enhancement layer and its co-located MB at the base layer. Our statistical analyses were made using several HD sequences with different motion characteristics. Experimental results show a significant improvement in terms of time encoding which is a major constraint for any real-time implementation. However, this gain is accompanied with an acceptable loss in video quality and a tolerable bit rate increase for most media supports. In fact, our proposed algorithm permits a major improvement that can reach up to 64.9 % in terms of computational effort. This gain will induce an average loss yield to 10.5 or 13.87 % that is comparable to the 13.12 % of the He Li’s algorithm with an acceptable loss in terms of subjective video quality.