Overview of the MPEG Reconfigurable Video Coding Framework

Video coding technology in the last 20 years has evolved producing a variety of different and complex algorithms and coding standards. So far the specification of such standards, and of the algorithms that build them, has been done case by case providing monolithic textual and reference software specifications in different forms and programming languages. However, very little attention has been given to provide a specification formalism that explicitly presents common components between standards, and the incremental modifications of such monolithic standards. The MPEG Reconfigurable Video Coding (RVC) framework is a new ISO standard currently under its final stage of standardization, aiming at providing video codec specifications at the level of library components instead of monolithic algorithms. The new concept is to be able to specify a decoder of an existing standard or a completely new configuration that may better satisfy application-specific constraints by selecting standard components from a library of standard coding algorithms. The possibility of dynamic configuration and reconfiguration of codecs also requires new methodologies and new tools for describing the new bitstream syntaxes and the parsers of such new codecs. The RVC framework is based on the usage of a new actor/ dataflow oriented language called CAL for the specification of the standard library and instantiation of the RVC decoder model. This language has been specifically designed for modeling complex signal processing systems. CAL dataflow models expose the intrinsic concurrency of the algorithms by employing the notions of actor programming and dataflow. The paper gives an overview of the concepts and technologies building the standard RVC framework and the non standard tools supporting the RVC model from the instantiation and simulation of the CAL model to software and/or hardware code synthesis.     

Quasi-Static Scheduling of CAL Actor Networks for Reconfigurable Video Coding

The upcoming Reconfigurable Video Coding (RVC) standard from MPEG (ISO / IEC SC29WG11) defines a library of coding tools to specify existing or new compressed video formats and decoders. The coding tool library has been written in a dataflow/actor-oriented language named CAL. Each coding tool (actor) can be represented with an extended finite state machine and the data communication between the tools are described as dataflow graphs. This paper proposes an approach to model the CAL actor network with Parameterized Synchronous Data Flow and to derive a quasi-static multiprocessor execution schedule for the system. In addition to proposing a scheduling approach for RVC, an extension to the well-known permutation flow shop scheduling problem that enables rapid run-time scheduling of RVC tasks, is introduced.     

Reconfigurable media coding: An overview

This paper provides an overview of the rationale of the Reconfigurable Media Coding framework developed by MPEG standardization committee to overcome the limits of traditional ways of providing decoder specifications. Such framework is an extension of the Reconfigurable Video coding framework now encompassing also 3D Graphics coding standard. The idea of this approach is to specify decoders using an actor dataflow based representation consisting of self-contained processing units (coding tools) connected altogether and communicating by explicitly exchanging data. Such representation provides a specification for which several properties of the algorithms interesting for codec implementations are explicitly exposed and can be used for exploring different implementation objectives.     

Optimized dynamic compilation of dataflow representations for multimedia applications

This paper proposes two optimization methods based on dataflow representations and dynamic compilation that enhance flexibility and performance of multimedia applications. These optimization methods are intended to be used in an adaptive decoding context, or, in other terms, where decoders have the ability to adapt their decoding processes according to a bitstream. This adaptation is made possible by coupling the decoding information to process a stream inside a coded stream. In this paper, we use dataflow representations from the upcoming MPEG Reconfigurable Media Coding (RMC) standard to supply the decoding information to adaptive decoders. The benefits claimed by MPEG RMC are a reuse of coding tools between different specifications of decoder and an execution scalability on different processing units with a single specification, which can target either hardware and/or software platforms. These benefits are not yet achievable in practice as these specifications are not used at the receiver side in MPEG RMC. We valid these benefits and propose two optimizations for the generation and the execution of dataflow models: the first optimization takes benefits of the reuse of coding tools to reduce the time to obtain—configure—enforceable decoders. The second provides an efficient, dynamic, and scalable execution according to the features of the execution platform. We show the practical impact of these two optimizations on two decoder representations compliant with the MPEG-4 part 2 Simple Profile standard and the MPEG-4 Advanced Video Coding standard. The results shows that configuration time can be reduced by 3 and the performance of decoders can be increased by 50 %.     

Methods to explore design space for MPEG RMC codec specifications

The recent MPEG Reconfigurable Media Coding (RMC) standard aims at defining media processing specifications (e.g. video codecs) in a form that abstracts from the implementation platform, but at the same time is an appropriate starting point for implementation on specific targets. To this end, the RMC framework has standardized both an asynchronous dataflow model of computation and an associated specification language. Either are providing the formalism and the theoretical foundation for multimedia specifications. Even though these specifications are abstract and platform-independent the new approach of developing implementations from such initial specifications presents obvious advantages over the approaches based on classical sequential specifications. The advantages appear particularly appealing when targeting the current and emerging homogeneous and heterogeneous manycore or multicore processing platforms. These highly parallel computing machines are gradually replacing single-core processors, particularly when the system design aims at reducing power dissipation or at increasing throughput. However, a straightforward mapping of an abstract dataflow specification onto a concurrent and heterogeneous platform does often not produce an efficient result. Before an abstract specification can be translated into an efficient implementation in software and hardware, the dataflow networks need to be partitioned and then mapped to individual processing elements. Moreover, system performance requirements need to be accounted for in the design optimization process. This paper discusses the state of the art of the combinatorial problems that need to be faced at this design space exploration step. Some recent developments and experimental results for image and video coding applications are illustrated. Both well-known and novel heuristics for problems such as mapping, scheduling and buffer minimization are investigated in the specific context of exploring the design space of dataflow program implementations.     

Low‐Complexity MPEG‐4 Shape Encoding towards Realtime Object‐Based Applications

Although frame‐based MPEG‐4 video services have been successfully deployed since 2000, MPEG‐4 video coding is now facing great competition in becoming a dominant player in the market. Object‐based coding is one of the key functionalities of MPEG‐4 video coding. Realtime object‐based video encoding is also important for multimedia broadcasting for the near future. Object‐based video services using MPEG‐4 have not yet made a successful debut due to several reasons. One of the critical problems is the coding complexity of object‐based video coding over frame‐based video coding. Since a video object is described with an arbitrary shape, the bitstream contains not only motion and texture data but also shape data. This has introduced additional complexity to the decoder side as well as to the encoder side. In this paper, we have analyzed the current MPEG‐4 video encoding tools and proposed efficient coding technologies that reduce the complexity of the encoder. Using the proposed coding schemes, we have obtained a 56 percent reduction in shape‐coding complexity over the MPEG‐4 video reference software (Microsoft version, 2000 edition).     

AVS解码器基于SystemC的实现

介绍了AVS视频编解码标准的关键技术和新一代硬件设计语言SystemC的特点以及利用SystemC进行软硬件协同设计的方法，并在此基础上介绍了AVS视频解码器基于SystemC的设计和实现。     

MPEG Reconfigurable Video Coding: From specification to a reconfigurable implementation

This paper demonstrates that it is possible to produce automatic, reconfigurable, and portable implementations of multimedia decoders onto platforms with the help of the MPEG Reconfigurable Video Coding (RVC) standard. MPEG RVC is a new formalism standardized by the MPEG consortium used to specify multimedia decoders. It produces visual representations of decoder reference software, with the help of graphs that connect several coding tools from MPEG standards. The approach developed in this paper draws on Dataflow Process Networks to produce a Minimal and Canonical Representation (MCR) of MPEG RVC specifications. The MCR makes it possible to form automatic and reconfigurable implementations of decoders which can match any actual platforms. The contribution is demonstrated on one case study where a generic decoder needs to process a multimedia content with the help of the RVC specification of the decoder required to process it. The overall approach is tested on two decoders from MPEG, namely MPEG-4 part 2 Simple Profile and MPEG-4 part 10 Constrained Baseline Profile. The results validate the following benefits on the MCR of decoders: compact representation, low overhead induced by its compilation, reconfiguration and multi-core abilities.     

MPEG—2视频解码的VHDL描述与验证

本文提出一种MPEG－2视频解码的硬件结构,并采用VHDL进行了描述。辚实现MPEG－2视频时的实时解码,本文针对时序控制、变长码解码、反量化、TDCT、运动补偿和输入输出控制等各部分都提出了相应的性能的电路结构。验证和仿真的结果表明：本文的设计可以完成相应的功能,能被用于实现MPEG－2MP@ML的实时解码芯片。     

Distributed Monoview and Multiview Video Coding

Growing percentage of the world population now uses image and video coding technologies on a regular basis. These technologies are behind the success and quick deployment of services and products such as digital pictures, digital television, DVDs, and Internet video communications. Today's digital video coding paradigm represented by the ITU-T and MPEG standards mainly relies on a hybrid of block- based transform and interframe predictive coding approaches. In this coding framework, the encoder architecture has the task to exploit both the temporal and spatial redundancies present in the video sequence, which is a rather complex exercise. As a consequence, all standard video encoders have a much higher computational complexity than the decoder (typically five to ten times more complex), mainly due to the temporal correlation exploitation tools, notably the motion estimation process. This type of architecture is well-suited for applications where the video is encoded once and decoded many times, i.e., one-to-many topologies, such as broadcasting or video-on-demand, where the cost of the decoder is more critical than the cost of the encoder.     

AVS视频码流解析软件QtAVS的设计与实现

码流分析对音视频编码标准产业化和应用十分重要.针对我国组织制订的AVS视频编解码标准GB/T 20090.2,利用新一代面向对象、跨平台的图形用户界面语言Qt设计实现了AVS视频码流解析软件QtAVS,该软件能够正确解析AVS视频码流,并可在序列、帧、块3个层次上对码流元素进行可视化显示.QtAVS的所有源代码已在AV...     

适用于MPEG2 MP@ML标准的视频解码器设计

设计了一个适用于 MPEG2 MP@ML 标准的视频解码器结构 ,用 VHDL 语言进行了系统级的仿真和综合。系统工作时钟频率 40 MHz。用标准图象测试序列进行了验证 ,给出了测试结果和有关参数 ,满足 MPEG2 MP@ML 视频解码的实时处理要求。     

Automated Design Flow for Multi-Functional Dataflow-Based Platforms

The implementation of processing platforms supporting multiple applications by runtime reconfigurations on dedicated hardware modules requires the solution of different problems. These problems are notably not-trivial since both platform and application complexities increase year after year. As a consequence, the design process is both time and resource demanding. System configuration along with resources management and mapping remain one of the most challenging problem, particularly when runtime adaptation is required. In this direction, the ISO/IEC SC29WG11 committee (MPEG) has developed the so called MPEG-RVC standards ISO/IEC 23001-4 and 23002-4. This standard provides specifications of video codecs in the form of dataflow programs. In this paper, an integrated design flow to derive optimized multi-functional platforms directly from disjoined high-level specifications is presented. To the authors’ best of knowledge, such an optimization, synthesis and mapping methodology for coarse-grained reconfigurable systems design does not exist within the MPEG-RVC framework. The design flow presented in this paper leverages on an integrated set of independently designed tools, all supporting the RVC standard. Results assessment has been carried out on three different scenarios: an MPEG-RVC decoder, a standard baseline MPEG-RVC JPEG codec and a generalized reconfigurable multi-quality JPEG encoder. For all these scenarios, the proposed design flow has been targeted for a Xilinx Virtex 5 FPGA. Results show how this approach is capable of yielding a reconfigurable design that preserves the original performance of the stand alone non-reconfigurable platform providing, at the same time, considerable area savings featuring a larger set of functionalities. Moreover, platforms programmability, on the basis of the required functionality ID, is automatically handled at runtime without any designer effort.     

Robust Video Transmission With Distributed Source Coded Auxiliary Channel

We propose a novel solution to the problem of robust, low-latency video transmission over lossy channels. Predictive video codecs, such as MPEG and H.26x, are very susceptible to prediction mismatch between encoder and decoder or “drift” when there are packet losses. These mismatches lead to a significant degradation in the decoded quality. To address this problem, we propose an auxiliary codec system that sends additional information alongside an MPEG or H.26x compressed video stream to correct for errors in decoded frames and mitigate drift. The proposed system is based on the principles of distributed source coding and uses the (possibly erroneous) MPEG/H.26x decoder reconstruction as side information at the auxiliary decoder. The distributed source coding framework depends upon knowing the statistical dependency (or correlation) between the source and the side information. We propose a recursive algorithm to analytically track the correlation between the original source frame and the erroneous MPEG/H.26x decoded frame. Finally, we propose a rate-distortion optimization scheme to allocate the rate used by the auxiliary encoder among the encoding blocks within a video frame. We implement the proposed system and present extensive simulation results that demonstrate significant gains in performance both visually and objectively (on the order of 2 dB in PSNR over forward error correction based solutions and 1.5 dB in PSNR over intrarefresh based solutions for typical scenarios) under tight latency constraints.      

Reconfigurable video coding on multicore

The current monolithic and lengthy scheme behind the standardization and the design of new video coding standards is becoming inappropriate to satisfy the dynamism and changing needs of the video coding community. Such scheme and specification formalism does not allow the clear commonalities between the different codecs to be shown, at the level of the specification nor at the level of the implementation. Such a problem is one of the main reasons for the typically long interval elapsing between the time a new idea is validated until it is implemented in consumer products as part of a worldwide standard. The analysis of this problem originated a new standard initiative within the International Organization for Standardization (ISO)/International Electrotechnical Commission (IEC) Moving Pictures Experts Group (MPEG) committee, namely Reconfigurable Video Coding (RVC). The main idea is to develop a video coding standard that overcomes many shortcomings of the current standardization and specification process by updating and progressively incrementing a modular library of components. As the name implies, flexibility and reconfigurability are new attractive features of the RVC standard. Besides allowing for the definition of new codec algorithms, such features, as well as the dataflow-based specification formalism, open the way to define video coding standards that expressly target implementations on platforms with multiple cores. This article provides an overview of the main objectives of the new RVC standard, with an emphasis on the features that enable efficient implementation on platforms with multiple cores. A brief introduction to the methodologies that efficiently map RVC codec specifications to multicore platforms is accompanied with an example of the possible breakthroughs that are expected to occur in the design and deployment of multimedia services on multicore platforms.