Extraction of Video Objects via Surface Optimization and Voronoi Order

We implement a video object segmentation system that integrates the novel concept of Voronoi Order with existing surface optimization techniques to support the MPEG-4 functionality of object-addressable video content in the form of video objects. The major enabling technology for the MPEG-4 standard are systems that compute video object segmentation, i.e., the extraction of video objects from a given video sequence. Our surface optimization formulation describes the video object segmentation problem in the form of an energy function that integrates many visual processing techniques. By optimizing this surface, we balance visual information against predictions of models with a priori information and extract video objects from a video sequence. Since the global optimization of such an energy function is still an open problem, we use Voronoi Order to decompose our formulation into a tractable optimization via dynamic programming within an iterative framework. In conclusion, we show the results of the system on the MPEG-4 test sequences, introduce a novel objective measure, and compare results against those that are hand-segmented by the MPEG-4 committee.     

A real-time video transport system for the best-effort Internet

It is a challenging problem to design an efficient real-time video transport system for the present best-effort Internet because of the strict bandwidth, loss and time constraints and the lack of quality of service (QoS) guarantees from the present IP networks. This paper presents an end-to-end system architecture based on the MPEG-4 Delivery Multimedia Integration Framework (DMIF) for transporting real-time live video over the Internet. The key features of the system include: (1) MPEG-4 DMIF-based terminal architecture; (2) combination of an end-to-end feedback control mechanism and a rate-adaptive encoding algorithm for the best use of the Internet; (3) a robust and efficient packetization scheme for the MPEG-4 bit-stream by using the resynchronization marker approach specified in the MPEG-4 standard; and (4) efficient error control algorithms adopted at the end systems for visual quality enhancement. Simulation and test results using the actual Internet showed that our system is capable of utilizing the available network resource and achieve good perceptual quality at the application level.     

Moving video object segmentation using statistical hypothesistesting

Object-based coding is one of the distinct features of the MPEG-4 standard. Hence, video object segmentation is an indispensable process for MPEG-4 coding. A method of localising moving objects for video object segmentation is introduced which utilises a statistical hypothesis test without the need for true statistical parameters to be known a priori     

Prioritized MPEG-4 Audio-Visual Objects Streaming over the DiffServ

The object-based scalable coding in MPEG-4 is investigated, and a prioritized transmission scheme of MPEG-4 audio-visual objects （AVOs） over the DiffServ network with the QoS guarantee is proposed. MPEG-4 AVOs are extracted and classified into different groups according to their priority values and scalable layers （visual importance）. These priority values are mapped to the 1P DiffServ per hop behaviors （PHB）. This scheme can selectively discard packets with low importance, in order to avoid the network congestion. Simulation results show that the quality of received video can gracefully adapt to network state, as compared with the ‘best-effort＇ manner. Also, by allowing the content provider to define prioritization of each audio-visual object, the adaptive transmission of object-based scalable video can be customized based on the content.     

Encoding and reconstruction of multiview video objects

The authors discuss how disparity-based processing can be used both for compression of multiview video data, and generation of arbitrary viewpoints from the available information of multiple cameras in the context of the MPEG-4 multimedia standard. The examples and results presented here show that viewpoint adaptation toward video objects can be accomplished with low-complexity schemes such as disparity-compensated projection, while high-quality results are presented. The ability to process multiview video is another example of the high flexibility of the MPEG-4 standard, which we expect to be applicable to various new challenging services in the multimedia market     

压缩域MPEG-2到MPEG-4视频转码中不匹配宏块的复原算法

作为一种最有应用前景的先进的压缩编码标准，MPEG-4将会很快在视频传输、存储和剪辑等许多领域中得到应用。不过，MPEG-2和MPEG-4这两种优秀的视频系统肯定会在很长一段时间内共存。本文提出了一种基于MC-DCT域的转码方法。重点解决了两种标准视频流中宏块编码模式不匹配的一些关键问题。仿真结果表明，本算法不仅能有效地实现MPEG-2到MPEG-4的转码，满足实时性和低处迟的需求，而且能隐藏或复原编码模式不匹配宏块，明显地改变视觉质量。     

MPEG-4: a multimedia compression standard for interactiveapplications and services

The Moving Pictures Experts Group (MPEG), which produced the MPEG-1 and MPEG-2 video and audio compression standards, is developing the MPEG-4 standard. MPEG-4 targets interactive multimedia applications and will become a standard in 1999. As well as an increased compression efficiency, MPEG-4 will also offer content-based functionality, i.e. the possibility of accessing and manipulating individual objects in the picture. Furthermore, MPEG-4 will offer possibilities for efficient video storage and for transmission over poor audio and video channels at bit rates between 5 kbit/s and 4 Mbit/s. This paper gives an overview of the state of the art of MPEG-4 development, concentrating especially on video content-based functionality, which is so important for interactive applications     

一种基于因特网的MPEG-4视频流端到端传输方案

在保证网络资源利用率高的条件下如何建立一个使所传图像主观质量最好且有效的MPEG－4传输系统，已成为一个具有挑战性的课题，就此问题提出了一个在因特网上建立MPEG－4视频流的端到端传输方案。     

A perspective: the role of identifiers in managing and protectingintellectual property in the digital age

There is an essential need for the owners of intellectual property rights to be able to identify uniquely the creative components which make up digitized multimedia content. The relationship between these identifiers and the “well-formed” metadata which describe them is critical to the successful management of rights in an electronic trading environment. By persistently associating creation identifiers with digital content, rights holders will effectively be able to licence, monitor, and track the usage of rights across digital networks. Within the evolution of the MPEG-4 standard, representatives of rights holders from across the creative industries worked with technologists to devise an effective solution for managing creation identifiers and their associated metadata within MPEG-4 objects. This was achieved in such a manner that the identifiers will continue to be associated with content that is manipulated and edited. In addition, through the valuable contribution of leading watermarking and encryption specialists, the MPEG-4 standard will contain an interface for applying intellectual property management and protection (IPMP) systems. These IPMP systems can be applied to MPEG-4 software and hardware devices that manage the delivery of copyright content to prevent unauthorized copying, the removal of identifiers, and therefore control and restrict fraudulent use. This paper describes and documents these issues which rights holders have worked collectively to address within the evolving MPEG-4 standard     

Segmentation of moving objects in image sequence: A review

Segmentation of objects in image sequences is very important in many aspects of multimedia applications. In second-generation image/video coding, images are segmented into objects to achieve efficient compression by coding the contour and texture separately. As the purpose is to achieve high compression performance, the objects segmented may not be semantically meaningful to human observers. The more recent applications, such as content-based image/video retrieval and image/video composition, require that the segmented objects be semantically meaningful. Indeed, the recent multimedia standard MPEG-4 specifies that a video is composed of meaningful video objects. Although many segmentation techniques have been proposed in the literature, fully automatic segmentation tools for general applications are currently not achievable. This paper provides a review of this important and challenging area of segmentation of moving objects. We describe common approaches including temporal segmentation, spatial segmentation, and the combination of temporal-spatial segmentation. As an example, a complete segmentation scheme, which is an informative part of MPEG-4, is summarized.     

Refreshment need metrics for improved shape and texture object-based resilient video coding

Video encoders may use several techniques to improve error resilience. In particular, for video encoders that rely on predictive (inter) coding to remove temporal redundancy, intra coding refreshment is especially useful to stop error propagation when errors occur in the transmission or storage of the coded streams, which can cause the decoded quality to decay very rapidly. In the context of object-based video coding, the video encoder can apply intra coding refreshment to both the shape and the texture data. In this paper, shape refreshment need and texture refreshment need metrics are proposed which can be used by object-based video encoders, notably MPEG-4 video encoders, to determine when the shape and the texture of the various video objects in the scene should be refreshed in order to improve the decoded video quality, e.g., for a given bit rate.     

基于MPEG-4视频标准数字视频录像机的设计与实现

国内外许多公司都在开发有关MPEG-4视频标准的产品,最具代表意义的即是数字视频录像机(DVR)。为了缩短开发周期,这里介绍基于嵌入式Linux操作系统,应用专用音视频编解码芯片AT2042实现数字视频录像机功能,该系统实现MPEG-4视频标准高级框架的编解码器,并在此基础上添加数字硬盘的功能,例如编码存储、解码播放、快进、快退和暂停等功能。最后给出系统的实际运行的测试结果。该系统已实现对视频数据的编、解码,且实现MPEG-4/MPEG-2/MPEG-1 H.263视频标准,并已成为成型产品推向市场。     

MPEG digital video-coding standards

The efficient digital representation of image and video signals has been the subject of considerable research over the past 20 years. Digital video-coding technology has developed into a mature field and products have been developed that are targeted for a wide range of emerging applications, such as video on demand, digital TV/HDTV broadcasting, and multimedia image/video database services. With the increased commercial interest in video communications, the need for international image- and video-compression standards arose. To meet this need, the Moving Picture Experts Group (MPEG) was formed to develop coding standards. MPEG-1 and MPEG-2 video-coding standards have attracted much attention worldwide, with an increasing number of very large scale integration (VLSI) and software implementations of these standards becoming commercially available. MPEG-4, the most recent MPEG standard that is still under development, is targeted for future content-based multimedia applications. We provide an overview of the MPEG video-coding algorithms and standards and their role in video communications. We review the basic concepts and techniques that are relevant in the context of the MPEG video-compression standards and outline MPEG-1 and MPEG-2 video-coding algorithms. The specific properties of the standards related to their applications are presented, and the basic elements of the forthcoming MPEG-4 standard are also described. We also discuss the performance of the standards and their success in the market place     

A real-time inverse quantised transform for multi-standard with dynamic resolution support

In this paper, a real-time configurable intelligent property (IP) core is presented for image/video decoding process in compatibility with the standard MPEG-4 Visual and the standard H.264/AVC. The inverse quantised discrete cosine and integer transform can be used to perform inverse quantised discrete cosine transform and inverse quantised inverse integer transforms which only required shift and add operations. Meanwhile, COordinate Rotation DIgital Computer iterations and compensation steps are adjustable in order to compensate for the video compression quality regarding various data throughput. The implementations are embedded in publicly available software XVID Codes 1.2.2 for the standard MPEG-4 Visual and the H.264/AVC reference software JM 16.1, where the experimental results show that the balance between the computational complexity and video compression quality is retained. At the end, FPGA synthesised results show that the proposed IP core can bring advantages to low hardware costs and also provide real-time performance for Full HD and 4K–2K video decoding.     

Chiariglione and the birth of MPEG

Leonardo Chiariglione, the father of the Moving Pictures Experts Group and the driving force behind its standards for digitized video is now pushing a standard for Internet images manipulable by the user. As first conceived, MPEG-2 was to standardize the digital coding of interlaced TV pictures (in which a field made up of every other scan line is followed by another field consisting of the remaining scan lines). The coding was to occur at about 10 Mb/s, and standard digital television was the main goal. Then MPEG-3 was to formulate coding standards for pictures at 40 Mb/s, with a view to HDTV. The current effort of the Moving Pictures Experts Group, MPEG-4, is geared to digital coding at low bit-rates and was at first conceived rather vaguely in a multimedia context     

Hidden Markov Model Inversion for Audio-to-Visual Conversion in an MPEG-4 Facial Animation System

MPEG-4 standard allows composition of natural or synthetic video with facial animation. Based on this standard, an animated face can be inserted into natural or synthetic video to create new virtual working environments such as virtual meetings or virtual collaborative environments. For these applications, audio-to-visual conversion techniques can be used to generate a talking face that is synchronized with the voice. In this paper, we address audio-to-visual conversion problems by introducing a novel Hidden Markov Model Inversion (HMMI) method. In training audio-visual HMMs, the model parameters {_av} can be chosen to optimize some criterion such as maximum likelihood. In inversion of audio-visual HMMs, visual parameters that optimize some criterion can be found based on given speech and model parameters {_av}. By using the proposed HMMI technique, an animated talking face can be synchronized with audio and can be driven realistically. The HMMI technique combined with MPEG-4 standard to create a virtual conference system, named VIRTUAL-FACE, is introduced to show the role of HMMI for applications of MPEG-4 facial animation.     

An MPEG-4 Renderer for High Quality Video Composition and Texture Mapping

This paper presents the hardware architecture of a co-processor supporting the real time rendering of all 2D natural or synthetic visual objects proposed by the MPEG-4 standard as well as sprite decoding. It enables the composition and the transformation of natural video objects and the texture mapping on triangles, allowing the 2D-mesh decoding. This architecture is able to render scenes that are compliant with MPEG-4 Main Profile, Level3 and Hybrid Visual Profile. The co-processor is designed to be used in a shared memory system architecture. It consists in a dedicated implementation that seeks the best compromise between cost and performances. In a first step, a software model is used to guarantee the visual quality of the rendered scene and to validate the algorithmic choices. Then, the complexity and performances of this novel architecture are evaluated. Finally, a behavioral model validates the architectural choices.     

Using MPEG-7 and MPEG-21 for personalizing video

As multimedia content has proliferated over the past several years, users have begun to expect that content be easily accessed according to their own preferences. One of the most effective ways to do this is through using the MPEG-7 and MPEG-21 standards, which can help address the issues associated with designing a video personalization and summarization system in heterogeneous usage environments. This three-tier architecture provides a standards-compliant infrastructure that, in conjunction with our tools, can help select, adapt, and deliver personalized video summaries to users. In extending our summarization research, we plan to explore semantic similarities across multiple simultaneous news media sources and to abstract summaries for different viewpoints. Doing so will allow us to track a semantic topic as it evolves into the future. As a result, we should be able to summarize news repositories into a smaller collection of topic threads.