Geometry-driven photorealistic facial expression synthesis

Expression mapping (also called performance driven animation) has been a popular method for generating facial animations. A shortcoming of this method is that it does not generate expression details such as the wrinkles due to skin deformations. In this paper, we provide a solution to this problem. We have developed a geometry-driven facial expression synthesis system. Given feature point positions (the geometry) of a facial expression, our system automatically synthesizes a corresponding expression image that includes photorealistic and natural looking expression details. Due to the difficulty of point tracking, the number of feature points required by the synthesis system is, in general, more than what is directly available from a performance sequence. We have developed a technique to infer the missing feature point motions from the tracked subset by using an example-based approach. Another application of our system is expression editing where the user drags feature points while the system interactively generates facial expressions with skin deformation details.     

表演驱动的二维人脸表情合成

为了由视频进行驱动生成人脸表情动画,提出一种表演驱动的二维人脸表情合成方法。利用主动外观模型算法对人脸的关键点进行定位,并从关键点提取出人脸的运动参数;对人脸划分区域,并获取目标人脸的若干样本图像;从人脸的运动参数获取样本图像的插值系数,对样本图像进行线性组合来合成目标人脸的表情图像。该方法具有计算简单有效、真实感强的特点,可以应用于数字娱乐、视频会议等领域。     

MPEG-4与MPEG-21

MPEG-4:互动性的视频技术 1999年1月,MPEG-4标准的第一版推出,不过MPEG组织很快又在同年12月公布了该标准的第二版,并被ISO正式编号为ISO/IEC14496".标准提出之后,MPEG-4引起人们的广泛关注,但当时还缺乏实质性的应用支撑,很多消费者都不了解它的具体优点体现在什么地方.本来,互动性是MPEG-4标准最大的要务,用户不仅仅可以被动地接收、播放MPEG-4视频流,更可以亲手进行制作、编辑,带领人们进入一个全新的媒体互动时代.然而,人们更直接体验到的好处还是MPEG-4可以在高压缩率条件下获得较高清晰度的画面质量.那么,MPEG-4如何做到这一点?     

Data-driven facial expression synthesis via Laplacian deformation

Realistic talking heads have important use in interactive multimedia applications. This paper presents a novel framework to synthesize realistic facial animations driven by motion capture data using Laplacian deformation. We first capture the facial expression from a performer, then decompose the motion data into two components: the rigid movement of the head and the change of the facial expression. By making use of the local-detail preserving property of the Laplacian coordinate, we clone the captured facial expression onto a neutral 3D facial model using Laplacian deformation. We choose some expression “independent points” in the facial model as the fixed points when solving the Laplacian deformation equations. Experimental results show that our approach can synthesize realistic facial expressions in real time while preserving the facial details. We compare our method with the state-of-the-art facial expression synthesis methods to verify the advantages of our method. Our approach can be applied in real-time multimedia systems.     

人脸表情识别与合成系统模型设计

本文提出了一种新的用于人脸表情识别与合成的情感模型,该模型是基于已泛化的和非线性映射关系的五层神经网络.模型的输入和输出层有相同数目的运动单元,在中间层可以实现特征的映射和情感空间的构造.从输入层到中间层的映射是表情识别,从中间层到输出层的映射是根据情感值进行表情合成.神经网络的训练采用典型的6种表情作为训练样本,最后通过实验证明了该模型在进行表情识别与合成时的可行性.     

MPEG-4标准概述

MPEG-4是基于对象的编码标准，以其基于内容的交互性、高效的压缩性与通用的访问性等特点，被广泛的应用于电子、通信和计算机三大产业领域。文章概述了MPEG-4标准的结构特点及其应用。     

Speech coding in MPEG-4

While previous MPEG Audio standards mainly were focused on the representation of audio signals close to or equal to CD quality, the new MPEG-4 Audio standard extends the range of applicability towards significantly lower bit rates. Furthermore it offers extended functionalities for the representation of natural and even synthetic audio signals in an object oriented fashion. This paper gives a brief overview on the complete audio part of the MPEG-4 standard and more detailed information on its parts related to speech coding.This paper was written while the author was research visitor at Lucent Technologies, Bell Laboratories, Murray Hill, NJ, USA.     

Hardwired MPEG-4 repetitive padding

We consider two hardwired solutions for repetitive padding, a performance restricting algorithm for real time MPEG-4 execution. The first solution regards application specific implementations, the second regards general purpose processing. For the application specific implementations we propose a systolic array structure. To determine the chip area and speed, we have synthesized its VHDL models for two field-programmable gate array families-Xilinx and Altera. Depending on the implemented configuration, the unit can process between 77 K and 950 K macroblocks per second (MB/s) when mapped on FPGA chips containing less than 10 K logical gates and frequency capabilities below 100 MHz. The second approach regards an augmentation of a general-purpose arithmetic logical units with an extra functionality added to perform repetitive padding. At trivial hardware costs of a few hundred 2/spl times/2 AND-OR logic gates, we achieve an order of magnitude speed-up compared to nonaugmented general purpose processor padding. The proposed hardware solutions meet the requirements of all MPEG-4 visual profile levels. Both approaches have been proven to be scalable and fit into different architectural concepts and operand widths.     

MPEG-4标准概述

MPEG-4是基于对象的编码标准,以其基于内容的交互性、高效的压缩性与通用的访问性等特点,被广泛的应用于电子、通信和计算机三大产业领域。文章概述了MPEG-4标准的结构特点及其应用。     

Dejittering in the transport of MPEG-2 and MPEG-4 video

When multimedia information is transported over a packet-switched network, the quality of presentation can be degraded due to network delay variation or jitter. This paper presents a dejittering scheme that can be used in the transport of MPEG-4 and MPEG-2 video to absorb any introduced network jitter, thus preserving the presentation quality of transported media streams. The dejittering scheme is based on the statistical approximation of delay variation in the arrival times of video packets carrying encoded clock reference values and a filtering and re-stamping mechanism. In addition, a brief overview of the MPEG-4 system is presented.     

Expressive facial animation synthesis by learning speech coarticulation and expression spaces

Synthesizing expressive facial animation is a very challenging topic within the graphics community. In this paper, we present an expressive facial animation synthesis system enabled by automated learning from facial motion capture data. Accurate 3D motions of the markers on the face of a human subject are captured while he/she recites a predesigned corpus, with specific spoken and visual expressions. We present a novel motion capture mining technique that "learns" speech coarticulation models for diphones and triphones from the recorded data. A phoneme-independent expression eigenspace (PIEES) that encloses the dynamic expression signals is constructed by motion signal processing (phoneme-based time-warping and subtraction) and principal component analysis (PCA) reduction. New expressive facial animations are synthesized as follows: First, the learned coarticulation models are concatenated to synthesize neutral visual speech according to novel speech input, then a texture-synthesis-based approach is used to generate a novel dynamic expression signal from the PIEES model, and finally the synthesized expression signal is blended with the synthesized neutral visual speech to create the final expressive facial animation. Our experiments demonstrate that the system can effectively synthesize realistic expressive facial animation     

MPEG-4标准及其应用

简单介绍了多媒体标准MPEG-4的发展和组成部分,结合其特点和关键技术,对其应用情况作了简要说明。     

基于MPEG-4 FGS编码视频流的网络方案

针对精细可扩展性编码FGS的特点,该文提出了已编码数据三层分级存储的思想,并给出了一个相应的端到端的网络架构,同时给出了一个简单实用的基于速率的TCP友好协议TLCTFP和相应的启发式速率调整算法。利用编码与传输分离达到服务器发送端低开销、同时利用视频数据分层和网络带宽的有效利用尽量保证和优化视频图像质量,并保持了与TCP的友好相处。     

MPEG-4中的运动图像分割算法

MPEG-4采纳了基于对象的编码技术,它要求对图像和视频作更多的分析,甚至是理解。基于对象的编码是MPEG-4的一个重要特点,但是对象的分割问题至今仍未得到满意的解决。为了应用MPEG-4视频标准,视频序列的每一帧应该根据视频对象平面来描述,编码前的首要工作是视频对象的识别,把每一帧分解为若干视频对象平面,每个视频对象平面代表不同的有语义的对象。该论文介绍了MPEG-4的标准、国际上图像分割的发展状况,在此基础上提出了利用空时域信息实现MPEG-4视频对象的自动分割的算法。     

MPEG-4中的运动图像分割算法

MPEG-4采纳了基于对象的编码技术，它要求对图像和视频作更多的分析，甚至是理解。基于对象的编码是MPEG-4的一个重要特点，但是对象的分割问题至今仍未得到满意的解决。为了应用MPEG-4视频标准，视频序列的每一帧应该根据视频对象平面采描述，编码前的首要工作是视频对象的识别，把每一帧分解为若干视频对象平面，每个视频对象平面代表不同的有语义的对象。该论文介绍了MPEG-4的标准、国际上图像分割的发展状况，在此基础上提出了利用空时域信息实现MPEG-4视频对象的自动分割的算法。     

播放器

2005年MP3播放器出现了一些新趋势，彩屏的广泛使用，大容量存储的走红，可播放视频功能的兴起等激起了人们对MP3播放器的新一轮消费热情。利润可观自然让不少人红眼，品牌林立，产品质量参并不齐成为了今年市场的又一大特点，[编者按]     

Watermarking of MPEG-4 video objects

The recent finalization of MPEG-4 will make this standard very attractive for a large range of applications such as video editing, Internet video distribution, wireless video communications. Some of these applications are likely to get great benefit from watermarking technology, since it can enable a number of innovative services, such as conditional access policies, data annotation, data labeling, content authentication, to be implemented at a low price. One of the key points of the MPEG-4 standard is the possibility to access and manipulate objects within a video sequence. Thus object watermarking has to be achieved in such a way that, while a video object is transferred from a sequence to another, it is still possible to correctly access the data embedded within the object itself. The algorithm proposed in this paper embeds a watermark in each video object by imposing a particular relationship between some predefined pairs of quantized discrete cosine transform (DCT) coefficients in the luminance blocks of pseudo-randomly selected macroblocks (MBs). Watermarks are equally embedded into intra and inter MBs. Experimental results are presented validating the effectiveness of the proposed approach.     

Scalable authentication of MPEG-4 streams

This paper presents three scalable and efficient schemes for authenticating MPEG-4 streams: the Flat Authentication Scheme, the Progressive Authentication Scheme, and the Hierarchical Authentication Scheme. All the schemes allow authentication of MPEG-4 streams over lossy networks by integrating seamlessly digital signatures and erasure correction coding with MPEG-4's fine granular scalability. A prominent feature of our schemes is their "sign once, verify many ways" property, i.e., they generate only one digital signature per compressed MPEG-4 object group, but allow clients to verify the authenticity of any down-scaled version of the original signed object group.     

MPEG-4视频编解码系统的优化设计

本文提出并实现了一种四步搜索块匹配的运动估计算法，并在运动估计算法的实现中给出了整象素搜索与半象素搜索相结合、单矢量估计与四矢量估计相结合的改进方法，该算法能够有效的提高编码效率。