Automatic face location detection for model-assisted rate control in H.261-compatible coding of video

We present a novel and practical way to integrate techniques from computer vision to low bit-rate coding systems for video teleconferencing applications. Our focus is to locate and track the faces and selected facial features of persons in typical head-and-shoulders video sequences, and to exploit the location information in a ‘classical’ video coding/decoding system. The motivation is to enable the system to encode selectively various image areas and to produce perceptually pleasing coded images where faces are sharper. We refer to this approach—a mix of classical waveform coding and model—based coding-as model-assisted coding. We propose two totally automatic algorithms which, respectively, perform the detection of a head outline, and identify an ‘eyes-nose-mouth’ region, both from downsampled binary thresholded edge images. The algorithms operate accurately and robustly, even in cases of significant head rotation or partial occlusion by moving objects. We show how the information about face and facial feature location can be advantageously exploited by low bit-rate waveform-based video coders. In particular, we describe a method of object-selective quantizer control in a standard coding system based on motion-compensated discrete cosine transform—CCITT's recommendation H.261. The approach is based on two novel algorithms, namely buffer rate modulation and buffer size modulation. By forcing the rate control algorithm to transfer a fraction of the total available bit-rate from the coding of the non-facial to that of the facial area, the coder produces images with better-rendered facial features, i.e. coding artefacts in the facial area are less pronounced and eye contact is preserved. The improvement was found to be perceptually significant on video sequences coded at the ISDN rate of 64 kbps, with 48 kbps for the input (color) video signal in QCIF format.     

Error-resilient decoding of context-based adaptive binary arithmetic codes

This paper addresses the problem of error-resilient decoding of bitstreams produced by the CABAC (context-based adaptive binary arithmetic coding) algorithm used in the H.264 video coding standard. The paper describes a maximum a posteriori (MAP) estimation algorithm improving the CABAC decoding performances in the presence of transmission errors. Methods improving the re-synchronization and error detection capabilities of the decoder are then described. A variant of the CABAC algorithm supporting error detection based on a forbidden interval is presented. The performances of the decoding algorithm are first assessed with theoretical sources and by considering different binarization codes. They are compared against those obtained with Exp-Golomb codes and with a transmission chain making use of an error-correcting code. The approach has been integrated in an H.264/MPEG-4 AVC video coder and decoder. The PSNR gains obtained are discussed.     

Very low-bit-rate segmentation-based video coding using contour andtexture prediction

The most efficient video coding standard for low bit rates (around 64 kb/s) is the H.261 algorithm recommended by ITU-TS. However, in certain applications such as mobile audiovisual communications and videophone through PSTN, the available transmission bandwidth is very limited. Therefore codecs working at very low bit rates are required. The paper presents a segmentation-based video coding algorithm that can work at rates as low as 10 kb/s. A novel representation of the contour information using a number of control points is proposed to estimate the contour shapes and locations from the previous frame by using the motion information. The texture parameters are also predicted and only the residual values are entropy coded. In addition two novel postprocessing techniques for edge-profile smoothing and jagged-edge rectification are described     

A broadband switching experiment

A broadband switching experiment was presented at the ISS'87 and Telecom'87 conferences in Phoenix, Arizona, and Geneva, Switzerland, respectively and at the March 1988 Fair in Hanover, West Germany. This experiment was significant in that it illustrated techniques for switching broadband signals with a wide range of frequencies, formats and ultimate purposes. In particular, simultaneous switching (within the same switch fabric) of 30 MHz PFM, 45 Mb/s PCM, and 140 Mb/s PCM was demonstrated. These signals carried full-motion, full-colour NTSC video for desk-to-desk video teleconferencing, off-the-air broadcast video programming, and surveillance camera video. They also carried RGB video, digitized stored video images, and computer-computer communications. Software features afforded a friendly human interface, allowing multiple, flexible service capabilities. Further, the experimental system, when deployed in Phoenix, demonstrated control of its capabilities via a narrowband ISDN link to an optically remoted 5ESS® switch module one km distant, which homed on a 5ESS host switch about 32 km further away. The technologies employed in the switch and the surrounding equipment are detailed, and the significance of this experiment relative to the broadband ISDN (BISDN) thinking is discussed.     

Packet video

The 1990s will see a large increase in the use of video for many purposes, made possible through the use of new optical storage and transmission methods, as well as radio and satellite communication. HDTV, new broadcast services and 64 kbit/s ISDN video are all at an advanced stage of development. Digital methods of transmission and storage feature prominently in future scenarios of video usage. To avoid long delays in transmission, packet video techniques for storage of high-quality material on optical discs are being considered. Coded video signals can be transmitted in packet form at variable bit rate. The author discusses the principles of packet video. Packet loss, queueing delay and layered coding are discussed. The future developments are also discussed     

分析视频图像的压缩传输技术

介绍视频图像信号的压缩传输技术,简单论述几种压缩编码方式,讨论基于MPEG－2编码的视频图像传输计划,阐述JPEG2000编码系统整体流程,并研究JPEG2000中对视频图像进行压缩的相关传输技术。     

A survey of hybrid MC/DPCM/DCT video coding distortions

The motion-compensated hybrid DCT/DPCM algorithm has been successfully adopted in various video coding standards, such as H.261, H.263, MPEG-1 and MPEG-2. However, its robustness is challenged in the face of an inadequate bit allocation, either globally for the whole video sequence, or locally as a result of an inappropriate distribution of the available bits. In either of these situations, the trade-off between quality and the availability of bits results in a deterioration in the quality of the decoded video sequence, both in terms of the loss of information and the introduction of coding artifacts. These distortions are an important factor in the fields of filtering, codec design, and the search for objective psychovisual-based quality metrics; therefore, this paper presents a comprehensive analysis and classification of the numerous coding artifacts which are introduced into the reconstructed video sequence through the use of the hybrid MC/DPCM/DCT video coding algorithm. Artifacts which have already been briefly described in the literature, such as the blocking effect, ringing, the mosquito effect, MC mismatch, blurring, and color bleeding, will be comprehensively analyzed. Additionally, we will present artifacts with unique properties which have not been previously identified in the literature.     

A 3D-DCT video encoder using advanced coding techniques for low power mobile device

The three-dimensional discrete cosine transform (3D-DCT) has been researched as an alternative to existing dominant video standards based on motion estimation and compensation. Since it does not need to search macro block for inter/intra prediction, 3D-DCT has great advantages for complexity. However, it has not been developed well because of poor video quality while video standards such as H.263(+) and HEVC have been blooming. In this paper, we propose a new 3D-DCT video coding as a new video solution for low power mobile technologies such as Internet of Things (IoT) and Drone. We focus on overcoming drawbacks reported in previous research. We build a complete 3D-DCT video coding system by adopting existing advanced techniques and devising new coding algorithms to improve overall performance of 3D-DCT. Experimental results show proposed 3D-DCT outperforms H.264 low power profiles while offering less complexity. From GBD-PSNR, proposed 3D-DCT provides better performance by average 4.6 dB.     

运动补偿视频编码中DCT编码效率的研究

在视频序列图像编码中，运动补偿帧差图像的统计特性与静止自然图像有明显差别，对静止自然图像可以称为准最佳变换的离散余弦变换对帧差图像的编码效率有所下降。本文对这一问题进行了理论分析和实验验证，从而指出进一步提高视频序列图像编码效率的潜力所在。     

Image sequence coding at very low bit rates: a review

This paper presents a review of promising techniques for very low bit-rate, below 64 kb/s, image sequence coding. Image sequence coding at such low rates will be a crucial technique in forthcoming visual services, e.g., visual information transmission and storage. A typical application is to transmit moving videophone scenes through the existing analog telephone lines or via a mobile channel. Two types of potential coding techniques are addressed: waveform-based image sequence coding and model-based image sequence coding.     

Video on phone lines: technology and applications

This paper reviews the telephone loop plant characteristics, current DSL (digital subscriber line) technologies, recent efforts in video coding standards, and the interrelationship between DSL technologies and visual communications over subscriber lines. In overview of the loop plant characteristics we examine its physical makeup and transmission properties, where for the latter we discuss frequency and time responses of wire-pair lines and the impairments of echo, crosstalk, impulse noise, and radio frequency interference. We trace the historical development of various DSL technologies and comment on possible future evolution. Transmission technologies used in the ISDN basic-access DSL, the high bit-rate DSL, and the asymmetric DSL are portrayed. And the issue of spectrum compatibility among different transmission systems is explained. Several important video coding standards are briefly described, including ITU-T's H.261 and ISO's JPEG and MPEG series, which are either completed or emerging. The synergistic relationship between these standards and the DSL technologies is elucidated. As a result, DSL technologies provide the potential of delivering certain broadband services well in advance of direct fiber access for telephone subscribers     

World-wide standardization of broadband ISDN

A growing market demand for high-performance voice, data, image and video networks, coupled with advances in high-speed technologies has heightened interest in broadband ISDN (BISDN) throughout the world. Network providers, equipment manufacturers and standards bodies are working jointly to accelerate the definition of BISDN interfaces in anticipation of emerging market opportunities. BISDN is being designed as the foundation upon which to implement very flexible, high-bandwidth, high-performance networks. A multiplexing concept called asynchronous transfer mode (ATM), which can offer bandwidth on demand at rates exceeding 150 Mb/s, high-speed packet switching, and an efficient protocol which adapts existing information streams into the ATM format are key attributes of the emerging BISDN network standards. This paper reviews the motivations behind BISDN, explains its key attributes and challenges and discusses possible strategies for evolving today's network toward BISDN.     

基于深度学习的视频编码发展现状与未来展望

近年来,深度学习技术不仅在人工智能领域取得了巨大成功,也为视频编码领域带来了新的发展机遇。文章从两个方面介绍了深度学习技术在视频编码领域的发展现状,即传统编码框架下深度学习视频编码工具和以深度学习模型为基础的视频编码新框架,并对相关代表性工作进行了详细介绍和性能分析。最后,对深度学习视频编码技术面临的挑战和未来发展方向做了分析和展望。     

White and narrow band image compressor based on a new color space for capsule endoscopy

In this paper, we present the design of a low power and hardware efficient image compressor integrated circuit for wireless capsule endoscopy application. The proposed compression algorithm supports dual-band imaging, that is, works on both white-band imaging (WBI) and narrow-band imaging (NBI). The scheme uses a novel color-space and simple predictive coding for optimized performance. Based on the nature of the narrow- and white-band endoscopic images and video sequences, several sub-sampling schemes are introduced. The proposed dual-band compressor is designed in such as way that it can easily be interfaced with any commercial low power image sensor that outputs RGB image pixels in a raster scan fashion, eliminating the need of large buffer memory and temporary storage. Both NBI and WBI reconstructed images have been verified by medical doctors for acceptability. Compared to other designs targeted to video capsule endoscopy, the proposed algorithm performs strongly with a compression ratio of 80.4% (for WBI) and 79.2% (for NBI), and a high reconstruction peak-signal-to-noise-ratio (over 43.7 dB for both bands). The results of the fabricated chip are also presented.     

A neural synaptic compression codec for efficient image transmission

For efficient cellular communication channel usage, we propose a neural computation model for image coding. In a constant-time unsupervised learning, our neural model approximates optimal pattern clustering from training example images through a memory adaptation process, and builds a compression codebook in its synaptic weight matrix. This neural codebook can be distributed to both ends of a transmission channel for fast codec operations on general images. The transmission is merely the indices of the codebook entries best matching the patterns in the image to be transmitted. These indices can further be compressed through a classical entropy coding method to yield even more transmission reduction. Other advantages of our model are the low training time complexity, high utilization of neurons, robust pattern clustering capability, and simple computation. A VLSI implementation is also highly suitable for the intrinsic parallel nature of neural networks. Our compression results are competitive compared to JPEG and wavelet methods. We also reveal the general codebook's cross-compression results, filtering effects by special training methods, and learning enhancement techniques for obtaining a compact codebook to yield both high compression and picture quality.     

Model-based coding of videophone images

The requirement for improved picture quality in videophone and videoconference systems operating at low bit-rates has stimulated interest in model-based image coding. Two model-based coding techniques are described which are capable of producing either improved picture quality at bit-rates around 64 kbit/s or acceptable picture quality at bit-rates far lower than 64 kbit/s. The first technique produces facial expressions by using feature code-books; the second technique produces facial expressions by distorting an underlying three-dimensional model. The problems of image analysis and synthesis, which are concomitant in model-based coding, are discussed     

一种图象序列的神经网络宏块活动性分类算法

在视频编码中，图象序列宏块活动性的正确分类有着非常重要的意义。本文提出了一种神经网络的图象序列宏块活动性分类新算法。     

用VC++实现视频监控录像

文章分析了视频编码和监控录像的VC 编程实现，重点提出在录像过程中用MPEG4再编码，实现视频监控系统中高性能硬盘录像。     

SVM based approach for complexity control of HEVC intra coding

The High Efficiency Video Coding (HEVC) is adopted by various video applications in recent years. Because of its high computational demand, controlling the complexity of HEVC is of paramount importance to appeal to the varying requirements in many applications, including power-constrained video coding, video streaming, and cloud gaming. Most of the existing complexity control methods are only capable of considering a subset of the decision space, which leads to low coding efficiency. While the efficiency of machine learning methods such as Support Vector Machines (SVM) can be employed for higher precision decision making, the current SVM-based techniques for HEVC provide a fixed decision boundary which results in different coding complexities for different video content. Although this might be suitable for complexity reduction, it is not acceptable for complexity control. This paper proposes an adjustable classification approach for Coding Unit (CU) partitioning, which addresses the mentioned problems of complexity control. Firstly, a novel set of features for fast CU partitioning is designed using image processing techniques. Then, a flexible classification method based on SVM is proposed to model the CU partitioning problem. This approach allows adjusting the performance-complexity trade-off, even after the training phase. Using this model, and a novel adaptive thresholding technique, an algorithm is presented to deliver video encoding within the target coding complexity, while maximizing the coding efficiency. Experimental results justify the superiority of this method over the state-of-the-art methods, with target complexities ranging from 20% to 100%.     

Performance of the color set partitioning in hierarchical tree scheme (CSPIHT) in video coding

This paper describes the implementation of the recently introducedcolor set partitioning in hierarchical tree (CSPIHT)-based scheme for video coding. The intra- and interframe coding performance of a CSPIHT-based video coder (CVC) is compared against that of the H.263 at bit rates lower than 64 kbit/s. The CVC performs comparably or better than the H.263 at lower bit rates, whereas the H.263 performs better than the CVC at higher bit rates. We identify areas that hamper the performance of the CVC and propose an improved scheme that yields better performance in image and video coding in low bit-rate environments.