VLSI Implementation of H.264 Video Decoder for Mobile Multimedia Application

In this letter, we present a design of a single chip video decoder called advanced mobile video ASIC (A‐MoVa) for mobile multimedia applications. This chip uses a mixed hardware/software architecture to improve both its performance and its flexibility. We designed the chip using a partition between the hardware and software blocks, and developed the architecture of an H.264 decoder based on the system‐on‐a‐chip (SoC) platform. This chip contains 290,000 logic gates, 670,000 memory gates, and its size is 7.5 mm×7.5 mm (using 0.25 micron 4‐layers metal CMOS technology).     

Hardware implementation of a disparity estimation scheme for real-time compression in 3D imaging applications

This paper presents a novel hardware implementation of a disparity estimation scheme targeted to real-time Integral Photography (IP) image and video sequence compression. The software developed for IP image compression achieves high quality ratios over classic methodologies by exploiting the inherent redundancy that is present in IP images. However, there are certain time constraints to the software approach that must be confronted in order to address real-time applications. Our main effort is to achieve real-time performance by implementing in hardware the most time-consuming parts of the compression algorithm. The proposed novel digital architecture features minimized memory read operations and extensive simultaneous processing, while taking into concern the memory and data bandwidth limitations of a single FPGA implementation. Our results demonstrate that the implemented hardware system can successfully process high resolution IP video sequences in real-time, addressing a vast range of applications, from mobile systems to demanding desktop displays.     

一种新型多媒体SoC验证平台原理及其实现

提出了一种基于ADSP-BF537的新型多媒体SoC验证平台,以满足多媒体SoC音视频编解码功能模块的实时验证。介绍了整个平台的基本组成,以及BF537与SoC接口的软硬件设计;最后,以验证用于SoC的MP3硬件解码器模块为例,讨论了如何利用BF537,在多媒体SoC的FPGA原型内进行软硬件协同验证。该验证方案已经成功应用在深圳艾科创新微电子有限公司的一款多媒体SoC设计流程中。     

Distributed multimedia systems

A distributed multimedia system (DMS) is an integrated communication, computing, and information system that enables the processing, management, delivery, and presentation of synchronized multimedia information with quality-of-service guarantees. Multimedia information may include discrete media data, such as text, data, and images, and continuous media data, such as video and audio. Such a system enhances human communications by exploiting both visual and aural senses and provides the ultimate flexibility in work and entertainment, allowing one to collaborate with remote participants, view movies on demand, access on-line digital libraries from the desktop, and so forth. In this paper, we present a technical survey of a DMS. We give an overview of distributed multimedia systems, examine the fundamental concept of digital media, identify the applications, and survey the important enabling technologies     

Synthesis of hardware models in C with pointers and complex datastructures

One of the greatest challenges in a C/C++-based design methodology is efficiently mapping C/C++ models into hardware. Many networking and multimedia applications implemented in hardware or mixed hardware/software systems now use complex data structures stored in multiple memories, so many C/C++ features that were originally designed for software applications are now making their way into hardware. Such features include dynamic memory allocation and pointers for managing data. We present a solution for efficiently mapping arbitrary C code with pointers and malloc/free into hardware. Our solution, which fits current memory management methodologies, instantiates an application-specific hardware memory allocator coupled with a memory architecture. Our work also supports the resolution of pointers without restriction on the data structures. We present an implementation based on the SUIF framework along with case studies such as the realization of a video filter and an ATM segmentation engine     

Models for motion-based video indexing and retrieval

With the rapid proliferation of multimedia applications that require video data management, it is becoming more desirable to provide proper video data indexing techniques capable of representing the rich semantics in video data. In real-time applications, the need for efficient query processing is another reason for the use of such techniques. We present models that use the object motion information in order to characterize the events to allow subsequent retrieval. Algorithms for different spatiotemporal search cases in terms of spatial and temporal translation and scale invariance have been developed using various signal and image processing techniques. We have developed a prototype video search engine, PICTURESQUE (pictorial information and content transformation unified retrieval engine for spatiotemporal queries) to verify the proposed methods. Development of such technology will enable true multimedia search engines that will enable indexing and searching of the digital video data based on its true content.     

Parallel Architecture Core (PAC)—the First Multicore Application Processor SoC in Taiwan Part I: Hardware Architecture &; Software Development Tools

In order to develop a low-power and high-performance SoC platform for multimedia applications, the Parallel Architecture Core (PAC) project was initiated in Taiwan in 2003. A VLIW digital signal processor (PACDSP) has been developed from a proprietary instruction set with multimedia-rich instructions, a complexity-effective microarchitecture with an innovative distributed & ping-pong register organization and variable-length VLIW encoding, to a highly-configurable soft IP with several successful silicon implementations. A complete toolchain with an optimizing C compiler has also been developed for PACDSP. A dual-core PAC SoC has been designed and fabricated, which consists of a PACDSP core, an ARM9 core, scratchpad memories, and various on-chip peripherals, to demonstrate the outstanding performance and energy efficiency for multimedia processing such as the real-time H.264 codec. The first part of the two introductory papers of PAC describes the hardware architecture of the PACDSP core, its software development tools, and the PAC SoC with dynamic voltage and frequency scaling (DVFS).     

ViewStation applications: implications for network traffic

This paper describes applications built on the ViewStation, a distributed multimedia system based on Unix workstations and a gigabit per second local area network. A key tenet of the ViewStation project is the delivery of media data not just to the desktop but all the way to the application program. As processing power continues to improve, our approach enables applications that perform intensive processing of audio and video data. We hypothesize that as media data are shaped by this software-based processing, the resultant network traffic patterns will be dominated more by software behavior than by so-called real-time issues. We have written applications that directly process live video to provide more responsive human-computer interaction. We have also developed applications to explore the potential of media processing to support content-based retrieval of prerecorded television broadcasts. These applications perform intelligent processing on video, as well as straightforward presentation. They demonstrate the utility of network-based multimedia systems that deliver audio and video data all the way to the application. The network requirements of the applications are modeled as a combination of bursty transfers and periodic packet-trains     

3级架构多媒体共享平台的设计及实现

提出了一种3级架构多媒体共享平台系统设计与实现方法。视频多媒体共享平台将是公安或民用系统的通用工作平台。设计与实现一个具有灵活的视音频处理和提取、实现快速查询、提供及时的信息获取、实现按需分配的信息传送、智能化的信息处理和运用、供多级用户共同访问的安全的信息平台。重点介绍了系统的主要技术指标、硬件构成和软件功能组成,研究了系统软件部分所涉及的视频采集压缩、数字视频压缩编码标准、流媒体管理、数据传输以及数据存储等技术。该系统已应用于广场视频监控。     

Multimedia friendly home wireless LANs

Home wireless local area networks (LANs) will carry diverse multimedia applications such as data, video, voice, and time sensitive control information. To enable quality delivery of such applications, these networks should incorporate an efficient quality of service (QoS) support mechanism. However, existing home wireless LANs do not provide support for multimedia applications. In this paper, we introduce our software based solution that provides QoS support for voice, video and data while using existing wireless home networks. The experimental results we have obtained provide evidence that our solution provides QoS support for multimedia applications.     

基于SDR SDRAM的视频数据逻辑分析存储器的设计与实现

针对传统硬件测试软件的弊端,文章提出一种便携式视频数据逻辑分析存储器的设计方法,这种分析存储器能够针对网络多媒体数据进行采集、分析和存储等操作。在不影响网络正常传输的前提下,针对TS流进行采集、存储和分析。由于多媒体数据网络传输具有数据量大、网络带宽占用量大等特点,因此设计中根据系统结构需要,设计SDR SDRAM存储器控制器,采用分层状态机的设计模式,可以对结构较大的网络数据包或是大量高速传输中的数据进行突发存储操作。测试结果表明系统工作稳定,数据处理速度快,精度可达纳秒级,满足系统设计要求。     

基于Hi3516的Web视频监控系统的设计与实现

随着嵌入式技术应用于各个领域,越来越多的信息化产品需要接入Internet。文中基于数字多媒体处理器 Hi3516开发板,利用其软硬件资源,移植Boa,搭建视频网络服务器,不仅可以通过网络监听音视频,而且可以修改相应的音视频参数,使嵌入式设备的易用性大幅提高,方便快捷便于管理。     

多媒体通信综述

本文在简述了语音、数据及视频等信息处理技术和传输、组网等网络技术的基础上,着重介绍了多媒体通信业务的应用,主要包括;视频电子信箱,远程医疗诊断、计算机辅助设计,数字网络图书馆,多媒体会议系统,高清晰度图像系统和视频点播等。文章最后还简单介绍了国内外多媒体通信业务发展的情况。     

MPEG2解码系统集成芯片音频处理存储优化策略

在多媒体系统的系统集成芯片(SoC)中,从系统集成芯片工作实时性要求,应用程序和数据尽可能存放在片上存储或Cache,执行方便,处理速度快,就要使用大量的存储部件,使得存储部件的面积和功耗占到整个芯片的很大部分.为了减少片上存储部件,则部分程序和数据移到片外存储,在执行时轮流调进到芯片内,势必增加I/O的开销.因此如何使设计优化是软硬件协同设计中的一个问题.本文以MPEG2集成解码芯片中音频存储优化为例给出了系统集成芯片存储优化的一些方法.包括通过LGDFG(Large Grain Data Flow Graph)模型分析改变程序结构,共享数据空间,改变数据类型以及添加片上SRAM并减少片上Cache容量从而减少系统存储消耗等.这些方法显著地减少系统的存储消耗,降低系统芯片的面积和功耗.     

Platform-independent MB-based AVS video standard implementation

AVS1-P2 is the newest video standard of Audio Video coding Standard (AVS) workgroup of China, which provides close performance to H.264/AVC main profile with lower complexity. In this paper, a platform-independent software package with macroblock-based (MB-based) architecture is proposed to facilitate AVS video standard implementation on embedded system. Compared with the frame-based architecture, which is commonly utilized for PC platform oriented video applications, the MB-based decoder performs all of the decoding processes, except the high-level syntax parsing, in a set of MB-based buffers with adequate size for saving the information of the current MB and the neighboring reference MBs to minimize the on-chip memory and to save the time consumed in on-chip/off-chip data transfer. By modifying the data flow and decoding hierarchy, simulating the data transfer between the on-chip memory and the off-chip memory, and modularizing the buffer definition and management for low-level decoding kernels, the MB-based system architecture provides over 80% reduction in on-chip memory compared to the frame-based architecture when decoding 720p sequences. The storage complexity is also analyzed by referencing the performance evaluation of the MB-based decoder. The MB-based decoder implementation provides an efficient reference to facilitate development of AVS applications on embedded system. The complexity analysis provides rough storage complexity requirements for AVS video standard implementation and optimization.     

A selective encryption and energy efficient clustering scheme for video streaming in wireless sensor networks

The past few years have witnessed increased interest in the potential use of wireless sensor networks (WSNs) in applications such as battlefield assistance, adversary intrusion detection, distributed signal processing, etc. In these applications, multimedia semantic information and video content are extremely sensitive to malicious attacks, including eavesdropping/intercepting of ongoing traffic, and manipulating/counterfeiting of the existing media flows. From a security standpoint, it is very important to transmit authentic and accurate data to surrounding sensor nodes and to the sink. However, securing sensor networks poses unique challenges because these tiny wireless devices are deployed in large numbers, usually in unattended environments, and are severely limited in their capabilities and resources like power, computational capacity, and memory. In this work, a secure and selective encryption framework is proposed so as to optimize network lifetime and video distortion for an energy constrained wireless video sensor network (WVSN).     

Smart Phased Array SoCs: A Novel Application for Advanced SiGe HBT BiCMOS Technology

As BiCMOS IC technology continues to advance in scaling and performance, new applications are continually enabled. One such concept is a smart phased array system on a chip (SoC). The combination of high-performance SiGe heterojunction bipolar transistor (HBT) bipolar devices, well-characterized RF/analog passive components, and dense CMOS digital technology provides the capability to create large multielement, electronically tunable phased arrays with onboard processing intelligence, inside a single die. This SoC will have superior characteristics of lower cost, weight, and size as compared to the large multichip, multitechnology, and multipackage systems in deployment today. Furthermore, using reconfigurable logic and embedded memory, this SoC has the advantage of dynamic software and digital signal processing engine updates, without expensive redesigns of the chip. This publication will describe the necessary ingredients to create such an SoC as well as relevant applications of smart phased arrays that require an SiGe HBT BiCMOS technology. Potential markets for this technology include communications systems, weather tracking, radio astronomy, automotive radar, cellular basestation capacity improvement, satellite and aerial resource imaging, ground-level airplane collision avoidance, as well as military tracking and guidance systems.     

EDMA数据传输方式在基于DSP的视频信号处理系统中的应用

视频信号传输是视频信号处理系统的关键技术,文章介绍利用EDMA数据传输方式、在基于TI公司的数字信号处理器(DSP)TMS320C6713芯片的视频信号处理系统中实现视频数据信号高速传输的一种方案。介绍了方案中的DSP外部储存器接口与FIFO存储器的硬件接口设计,并着重描述了基于EDMA数据传输方式的实现方法和软件设计流程。利用EDMA在CPU后台高效地实现存储空间的数据搬移,并在EDMA中断服务程序中对视频数据信号进行处理,DSP的中央处理单元就能够专注于信号处理和系统功能控制,从而满足视频信号处理系统的高速实时性要求。     

用于视频监控的便携式数字录像机设计

针对各类视频监控和记录的应用特点和要求,设计并研制了一个基于嵌入式微处理器、专用视频压缩芯片及大容量IDE硬盘或电子盘的便携式数字录像机.讨论了系统结构设计及各功能模块的电路选择,介绍了关键接口电路设计和系统软件设计.系统采用单一直流电源供电,为了方便视频压缩数据的存取,设计了专用的文件系统.测试结果表明,系统性能良好、功能灵活,功耗低、集成度高、独立、便携,非常适合各类视频监控和记录应用.     

基于ARM7平台的多媒体监控系统研究

随着数字视频压缩技术的不断发展,基于数字技术的多媒体监控系统开始逐步替代传统的图像监控系统。基于多媒体监控系统采用数字技术对信号进行采样和压缩,所以它可以实现网络传输,而不用采用传统的视频电缆或者光纤传输,大大节省了成本,同时更容易实现对图像的存储、处理和控制。论文采用嵌入式操作系统,在ARM7平台上实现多媒体的监控。