Video JET: packet loss-resilient video joint encryption and transmission based on media-hash-embedded residual data

Media encryption technologies actively play the first line of defense in securing the access of multimedia data. Traditional cryptographic encryption can achieve provable security but is unfortunately sensitive to a single bit error, which will cause an unreliable packet to be dropped creating packet loss. In order to achieve robust media encryption, the requirement of error resilience can be achieved with error-resilient media transmission. This study proposes a video joint encryption and transmission (video JET) scheme by exploiting media hash-embedded residual data to achieve motion estimation and compensation for recovering lost packets, while maintaining format compliance and cryptographic provable security. Interestingly, since video block hash preserves the condensed content to facilitate search of similar blocks, motion estimation is implicitly performed through robust media hash matching – which is the unique characteristic of our method. We analyze and compare the performance of resilience to (bursty) packet loss between the proposed method and forward error correction (FEC), which has been extensively employed to protect video packets over error-prone networks. The feasibility of our packet loss-resilient video JET approach is further demonstrated through experimental results.

Information access in multimedia databases based on feature models

With the increasing popularity of the WWW, the main challenge in computer science has become content-based retrieval of multimedia objects. Access to multimedia objects in databases has long been limited to the information provided in manually assigned keywords. Now, with the integration of feature-detection algorithms in database systems software, content-based retrieval can be fully integrated with query processing. We describe our experimentation platform under development, making database technology available to multimedia. Our approach is based on the new notion of feature databases. Its architecture fully integrates traditional query processing and content-based retrieval techniques. Arjen P. de Vries, Ph.D.: He received his Ph.D. in Computer Science from the University of Twente in 1999, on the integration of content management in database systems. He is especially interested in the new requirements on the design of database systems to support content-based retrieval in multimedia digital libraries. He has continued to work on multimedia database systems as a postdoc at the CWI in Amsterdam as well as University of Twente. Menzo Windhouwer: He received his MSc in Computer Science and Management from the University of Amsterdam in 1997. Currently he is working in the CWI Database Research Group on his Ph.D., which is concerned with multimedia indexing and retrieval using feature grammars. Peter M.G. Apers, Ph.D.: He is a full professor in the area of databases at the University of Twente, the Netherlands. He obtained his MSc and Ph.D. at the Free University, Amsterdam, and has been a visiting researcher at the University of California, Santa Cruz and Stanford University. His research interests are query optimization in parallel and distributed database systems to support new application domains, such as multimedia applications and WWW. He has served on the program committees of major database conferences: VLDB, SIGMOD, ICDE, EDBT. In 1996 he was the chairman of the EDBT PC. In 2001 he will, for the second time, be the chairman of the European PC of the VLDB. Currently he is coordinating Editor-in-Chief of the VLDB Journal, editor of Data & Knowledge Engineering, and editor of Distributed and Parallel Databases. Martin Kersten, Ph.D.: He received his PhD in Computer Science from the Vrije Universiteit in 1985 on research in database security, whereafter he moved to CWI to establish the Database Research Group. Since 1994 he is professor at the University of Amsterdam. Currently he is heading a department involving 60 researchers in areas covering BDMS architectures, datamining, multimedia information systems, and quantum computing. In 1995 he co-founded Data Distilleries, specialized in data mining technology, and became a non-executive board member of the software company Consultdata Nederland. He has published ca. 130 scientific papers and is member of the editorial board of VLDB journal and Parallel and Distributed Systems. He acts as a reviewer for ESPRIT projects and is a trustee of the VLDB Endowment board.     

Synchronization-Oriented Placement and Retrieval Strategies for Delay-Sensitive Media Streams

Multimedia data include text,image,audio and video,etc.Recent developments and advances in the areas of mass storage technology and high speed networks make it feasible for multimedia computing systems to offer services such as multimedia e-mial,teleconferencing and various vivid games.The storage and retrieval of multimedia data are two of the most important technologies in the futre multimedia systems.This paper analyzes the synchronization requirements of the delay-sensitive media streams,classifies the synchronization hierarchically and proposes several effective strategies for the intra-media and inter-media data placement problems.     

Cross-covariance-based features for speech classification in film audio

As multimedia becomes the dominant form of entertainment through an ever increasing range of digital formats, there has been a growing interest in obtaining information from entertainment media. Speech is one of the core resources in multimedia, providing a foundation for the extraction of semantic information. Thus, detecting speech is a critical first step for speech-based information retrieval systems. This work focuses on speech detection in one of the dominant forms of entertainment media: feature films. A novel approach for voice activity detection (VAD) in film audio is proposed. The approach uses correlation to analyze associations of Mel Frequency Cepstral Coefficient (MFCC) pairs in speech and non-speech data. This information then drives feature selection for the creation of MFCC cross-covariance feature vectors (MFCC-CCs) which are used to train a random forest classifier to solve a binary speech/non-speech classification problem on audio data from entertainment media. The classifier performance is evaluated on a number of test sets and achieves a classification accuracy of up to 94%. The approach is also compared with state of the art and contemporary VAD algorithms, and demonstrates competitive results.     

A comprehensive agent-based architecture for intelligent information retrieval in a distributed heterogeneous environment

The Internet has become the global infrastructure supporting information acquisition and retrieval from many heterogeneous data sources containing high-speed text and rich multimedia images, audio, and video. AgentRAIDER is an ongoing research project at Texas Tech University designed to develop a comprehensive architecture for an intelligent information retrieval system with distributed heterogeneous data sources. The system is designed to support intelligent retrieval and integration of information from the Internet. Current systems of this nature focus only on specific aspects of the distributed heterogeneous problem such as database queries or information filtering. Consequently, these concepts and others have never been successfully integrated into a unified, cohesive architecture. This paper discusses the design and implementation of the AgentRAIDER system and identifies areas for applications of the system in various domains.     

音频信息检索综述

随着多媒体和Internet技术的广泛应用和深入普及，多媒体数据的数量正在飞速增长，音频数据作为多媒体数据的重要组成部分，其信息量也在不断膨胀。如何有效的对音频信息进行检索成为现代信息检索的一个重要的研究领域。然而，音频数据同传统的文本数据不同，文本检索中的相关技术不能简单地应用到音频信息检索中，因为后者往往是基于语义的，所以不可避免地要用到音频数据的特征提取和模式匹配等技术。本文将对音频数据检索相关技术和系统做一综述性的介绍。     

DM642在网络多媒体开发平台中的应用

随着宽带网络的普及和高效音、视频标准的不断推出,利用互联网进行音、视频节目的实时传榆已经成为多媒体技术应用领域一个重要的研究方向。DM642是美国德州仪器公司推出的最新的一款媒体处理器,它为多媒体产品应用提供了理想的解决方案。DM642的性能在已推向市场的各种媒体处理器中位于领先地位,将会广泛应用在网络多媒体开发平台的设计中。     

Content-based retrieval in digital libraries

With the recent developments in multimedia and telecommunication technologies, content-based information is becoming increasingly important for various areas such as digital libraries, interactive video and multimedia publishing. Multimedia data refers to simple structured data (such as numbers and short strings), large unstructured data (such as text documents, images, audio and video data) and complex structured data (such as maps, graphs, charts and tables). In this article, we briefly address content-based retrieval and the issues of representation, storage and retrieval of multimedia objects in digital libraries. We then very briefly identify some open areas of research     

An Interactive Multimedia Diary for the Home

A system for retrieval and summarization of multimedia data from a home-like environment continuously captures video and audio sequences as the inhabitants are moving inside the house. An interactive user interface based on hierarchical media segmentation incorporates user memory and intelligence data. In the long term, this system can provide valuable information for studies related to housing design, evaluating human behavior, and so on.     

多媒体信息检索技术研究应用

在网络和多媒体技术快速发展的今天,如何在众多以图像、音频、视频等信息为代表的多媒体数据中有效的对其进行组织、存取、查询、提取是当前面临的一个重要问题。通过分析目前的多媒体信息检索技术,介绍了用于多媒体内容描述的国际标准MPEG-7;从图像、音频、视频三方面讨论当前广泛使用的一些信息检索技术并对这些技术的实际应用作了分析;最后,给出了一个视频应用模型。     

基于细节层次与最小生成树的三维地形识别与检索

图像、视频、音频和图形等均是多媒体数据流中的信息载体,对上述数据所蕴涵的内容进行分析,可以极大地方便人们对它们的使用与管理.基于内容的图像(视频)和音频检索已经取得了不少进展,但是对于图形,特别是3D图形进行识别与检索的有效方法还很少见.提出了对相似3D物体识别与检索的算法,在这个算法中,首先使用细节层次模型对3D物体进行三角面片约减,然后提取3D物体的特征.由于所提取的特征维数很大,最小生成树(minimum spanning tree,简称MST)被用来对每一个3D物体的特征进行约减,基于约减后的特征,实现了基于支持向量机的3D物体识别与检索方法.这个算法被使用到3D丘陵与山地的地形识别中,取得了良好效果.     

维吾尔语语音检索技术研究

随着大数据时代的到来,各种音频、视频文件日益增多,如何高效地定位关键敏感信息具有非常重要的研究意义。目前研究人员对针对英语和汉语的语音检索技术进行了深入的研究,而针对维吾尔语的语音检索技术还处于起步阶段。该文对维吾尔语语音关键词检索技术进行了研究并采用了大词汇量连续语音识别、利用聚类算法将多候选词图转换为混淆网络、倒排索引、置信度以及相关度的计算等技术和方法,对维吾尔语语音检索系统进行了研究与搭建。最后在测试集上对该系统进行测试,测试结果显示,在语音识别正确率为82.1%的情况下,检索系统的召回率分别达到97.0%和79.1%时,虚警率分别为13.5%和8.5%。     

Efficient data reduction in multimedia data

As the amount of multimedia data is increasing day-by-day thanks to cheaper storage devices and increasing number of information sources, the machine learning algorithms are faced with large-sized datasets. When original data is huge in size small sample sizes are preferred for various applications. This is typically the case for multimedia applications. But using a simple random sample may not obtain satisfactory results because such a sample may not adequately represent the entire data set due to random fluctuations in the sampling process. The difficulty is particularly apparent when small sample sizes are needed. Fortunately the use of a good sampling set for training can improve the final results significantly. In KDD’03 we proposed EASE that outputs a sample based on its ‘closeness’ to the original sample. Reported results show that EASE outperforms simple random sampling (SRS). In this paper we propose EASIER that extends EASE in two ways. (1) EASE is a halving algorithm, i.e., to achieve the required sample ratio it starts from a suitable initial large sample and iteratively halves. EASIER, on the other hand, does away with the repeated halving by directly obtaining the required sample ratio in one iteration. (2) EASE was shown to work on IBM QUEST dataset which is a categorical count data set. EASIER, in addition, is shown to work on continuous data of images and audio features. We have successfully applied EASIER to image classification and audio event identification applications. Experimental results show that EASIER outperforms SRS significantly. Surong Wang received the B.E. and M.E. degree from the School of Information Engineering, University of Science and Technology Beijing, China, in 1999 and 2002 respectively. She is currently studying toward for the Ph.D. degree at the School of Computer Engineering, Nanyang Technological University, Singapore. Her research interests include multimedia data processing, image processing and content-based image retrieval. Manoranjan Dash obtained Ph.D. and M. Sc. (Computer Science) degrees from School of Computing, National University of Singapore. He has worked in academic and research institutes extensively and has published more than 30 research papers (mostly refereed) in various reputable machine learning and data mining journals, conference proceedings, and books. His research interests include machine learning and data mining, and their applications in bioinformatics, image processing, and GPU programming. Before joining School of Computer Engineering (SCE), Nanyang Technological University, Singapore, as Assistant Professor, he worked as a postdoctoral fellow in Northwestern University. He is a member of IEEE and ACM. He has served as program committee member of many conferences and he is in the editorial board of “International journal of Theoretical and Applied Computer Science.” Liang-Tien Chia received the B.S. and Ph.D. degrees from Loughborough University, in 1990 and 1994, respectively. He is an Associate Professor in the School of Computer Engineering, Nanyang Technological University, Singapore. He has recently been appointed as Head, Division of Computer Communications and he also holds the position of Director, Centre for Multimedia and Network Technology. His research interests include image/video processing & coding, multimodal data fusion, multimedia adaptation/transmission and multimedia over the Semantic Web. He has published over 80 research papers.     

多信息融合的新闻节目主题划分方法

对新闻播报节目进行自动主题划分,可以有效地组织和利用新闻播报类数据。目前自动故事单元划分的研究以视频数据为主,音频的语音识别文本中包含丰富的语义信息,同时声音事件的转换也可以提供很多重要信息,能够有效的进行基于语义的主题划分。根据这些信息,该文提出了一种基于规则的多信息融合的方法,利用切分点邻域的音频类型信息来修正使用语义信息的切分结果,完成主题划分。实验表明根据规则进行特征融合后,新闻节目主题划分的F-估值为64.8%,错误概率Pk和WindowDiff分别达到18.3%和24.5%。     

Brahma: Browsing and Retrieval Architecture for Hierarchical Multimedia Annotation

Traditional browsing of large multimedia documents (e.g., video, audio) is primarily sequential. In the absence of an index structure browsing and searching for relevant information in a long video, audio or other multimedia document becomes difficult. Manual annotation can be used to mark various segments of such documents. Different segments can be combined to create new annotated segments, thus creating hierarchical annotation structures. Given the lack of structure in media data, it is natural for different users to have different views on the same media data. Therefore, different users can create different annotation structures. Users may also share some or all of each other's annotation structures. The annotation structure can be browsed or used to playback as a composed video consisting of different segments. Finally, the annotation structures can be manipulated dynamically by different users to alter views on a document. BRAHMA is a multimedia environment for browsing and retrieval of multimedia documents based on such hierarchical annotation structures.     

基于多模态子空间相关性传递的视频语义挖掘

在视频语义信息理解和挖掘中,充分利用图像、音频和文本等多模态媒质之间的交互关联是非常重要的研究方向.考虑到视频的多模态和时序关联共生特性,提出了一种基于多模态子空间相关性传递的语义概念检测方法来挖掘视频的语义信息.该方法对所提取视频镜头的多模态底层特征,根据共生数据嵌入(co-occurrence data embedding)和相似度融合(SimFusion)进行多模态子空间相关性传递而得到镜头之间的相似度关系,接着通过局部不变投影(locality preserving projections)对原始数据进行降维以获得低维语义空间内的坐标,再利用标注信息训练分类模型,从而可对训练集外的测试数据进行语义概念检测,实现视频语义信息挖掘.实验表明该方法有较高的准确率.     

校园网络教学媒体同步直播系统的设计与实现

介绍了校园网络教学媒体同步直播系统的设计目标、设计方法和实现步骤。采用先进的流媒体传输技术,基于跨平台Web数据直播技术,实时采集直播现场的音视频信号,并通过IP网络实时地将这些现场信息直播出去。通过基于Web的同步媒体技术,把视频/音频信号与课件数据完整同步地集成在一起,实现远程多媒体同步直播和同步录制,生成完整的多媒体课件。