Models for motion-based video indexing and retrieval

With the rapid proliferation of multimedia applications that require video data management, it is becoming more desirable to provide proper video data indexing techniques capable of representing the rich semantics in video data. In real-time applications, the need for efficient query processing is another reason for the use of such techniques. We present models that use the object motion information in order to characterize the events to allow subsequent retrieval. Algorithms for different spatiotemporal search cases in terms of spatial and temporal translation and scale invariance have been developed using various signal and image processing techniques. We have developed a prototype video search engine, PICTURESQUE (pictorial information and content transformation unified retrieval engine for spatiotemporal queries) to verify the proposed methods. Development of such technology will enable true multimedia search engines that will enable indexing and searching of the digital video data based on its true content.     

Audio Feature Extraction and Analysis for Scene Segmentation and Classification

Understanding of the scene content of a video sequence is very important for content-based indexing and retrieval of multimedia databases. Research in this area in the past several years has focused on the use of speech recognition and image analysis techniques. As a complimentary effort to the prior work, we have focused on using the associated audio information (mainly the nonspeech portion) for video scene analysis. As an example, we consider the problem of discriminating five types of TV programs, namely commercials, basketball games, football games, news reports, and weather forecasts. A set of low-level audio features are proposed for characterizing semantic contents of short audio clips. The linear separability of different classes under the proposed feature space is examined using a clustering analysis. The effective features are identified by evaluating the intracluster and intercluster scattering matrices of the feature space. Using these features, a neural net classifier was successful in separating the above five types of TV programs. By evaluating the changes between the feature vectors of adjacent clips, we also can identify scene breaks in an audio sequence quite accurately. These results demonstrate the capability of the proposed audio features for characterizing the semantic content of an audio sequence.     

Concept-oriented indexing of video databases: toward semantic sensitive retrieval and browsing

Digital video now plays an important role in medical education, health care, telemedicine and other medical applications. Several content-based video retrieval (CBVR) systems have been proposed in the past, but they still suffer from the following challenging problems: semantic gap, semantic video concept modeling, semantic video classification, and concept-oriented video database indexing and access. In this paper, we propose a novel framework to make some advances toward the final goal to solve these problems. Specifically, the framework includes: 1) a semantic-sensitive video content representation framework by using principal video shots to enhance the quality of features; 2) semantic video concept interpretation by using flexible mixture model to bridge the semantic gap; 3) a novel semantic video-classifier training framework by integrating feature selection, parameter estimation, and model selection seamlessly in a single algorithm; and 4) a concept-oriented video database organization technique through a certain domain-dependent concept hierarchy to enable semantic-sensitive video retrieval and browsing.     

智能媒资检索系统设计与实现*

针对目前媒资管理系统依赖编目信息进行检索而出现的编目信息难以覆盖媒资数据的所有语义内容、由于人的理解不同而导致的编目信息不一致、媒资编目费力费时等问题,设计了不依赖编目信息的基于全文检索、语音识别、人脸识别、关键帧提取等的智能媒资检索系统,对媒资内容自动分析、媒资特征索引、媒资特征检索进行了阐述,并采用基于B/S的分布式架构进行了实现.结果证明,该方案设计具有较高的可靠性和稳定性,在媒资管理中得到了良好的应用.     

A New Learning Algorithm for the Fusion of Adaptive Audio–Visual Features for the Retrieval and Classification of Movie Clips

This paper presents a new learning algorithm for audiovisual fusion and demonstrates its application to video classification for film database. The proposed system utilized perceptual features for content characterization of movie clips. These features are extracted from different modalities and fused through a machine learning process. More specifically, in order to capture the spatio-temporal information, an adaptive video indexing is adopted to extract visual feature, and the statistical model based on Laplacian mixture are utilized to extract audio feature. These features are fused at the late fusion stage and input to a support vector machine (SVM) to learn semantic concepts from a given video database. Based on our experimental results, the proposed system implementing the SVM-based fusion technique achieves high classification accuracy when applied to a large volume database containing Hollywood movies.     

Reference model of future ubiquitous convergent network and context-aware telecommunication service platform

1 Introduction Future ubiquitous convergent network is a user-centric harmony communication network. It integrates various heterogeneous networks (i.e. mobile/fixed network, Internet, and some emerging networks) and new technologies [1]. The object of ubi…     

The holy grail of content-based media analysis

Tools and systems for content-based access to multimedia and-image., video, audio, graphics, text, and any number of combinations-has increased in the last decade. We've seen a common theme of developing automatic analysis techniques for deriving metadata (data describing information in the content at both syntactic and semantic levels). Such metadata facilitates developing innovative tools and systems for multimedia information retrieval, summarization, delivery, and manipulation. Many interesting demonstrations of potential applications and services have emerged-finding images visually similar to a chosen picture (or sketch); summarizing videos with thumbnails of keyframes; finding video clips of a specific event, story, or person; and producing a two-minute skim of an hour-long program. In order to evaluate content-based research methodologies, the article considers intended users and whether alternative solutions exist and areas of research     

基于嵌入式零树小波编码直方图图像检索

图像和视频应用的快速增长，使得根据图像和视频内容进行查询的技术变得越来越重要，人们提出了许多基于像素域或压缩域的图像检索技术，因为多媒体数据库通常具有相当大的数据量，所以基于像素域图像检索技术的计算复杂度相当大，因此，许多文献提出更快的基于压缩域的图像检索技术，本文提出一种改进的基于嵌入式零树小波编码直方图的图像检索技术，特征提取综合考虑图像的颜色，纹理，频率和空间信息，所有的特征可以在压缩过程中自动得到，图像检索的过程就是匹配待检索图像和来自数据库的侯选图像的索引，实验证明这种方法具有好的检索性能。     

一种基于加速近端梯度法的视频散列算法研究

散列算法已经被广泛应用于视频数据的索引。然而,当前大多数视频散列方法将视频看成是多个独立帧的简单集合,通过综合帧的索引来对每个视频编制索引,在设计散列函数时往往忽略了视频的结构信息。首先将视频散列问题建模为结构正规化经验损失的最小化问题。然后提出一种有监管算法,通过利用结构学习方法来设计高效的散列函数。其中,结构正规化利用了出现于视频帧(与相同的语义类别存在关联)中的常见局部视觉模式,同时对来自同一视频的后续帧保持时域一致性。证明了通过使用加速近端梯度(APG)法可有效求解最小化目标问题。最后,基于两个大规模基准数据集展开全面实验(150 000个视频片断,1 200万帧),实验结果证明了该方法性能优于当前其他算法。     

Shape-sequence-based key image generation algorithm for browsingand retrieval of video clips

A new shape-sequence-based algorithm that can effectively generate key images from video clips is introduced. Generated key images can be used as the feature information in the browsing and retrieval of video clips from a multimedia database. Experiments with MPEG-7 data sets were performed, and the results are compared with existing methods     

Generic content-based audio indexing and retrieval framework

Rapid increase in the amount of the digital audio collections presenting various formats, types, durations and other parameters that the digital multimedia world refers demands a generic framework for robust and efficient indexing and retrieval based on the aural content. Moreover, from the content-based multimedia retrieval point of view, the audio information can be even more important than the visual part as it is mostly unique and significantly stable within the entire duration of the content. A generic and robust audio-based multimedia indexing and retrieval framework, which has been developed and tested under the MUVIS system, is presented. This framework supports the dynamic integration of the audio feature extraction modules during the indexing and retrieval phases and therefore provides a test-bed platform for developing robust and efficient aural feature extraction techniques. Furthermore, the proposed framework is designed based on the high-level content classification and segmentation in order to improve the speed and accuracy of the aural retrievals. Both theoretical and experimental results are finally presented, including the comparative measures of retrieval performance with respect to the visual counterpart.     

Content-based indexing and retrieval of visual information

The ease of capturing and encoding digital images has produced a massive amount of visual information online. As a consequence, grand challenges have emerged in the areas of storage, indexing, and retrieval of visual information in large archives. How does one find a photograph from a large archive that contains millions of pictures? How does a CNN video journalist find a specific clip from the myriad of video tapes, ranging from historical to contemporary, from sports to humanities? Efficient, real-time algorithms and systems are needed to address these needs of not only professionals but for users who want to find visual information online     

An automatic algorithm for semantic object generation and temporal tracking

Automatic semantic video object extraction is an important step for providing content-based video coding, indexing and retrieval. However, it is very difficult to design a generic semantic video object extraction technique, which can provide variant semantic video objects by using the same function. Since the presence and absence of persons in an image sequence provide important clues about video content, automatic face detection and human being generation are very attractive for content-based video database applications. For this reason, we propose a novel face detection and semantic human object generation algorithm. The homogeneous image regions with accurate boundaries are first obtained by integrating the results of color edge detection and region growing procedures. The human faces are detected from these homogeneous image regions by using skin color segmentation and facial filters. These detected faces are then used as object seed for semantic human object generation. The correspondences of the detected faces and semantic human objects along time axis are further exploited by a contour-based temporal tracking procedure.     

基于OWL-S的服务发现语义匹配机制

 本文提出一种基于OWL-S的语义web服务匹配系统的机制,将语义元素引入UDDI系统中,从而在UDDI中保存语义信息.系统通过语义服务匹配算法提高web服务匹配的准确度和召回率,并通过数据映射机制保证模型对于当前的服务发现标准UDDI基础架构的兼容以及UDDI标准操作接口的透明性.系统使用本体概念的索引机制提高服务发现的效率.而且,系统在建立和维护索引的过程、或服务的匹配过程中使用近似概念搜寻算法进一步提高本体概念搜寻和服务匹配的效率.     

Long transition analysis for digital video sequences

Video indexing is a technique used to extract objects within a video sequence and index them so that they can be used for future retrieval. In many video sequences, special effects such as fade and wipe are incorporated, and in some cases, it is desirable to be able to include objects within such effects for indexing. In this paper, we introduce an automatic process that determines the type of transition and extracts information from it, so that this information can be used in object extraction. Such a process consists of four stages: shot boundary refinement, shot type determination, frame reconstruction for soft transitions, and shot classification for hard transitions. In this paper, we will give the implementation, timing, and performance analysis for each stage. Long transition analysis bridges the gap between shot boundary detection and object tracking and smoothes the process of automatic video indexing for video databases.