一种基于基本语义单元合成Petri网的足球视频查询描述模型

足球是最具世界性的体育运动之一，球迷遍布五大洲，因此在体育视频节目中足球备受广大观众青睐．在分析了足球视频特点的基础上，提出了一种基于基本语义单元合成Petri网的足球视频查询描述模型．该模型首先定义了一种类似文本字词集合的足球视频基本语义单元集合，在此基础上采用基本语义单元合成Petri网模型建立了一种足球查询语义的描述模型，并分别构建了进球、进攻、角球、犯规、换人等足球语义．初步的实验结果验证了该模型的有效性，并能推广至球类视频和其他体育视频．     

基于内容的通用视频检索系统框架设计

设计了一个支持所有种类的基于内容视频检索方式的通用视频检索系统框架，提出了设计通用系统框架关键问题的解决方案：基于内容的视频分析和检索的组件设计；遵循MPEG7标准的内容索引表示；Web服务体系结构和技术。通用系统框架具有分布、互操作、开放、可扩展的优点。     

基于领域本体的新闻视频检索

新闻视频作为视频数据中有代表性的一种媒体,受到人们的广泛关注,对新闻视频的检索要求也越来越高.传统的新闻视频检索大多是非语义层面的,采用的是基于关键词的检索方法，难于获得令人满意的查准率和查全率．本文提出一种基于领域本体的新闻视频检索框架，定义了新闻视频检索中的新闻视频对象，使用语义表达能力强的领域本体来指导视频语义对象的标注，并针对“一词多义”问题提出了“概念域-概念”两阶段概念消歧算法；针对自然语言检索问题，使用领域本体进行查询优化和查询扩展，并提出了查询语句自动生成方法．实验表明，基于领域本体的新闻视频检索方法可以有效的提高检索性能．     

视频结构挖掘的概念和框架

视频信息处理的关键是视频信息的结构化，视频除了有基本层次结构之外，还有隐藏其中的视频结构语法和结构语义。该文提出了一种视频结构挖掘的概念框架和视频结构挖掘的系统框架，在概念框架中对视频结构挖掘相关概念给出了明确定义和界定；视频结构挖掘框架主要包括：视频基本层次结构挖掘，视频结构语法挖掘和视频结构语义挖掘。讨论了视频结构模式和知识的具体应用，包括指导视频的组织与管理、实现基于内容的个性视频推荐和改善视频摘要系统。     

基于多特征距离学习的视频分类

视频分类在视频检索、内容分析等应用领域具有十分重要的意义。多模态视频特征,如音频、静态图像及视频动作特征等都已经应用于视频分类中,因此如何对多种视频特征进行最佳组合来改善视频分类的性能成为了一个重要研究课题。提出一种基于L1正则化的距离学习方法,对利用多种特征组合提高视频语义标注性能的问题进行研究。由于引入一阶范数正则项,使得模型拥有选取多种视频特征进行最优组合的能力。该方法在通用的Columbia Consumer Video(CCV)视频数据集上显著提高了视频分类的性能。     

基于时序概率超图模型的视频多语义标注

在基于语义的视频检索系统中,为了弥补视频底层特征与高层用户需求之间的差异,提出了时序概率超图模型。它将时间序列因素融入到模型的构建中,在此基础上提出了一种基于时序概率超图模型的视频多语义标注框架（TPH-VMLAF）。该框架结合视频时间相关性,通过使用基于时序概率超图的镜头多标签半监督分类学习算法对视频镜头进行多语义标注。标注过程中同时解决了已标注视频数据不足和多语义标注的问题。实验结果表明,该框架提高了标注的精确度,表现出了良好的性能。     

基于语义匹配的交互式视频检索框架

近年来基于内容的视频检索技术受到人们越来越多的关注. 本文提出了一套基于语义匹配的交互式视频检索框架, 其贡献主要为以下三方面: 1)定义新型的视频高层特征---语义直方图用以描述视频的高层语义信息; 2)使用主导集聚类算法建立基于非监督学习的检索机制, 用以降低在线计算复杂度和提高检索效率; 3)提出新型的相关反馈机制---基于语义的分支反馈, 该机制采用分支反馈结构和分支更新策略实现检索性能的提升. 实验结果表明了本框架的有效性.     

一种基于视频结构的场景分割方法

场景是视频结构中的语义单元。因此基于场景的视频分割,将会对视频的内容分析、检索和浏览提供有益的帮助。提出了一种新的场景分割算法,它利用流形学习理论获得视频的结构特征,然后用马尔科夫链蒙特卡罗方法(Markov chain Monte Carlo,MCMC)动态地进行模拟采样,寻找场景边界的最大后验概率分布,完成场景的自动分割。通过发掘视频结构的内在特征并考虑它的局部信息,使算法能够适合不同类型的视频数据。实验结果也证明了该方法的有效性。     

基于故事单元的长视频节目的组织和浏览技术

视频媒体是序列化、线性化的图象序列，为了使用户浏览查询视频的操作简化，在视频节目中引入非线性、非序列化的方法是必要的，对长视频节目更是如此。对于大多数视频节目，这种方法可通过定义既反映视频内容又反映视频对象组织情况的故事结构来实现．为了能自动分析视频，提取故事结构和故事单元，本文介绍了一种新的视频分析框架；并总结出了其相关技术及具体操作步骤。这种技术有助于表现一定地点和事件的故事单元的提取，这些是仅仅通过镜头边界检测得不到的。自动分析是在MPEG视频上进行的，无需对视频内容有先验知识，分析的结果是故事情节的简明概述表示，而且允许按视频内容进行分层组织。     

An analytical framework for event mining in video data

With the high availability of digital video contents on the internet, users need more assistance to access digital videos. Various researches have been done about video summarization and semantic video analysis to help to satisfy these needs. These works are developing condensed versions of a full length video stream through the identification of the most important and pertinent content within the stream. Most of the existing works in these areas are mainly focused on event mining. Event mining from video streams improves the accessibility and reusability of large media collections, and it has been an active area of research with notable recent progress. Event mining includes a wide range of multimedia domains such as surveillance, meetings, broadcast, news, sports, documentary, and films, as well as personal and online media collections. Due to the variety and plenty of Event mining techniques, in this paper we suggest an analytical framework to classify event mining techniques and to evaluate them based on important functional measures. This framework could lead to empirical and technical comparison of event mining methods and development of more efficient structures at future.     

Automatic composition of broadcast sports video

This study examines an automatic broadcast soccer video composition system. The research is important as the ability to automatically compose broadcast sports video will not only improve broadcast video generation efficiency, but also provides the possibility to customize sports video broadcasting. We present a novel approach to the two major issues required in the system’s implementation, specifically the camera view selection/switching module and the automatic replay generation module. In our implementation, we use multi-modal framework to perform video content analysis, event and event boundary detection from the raw unedited main/sub-camera captures. This framework explores the possible cues using mid-level representations to bridge the gap between low-level features and high-level semantics. The video content analysis results are utilized for camera view selection/switching in the generated video composition, and the event detection results and mid-level representations are used to generate replays which are automatically inserted into the broadcast soccer video. Our experimental results are promising and found to be comparable to those generated by broadcast professionals.     

A framework for flexible summarization of racquet sports video using multiple modalities

While most existing sports video research focuses on detecting event from soccer and baseball etc., little work has been contributed to flexible content summarization on racquet sports video, e.g. tennis, table tennis etc. By taking advantages of the periodicity of video shot content and audio keywords in the racquet sports video, we propose a novel flexible video content summarization framework. Our approach combines the structure event detection method with the highlight ranking algorithm. Firstly, unsupervised shot clustering and supervised audio classification are performed to obtain the visual and audio mid-level patterns respectively. Then, a temporal voting scheme for structure event detection is proposed by utilizing the correspondence between audio and video content. Finally, by using the affective features extracted from the detected events, a linear highlight model is adopted to rank the detected events in terms of their exciting degrees. Experimental results show that the proposed approach is effective.     

Using Webcast Text for Semantic Event Detection in Broadcast Sports Video

Sports video semantic event detection is essential for sports video summarization and retrieval. Extensive research efforts have been devoted to this area in recent years. However, the existing sports video event detection approaches heavily rely on either video content itself, which face the difficulty of high-level semantic information extraction from video content using computer vision and image processing techniques, or manually generated video ontology, which is domain specific and difficult to be automatically aligned with the video content. In this paper, we present a novel approach for sports video semantic event detection based on analysis and alignment of webcast text and broadcast video. Webcast text is a text broadcast channel for sports game which is co-produced with the broadcast video and is easily obtained from the web. We first analyze webcast text to cluster and detect text events in an unsupervised way using probabilistic latent semantic analysis (pLSA). Based on the detected text event and video structure analysis, we employ a conditional random field model (CRFM) to align text event and video event by detecting event moment and event boundary in the video. Incorporation of webcast text into sports video analysis significantly facilitates sports video semantic event detection. We conducted experiments on 33 hours of soccer and basketball games for webcast analysis, broadcast video analysis and text/video semantic alignment. The results are encouraging and compared with the manually labeled ground truth.      

Video Semantic Event/Concept Detection Using a Subspace-Based Multimedia Data Mining Framework

In this paper, a subspace-based multimedia data mining framework is proposed for video semantic analysis, specifically video event/concept detection, by addressing two basic issues, i.e., semantic gap and rare event/concept detection. The proposed framework achieves full automation via multimodal content analysis and intelligent integration of distance-based and rule-based data mining techniques. The content analysis process facilitates the comprehensive video analysis by extracting low-level and middle-level features from audio/visual channels. The integrated data mining techniques effectively address these two basic issues by alleviating the class imbalance issue along the process and by reconstructing and refining the feature dimension automatically. The promising experimental performance on goal/corner event detection and sports/commercials/building concepts extraction from soccer videos and TRECVID news collections demonstrates the effectiveness of the proposed framework. Furthermore, its unique domain-free characteristic indicates the great potential of extending the proposed multimedia data mining framework to a wide range of different application domains.     

A Novel Framework for Semantic Annotation and Personalized Retrieval of Sports Video

Sports video annotation is important for sports video semantic analysis such as event detection and personalization. In this paper, we propose a novel approach for sports video semantic annotation and personalized retrieval. Different from the state of the art sports video analysis methods which heavily rely on audio/visual features, the proposed approach incorporates web-casting text into sports video analysis. Compared with previous approaches, the contributions of our approach include the following. 1) The event detection accuracy is significantly improved due to the incorporation of web-casting text analysis. 2) The proposed approach is able to detect exact event boundary and extract event semantics that are very difficult or impossible to be handled by previous approaches. 3) The proposed method is able to create personalized summary from both general and specific point of view related to particular game, event, player or team according to user's preference. We present the framework of our approach and details of text analysis, video analysis, text/video alignment, and personalized retrieval. The experimental results on event boundary detection in sports video are encouraging and comparable to the manually selected events. The evaluation on personalized retrieval is effective in helping meet users' expectations.     

Automatic analysis of complex athlete techniques in broadcast taekwondo video

Athlete detection and action recognition in sports video is a very challenging task due to the dynamic and cluttered background. Several attempts for automatic analysis focus on athletes in many sports videos have been made. However, taekwondo video analysis remains an unstudied field. In light of this, a novel framework for automatic techniques analysis in broadcast taekwondo video is proposed in this paper. For an input video, in the first stage, athlete tracking and body segmentation are done through a modified Structure Preserving Object Tracker. In the second stage, the de-noised frames which completely contain the body of analyzed athlete from video sequence, are trained by a deep learning network PCANet to predict the athlete action of each single frame. As one technique is composed of many consecutive actions and each action corresponds a video frame, focusing on video sequences to achieve techniques analysis makes sense. In the last stage, linear SVM is used with the predicted action frames to get a techniques classifier. To evaluate the performance of the proposed framework, extensive experiments on real broadcast taekwondo video dataset are provided. The results show that the proposed method achieves state-of-the-art results for complex techniques analysis in taekwondo video.

     

Semantic and structural analysis of TV diving programs

Automatic content analysis of sports videos is a valuable and challenging task. Motivated by analogies between a class of sports videos and languages, the authors propose a novel approach for sports video analysis based on compiler principles. It integrates both semantic analysis and syntactic analysis to automatically create an index and a table of contents for a sports video. Each shot of the video sequence is first annotated and indexed with semantic labels through detection of events using domain knowledge. A grammar-based parser is then constructed to identify the tree structure of the video content based on the labels. Meanwhile, the grammar can be used to detect and recover errors during the analysis. As a case study, a sports video parsing system is presented in the particular domain of diving. Experimental results indicate the proposed approach is effective.     

Generation of Personalized Music Sports Video Using Multimodal Cues

In this paper, we propose a novel automatic approach for personalized music sports video generation. Two research challenges are addressed, specifically the semantic sports video content extraction and the automatic music video composition. For the first challenge, we propose to use multimodal (audio, video, and text) feature analysis and alignment to detect the semantics of events in broadcast sports video. For the second challenge, we introduce the video-centric and music-centric music video composition schemes and proposed a dynamic-programming based algorithm to perform fully or semi-automatic generation of personalized music sports video. The experimental results and user evaluations are promising and show that our systems generated music sports video is comparable to professionally generated ones. Our proposed system greatly facilitates the music sports video editing task for both professionals and amateurs