基于多模式分析自动解析新闻视频

提出一种结合视觉、声音、文字等多种模式信息自动解析新闻视频的方法,并对音频特征的提取以及综合多种模式信息解析新闻视频的算法进行了详细的探讨.多种模式信息的使用有效地弥补了仅基于图像分析技术分割新闻条目的不足,从而使该方法对不同方式存在的新闻条目在分割时具有更广泛的适应性.在包含184100帧的测试数据集上,对于新闻条目边界点的检测,系统获得了95.1%查全率,93.3%的正确率.实验结果证明了该方法的有效性、强壮性.

Automatic Parsing of News Video Using Multimodal Analysis

The paper presents an approach, which exploits multimodal information (video, audio and text) to automatically parse news video. In the paper, audio features extraction, as well as multimodal information integration scheme, are addressed in detail. Integration of multiple information sources can overcome the weakness of the approach only exploiting the image analysis techniques. That makes our approach have wider adaptation to variable existence situations of news items. On test data with 184 100 frames, when the system detects boundaries between news items, the recall 95.1% and the accuracy 93.3% are obtained. The experiment results show the approach is valid and robust.