视频语义分析两级多模式融合算法

为了全面准确地获取视频高层语义信息,提出了一种基于仿生的视频语义分析两级多模式融合算法。该算法仿照人脑多感觉融合机理,先将视频中多模式特征按不同类别划分为组,然后对每一组中的多模低层特征用层次隐马尔可夫模型(HHMM)进行数据融合;同时将以似然率表示的多个低层融合结果作为高层融合的输入,再通过基于核的非线性算法把输入空间变换到高维特征空间;最后在特征空间中求取最优线性分类面,即可得到最终的多模式两级融合结果。实验表明,该方法不仅能有效融合视频中的多模式特征,而且能获取全面、准确的高层语义信息。

Two Level Multimodal Fusion Algorithm for Semantic Video Analysis

To extract video semantic concepts combing different modalities,a two level multimodal fusion method for video semantic concept analysis is proposed.Multimodal features of video are divided into several groups.The fact that each group of has distinct features,a hierarchical hidden Markov models(HHMM) is constructed for the purpose of first-level fusion.Then outputs of first-level fusion are combined using a kernel function,by which a hyper-plane with better classification for video semantic concept is obtained.The results of experiments comparing to other fusion methods support that the two-level fusion method utilizes different modal feature in semantic concept analysis,and could effectively combine multimodal features.