人体运动视频关键帧优化及行为识别

在行为识别过程中,提取视频关键帧可以有效减少视频索引的数据量,从而提高 动作识别的准确性和实时性。为提高关键帧的代表性,提出一种关键帧序列优化方法,并在此 基础上进行行为识别。首先根据3D 人体骨架特征利用K-均值聚类算法提取人体运动视频序列 中的关键帧,然后根据关键帧所在序列中的位置进行二次优化以提取最优关键帧,解决了传统 方法中关键帧序列冗余等问题。最后根据最优关键帧利用卷积神经网络(CNN)分类器对行为视 频进行识别。在Florence3D-Action 数据库上的实验结果表明,该方法具有较高的识别率,并且 与传统方法相比大幅度缩短了识别时间。     

基于分级关键帧筛选的RGB-D SLAM算法研究

利用关键帧求解SLAM算法(simultaneous localization and mapping)能够提高SLAM系统的实时性与精确度。针对现存关键帧筛选算法中存在的计算复杂度高、图像帧冗余以及鲁棒性较差等问题,提出一种分级关键帧筛选方法。该算法考虑了SLAM系统在不同运行阶段时对关键帧的要求,首先结合旋转度指数与地图点跟踪筛选出一级关键帧用于后端优化与回环检测,再利用相邻帧在空间上的相对运动距离筛选出二级关键帧用于三维地图构建,最后,实现了基于此二级筛选算法的RGB-D SLAM系统。实验表明,一级关键帧算法能提高SLAM系统的定位和建图精度,二级关键帧算法则有效减少了数据冗余,提高了建图效率。     

基于关键帧序列的视频片段检索

提出了一种基于关键帧融合的视频片段检索方法。使用特征联合分布直方图将视频分割为子镜头,子镜头用关键帧表示。检索时,对范例视频片段的每个关键帧检索到相似的关键帧,所有的相似关键帧按照时间连续性融合为视频片段。提出一种快速的视频片段相似度计算模型。实验表明,本文的方法快速有效。     

基于层次约束的2维动画关键帧插值算法

2D动画制作是一个劳动密集型的过程,如何自动进行关键帧插值是研究的难点。关键帧插值的核心环节是2D形体渐变,现有2D形体渐变算法大多针对闭合的单一多边形,在应用到包含多个区域的复杂形体时,可能出现区域之间的重叠。本文算法获取输入关键帧的封闭区域,并迭代计算区域间的匹配关系;对每个子区域建立相对于父区域的局部坐标系,施加层次间的约束关系进行插值,以消除父子区域插值过程中“运动”的不匹配现象。实验结果表明,该算法合成结果自然光滑,具有一定的实用价值。     

基于向量空间模型的视频语义相关内容挖掘

对海量视频数据库中所蕴涵的语义相关内容进行挖掘分析，是视频摘要生成方法面临的难题。该文提出了一种基于向量空间模型的视频语义相关内容挖掘方法：对新闻视频进行预处理，将视频转化为向量形式的数据集，采用主题关键帧提取算法对视频聚类内容进行挖掘，保留蕴涵场景独特信息的关键帧，去除视频中冗余的内容，这些主题关键帧按原有的时间顺序排列生成视频的摘要。实验结果表明，使用该视频语义相关内容挖掘的算法生成的新闻视频具有良好的压缩率和内容涵盖率。     

基于目标变化的监控视频关键帧提取方法

监控视频关键帧提取技术作为监控视频分析的重要研究内容,能够有效地解决视频数据的高效存储和快速访问等问题。本文提出一种基于目标变化的监控视频关键帧提取方法,分析监控视频帧间的目标变化,并采用局部极大值优化方法将原监控视频划分成视频片段。最后,从每个视频片段中选取特征中心对应视频帧作为关键帧,并依据目标的属性删除冗余的关键帧得到最终的视频关键帧集合。实验结果表明,该方法所提取的视频关键帧冗余性较低,所包含的内容很具有代表性。同时,该方法的复杂度较低,适用于监控视频的关键帧提取工作。     

Fully automatic person segmentation in unconstrained video using spatio-temporal conditional random fields

The segmentation of objects and people in particular is an important problem in computer vision. In this paper, we focus on automatically segmenting a person from challenging video sequences in which we place no constraint on camera viewpoint, camera motion or the movements of a person in the scene. Our approach uses the most confident predictions from a pose detector as a form of anchor or keyframe stick figure prediction which helps guide the segmentation of other more challenging frames in the video. Since even state of the art pose detectors are unreliable on many frames –especially given that we are interested in segmentations with no camera or motion constraints –only the poses or stick figure predictions for frames with the highest confidence in a localized temporal region anchor further processing. The stick figure predictions within confident keyframes are used to extract color, position and optical flow features. Multiple conditional random fields (CRFs) are used to process blocks of video in batches, using a two dimensional CRF for detailed keyframe segmentation as well as 3D CRFs for propagating segmentations to the entire sequence of frames belonging to batches. Location information derived from the pose is also used to refine the results. Importantly, no hand labeled training data is required by our method. We discuss the use of a continuity method that reuses learnt parameters between batches of frames and show how pose predictions can also be improved by our model. We provide an extensive evaluation of our approach, comparing it with a variety of alternative grab cut based methods and a prior state of the art method. We also release our evaluation data to the community to facilitate further experiments. We find that our approach yields state of the art qualitative and quantitative performance compared to prior work and more heuristic alternative approaches.     

面向半稠密三维重建的改进单目ORB-SLAM

构建更详细的地图以及估计更精准的相机位姿一直都是同时定位与地图构建（Simultaneous Localization And Mapping,SLAM）技术所追求的目标,但是以上目标与实时性要求、较低的计算代价和受限的计算资源条件是相矛盾的。提出一种在单目ORB-SLAM（Oriented FAST and Rotated BRIEF-SLAM）方法的基础上利用关键帧中提取到的直线特征进行半稠密三维重建的方法。由ORB-SLAM实时提供一组关键帧及其对应的相机位姿信息和一系列地图点,提出一种关键帧再剔除算法进一步减少冗余帧数目,使用直线段提取方法提取各帧中的直线段,使用纯几何约束方法对以上检测得到的直线段进行匹配,生成一个由直线段构成的半稠密三维场景模型。实验结果表明新方法持续稳定的运行,能在低计算代价条件下快速地在线三维重建。     

Knowledge-based image retrieval with spatial and temporalconstructs

A knowledge-based approach to retrieve medical images by feature and content with spatial and temporal constructs is developed. Selected objects of interest in an image are segmented and contours are generated. Features and content are extracted and stored in a database. Knowledge about image features can be expressed as a type abstraction hierarchy (TAH), the high-level nodes of which represent the most general concepts. Traversing TAH nodes allows approximate matching by feature and content if an exact match is not available. TAHs can be generated automatically by clustering algorithms based on feature values in the databases and hence are scalable to large collections of image features. Since TAHs are generated based on user classes and applications, they are context- and user-sensitive. A knowledge-based semantic image model is proposed to represent the various aspects of an image object's characteristics. The model provides a mechanism for accessing and processing spatial, evolutionary and temporal queries. A knowledge-based spatial temporal query language (KSTL) has been developed that extends ODMG's OQL and supports approximate matching of features and content, conceptual terms and temporal logic predicates. Further, a visual query language has been developed that accepts point-click-and-drag visual iconic input on the screen that is then translated into KSTL. User models are introduced to provide default parameter values for specifying query conditions. We have implemented the KMeD (Knowledge-based Medical Database) system using these concepts     

Efficient subsequence matching over large video databases

Video similarity matching has broad applications such as copyright detection, news tracking and commercial monitoring, etc. Among these applications, one typical task is to detect the local similarity between two videos without the knowledge on positions and lengths of each matched subclip pair. However, most studies so far on video detection investigate the global similarity between two short clips using a pre-defined distance function. Although there are a few works on video subsequence detection, all these proposals fail to provide an effective query processing mechanism. In this paper, we first generalize the problem of video similarity matching. Then, a novel solution called consistent keyframe matching (CKM) is proposed to solve the problem of subsequence matching based on video segmentation. CKM is designed with two goals: (1) good scalability in terms of the query sequence length and the size of video database and (2) fast video subsequence matching in terms of processing time. Good scalability is achieved by employing a batch query paradigm, where keyframes sharing the same query space are summarized and ordered. As such, the redundancy of data access is eliminated, leading to much faster video query processing. Fast subsequence matching is achieved by comparing the keyframes of different video sequences. Specifically, a keyframe matching graph is first constructed and then divided into matched candidate subgraphs. We have evaluated our proposed approach over a very large real video database. Extensive experiments demonstrate the effectiveness and efficiency of our approach.     

鲁棒的非线性优化的立体视觉－惯导SLAM

针对基于视觉特征的同时定位与地图构建（SLAM）系统在图像模糊、运动过快和特征缺失的情况下存在鲁棒性和精度急剧下降甚至失败的问题,提出了紧耦合的非线性优化的立体视觉-惯导SLAM系统．首先,以关键帧的位姿作为约束,采用分而治之的策略估计惯性测量单元（IMU）的偏差．在前端,针对ORB-SLAM2在跟踪过程中由于运动过快导致匀速运动模型失效的问题,通过预积分上一帧到当前帧的IMU数据,预测当前帧的初始位姿,并在位姿优化中加入了IMU预积分约束．然后,在后端优化中,在滑动窗口内优化关键帧的位姿、地图点和IMU预积分,并更新IMU的偏差．最后,通过EuRoC数据集验证该系统的性能,对比ORB-SLAM2系统、VINS-Mono系统和OKVIS系统,该系统的精度分别提高了1.14倍、1.48倍和4.59倍;相比前沿的SLAM系统,该系统在快速运动、图像模糊和特征缺失条件下的鲁棒性也得到了提高．     

Skeleton-based control of fluid animation

We present a skeleton-based control method for fluid animation. Our method is designed to provide an easy and intuitive control approach while producing visually plausible fluid behavior. In our method, users are allowed to control animated fluid with skeleton keyframes. Expected results are then obtained by driving fluid towards a sequence of targets specified in these keyframes. In order to solve for an optimal driving solution, we propose a keyframe matching model based on the transportation principle. Moreover, to ensure that the fluid actors move as rigid bodies while preserving liquid properties during animation, we introduce an approach of driving solid-like liquid motion. Finally, we embed the skeleton-based control method into the standard fluid animation, and apply it to control fluid actors?? motion as well as liquid shape deformation. Experimental results show that our method can generate natural-looking interesting fluid behavior with little additional cost.     

基于差别子空间的用户击键特征识别

根据用户的击键行为特征，提出了一种基于差别子空间的识别算法，该算法仅依据用户前几次成功登录的击键特征计算出能够代表用户击键的共性特征向量，进而利用当前用户击键特征向量与共性特征向量的欧几里德距离作为判别依据来判定用户的身份。该算法主要进行内积运算，实现简单且识别速度快，实验结果表明该算法误报率较低，鲁棒性较强。     

Rapidly constructed appearance models for tracking in augmented reality applications

This paper presents a method for rapidly generating crude, appearance-based edge models consisting of a set of planes. The appearance of each plane is modeled using a set of keyframes containing a list of edgels. These models are generated from a short video sequence with a few annotated frames indicating the location of the object of interest. The data requires 3–5?min to collect using a handheld device instrumented with a camera. The models produced are used with an existing edge tracking algorithm modified to select the appropriate edge keyframe and detect occlusion. A framestore is also created containing several views of the object represented as sets of point features. The framestore is used to provide an initial, rough pose estimate for initializing contour tracking. The presented system is used to create an augmented reality application to guide a user through a machine tool setup and a printer maintenance task. The models are shown to be an accurate representation of the object. Additionally, the performance of various aspects of the model making and tracking algorithms are evaluated.     

面向边缘设备的高能效深度学习任务调度策略

近年来,深度学习在图像和自然语言处理等诸多领域表现出色,与深度学习相关的各类移动应用发展迅速,但由于移动网络状态的不稳定性及网络带宽的限制,基于云计算的深度模型任务可能出现较大响应延迟,严重影响用户体验.与此同时,深度模型对设备的计算及存储能力有较高的要求,无法直接在资源受限的移动设备中进行部署.因此,亟须设计一种新的计算模式,使得基于深度模型的移动应用能够满足用户对快速响应、低能耗及高准确率的期望.本文提出一种面向边缘设备的深度模型分类任务调度策略,该策略通过协同移动设备与边缘服务器,充分利用智能移动终端的便捷性和边缘服务器强大的计算能力,综合考虑分类任务的复杂度和用户期望,完成深度模型在移动设备和边缘服务器中的动态部署,并对推理任务进行动态调度,从而提升任务执行效率,降低深度学习模型推理开销.本文以基于卷积神经网络的图像识别应用为例,实验结果表明,在移动环境中,相比于准确率最高的深度模型,本文提出的高能效调度策略的推理能耗可降低93.2%、推理时间降低91.6%,同时准确率提升3.88%.     

Produktempfehlungssysteme mit minimalem Konsumentenaufwand und hoher Genauigkeit

In recent studies on recommendation systems, the choice-based conjoint analysis has been suggested as a method for measuring consumer preferences. This approach achieves high recommendation accuracy and does not suffer from the start-up problem because it is also applicable for recommendations for new consumers or of new products. However, this method requires massive consumer input, which causes consumer reluctance. In a simulation study, we demonstrate the high accuracy, but also the high user’s effort for using a utility-based recommendation system using a choice-based conjoint analysis with hierarchical Bayes estimation. In order to reduce the conflict between consumer effort and recommendation accuracy, we develop a novel approach that only shows Pareto-efficient alternatives and ranks them according to the number of dominated attributes. We demonstrate that, in terms of the decision accuracy of the recommended products, the ranked Pareto-front approach performs better than a recommendation system that employs choice-based conjoint analysis. Furthermore, the consumer’s effort is kept low and comparable to that of simple systems that require little consumer input.     

Key Probe: a technique for animation keyframe extraction

We present a novel constraint-based keyframe extraction technique, Key Probe. Based on animator-specified constraints, the method converts a skeleton-based motion or animated mesh to a keyframe-based representation. In contrast to previous curve simplification or clustering methods, we cast the keyframe extraction problem as a constrained matrix factorization problem and solve the problem based on the least-squares optimization technique. The extracted keyframes have two uses: they could be used for browsing or they may be blended to reconstruct all other frames of an animation. Our approach is general and suitable for both rigid-body and soft-body animations. Experiments on various types of animation examples show that the proposed method produces remarkable results in terms of quality and compression ratio. Empirical tests also show that our algorithm consistently offers better efficiency than those by principal component analysis (PCA) and independent component analysis (ICA).     

面向轻轨的高精度实时视觉定位方法

轻轨作为城市公共交通系统的重要组成部分, 对其实现智能化的管理势在必行. 针对城市轻轨定位系统要求精度高、实时强且易于安装等特点, 本文提出一种基于全局?局部场景特征与关键帧检索的定位方法. 该方法在语义信息的指导下, 从单目相机获取的参考帧中提取区别性高的区域作为关键区域. 并结合像素点位置线索利用无监督学习的方式筛选关键区域中描述力强的像素对生成二值化特征提取模式, 不仅能够提升匹配精度还显著提高了在线模块场景特征提取与匹配的速度. 其次, 以场景显著性分数为依据获取的关键帧避免了具有相似外观的场景给定位带来的干扰, 并能辅助提高场景在线匹配的精度与效率. 本文使用公开测试数据集以及具有挑战性的轻轨数据集进行测试. 实验结果表明, 本系统在满足实时性要求的同时, 其定位准确率均可达到90%以上.