三焦点张量重投影视频稳像算法

目的 目前,特征点轨迹稳像算法无法兼顾轨迹长度、鲁棒性及轨迹利用率,因此容易造成该类算法的视频稳像结果扭曲失真或者局部不稳。针对此问题,提出基于三焦点张量重投影的特征点轨迹稳像算法。方法 利用三焦点张量构建长虚拟轨迹,通过平滑虚拟轨迹定义稳定视图,然后利用三焦点张量将实特征点重投影到稳定视图,以此实现实特征点轨迹的平滑,最后利用网格变形生成稳定帧。结果 对大量不同类型的视频进行稳像效果测试,并且与典型的特征点轨迹稳像算法以及商业软件进行稳像效果对比,其中包括基于轨迹增长的稳像算法、基于对极几何点转移的稳像算法以及商业软件Warp Stabilizer。本文算法的轨迹长度要求低、轨迹利用率高以及鲁棒性好,对于92%剧烈抖动的视频,稳像效果优于基于轨迹增长的稳像算法;对于93%缺乏长轨迹的视频以及71.4%存在滚动快门失真的视频,稳像效果优于Warp Stabilizer;而与基于对极几何点转移的稳像算法相比,退化情况更少,可避免摄像机阶段性静止、摄像机纯旋转等情况带来的算法失效问题。结论 本文算法对摄像机运动模式和场景深度限制少,不仅适宜处理缺少视差、场景结构非平面、滚动快门失真等常见的视频稳像问题,而且在摄像机摇头、运动模糊、剧烈抖动等长轨迹缺乏的情况下,依然能取得较好的稳像效果,但该算法的时间性能还有所不足。     

Generation of augmented video sequences combining behavioral animation and multi‐object tracking

In this paper we present a novel approach to generate augmented video sequences in real‐time, involving interactions between virtual and real agents in real scenarios. On the one hand, real agent motion is estimated by means of a multi‐object tracking algorithm, which determines real objects' position over the scenario for each time step. On the other hand, virtual agents are provided with behavior models considering their interaction with the environment and with other agents. The resulting framework allows to generate video sequences involving behavior‐based virtual agents that react to real agent behavior and has applications in education, simulation, and in the game and movie industries. We show the performance of the proposed approach in an indoor and outdoor scenario simulating human and vehicle agents. Copyright © 2009 John Wiley & Sons, Ltd.     

Crowd of Virtual Humans: a New Approach for Real Time Navigation in Complex and Structured Environments

The navigation activity is an every day practice for any human being capable of locomotion. Our objective in this work is to reproduce this crucial human activity inside virtual environments. Putting together the high complexity of a realistic environment such as a city, a big amount of virtual humans and the real‐time constraint requires to optimize each aspect of the animation process. In this paper, we present a suitable topological structuring of the geometric environment to allow fast path finding as well as an efficient reactive navigation algorithm for virtual humans evolving inside a crowd.     

Facial animation based on context-dependent visemes

This paper presents a novel approach for the generation of realistic speech synchronized 3D facial animation that copes with anticipatory and perseveratory coarticulation. The methodology is based on the measurement of 3D trajectories of fiduciary points marked on the face of a real speaker during the speech production of CVCV non-sense words. The trajectories are measured from standard video sequences using stereo vision photogrammetric techniques. The first stationary point of each trajectory associated with a phonetic segment is selected as its articulatory target. By clustering according to geometric similarity all articulatory targets of a same segment in different phonetic contexts, a set of phonetic context-dependent visemes accounting for coarticulation is identified. These visemes are then used to drive a set of geometric transformation/deformation models that reproduce the rotation and translation of the temporomandibular joint on the 3D virtual face, as well as the behavior of the lips, such as protrusion, and opening width and height of the natural articulation. This approach is being used to generate 3D speech synchronized animation from both natural and synthetic speech generated by a text-to-speech synthesizer.     

基于MEBN的大型舰船舰面保障作业检测

大型舰船舰面保障作业指驻扎于大型舰船的飞机的一次出动过程中,发生于舰面的针对飞机的出动回收准备活动（如飞机调运、物资转运、飞机滑跑起飞等）。基于舰面监控视频检测舰面保障作业,利于分析并提高保障作业效率。在大型舰船携带的机群出动过程中,各飞机的保障作业同时进行,涉及数以百计的操作员、高级器械及设施设备的协作。针对大型舰船舰面保障作业检测问题中,同一视频场景内同时发生多个保障作业且参与者位置交叉、重叠的难点,提出一种基于多实体贝叶斯网络的大型舰船舰面保障作业检测技术。该技术假设在视频数据预处理阶段获取视频中各目标物清晰连贯的运动轨迹;在保障作业检测阶段,输入目标物轨迹,先基于带约束的轨迹自动聚类算法,自动计算聚类个数,并将参与同一保障作业的轨迹聚至同一类;在各聚类内,采用基于MEBN的保障作业识别技术,推断保障作业类型。大型舰船舰面监控视频数据集上的实验表明,该方法对同一视频场景中同时发生的且参与者位置交叉的舰面保障作业具有良好的检测效果。     

Identification and analysis of unsatisfactory psychosocial work situations: a participatory approach employing video-computer interaction

A method for psychosocial evaluation of potentially stressful or unsatisfactory situations in manual work was developed. It focuses on subjective responses regarding specific situations and is based on interactive worker assessment when viewing video recordings of oneself. The worker is first video-recorded during work. The video is then displayed on the computer terminal, and the filmed worker clicks on virtual controls on the screen whenever an unsatisfactory psychosocial situation appears; a window of questions regarding psychological demands, mental strain and job control is then opened. A library with pictorial information and comments on the selected situations is formed in the computer. The evaluation system, called PSIDAR, was applied in two case studies, one of manual materials handling in an automotive workshop and one of a group of workers producing and testing instrument panels. The findings indicate that PSIDAR can provide data that are useful in a participatory ergonomic process of change.     

EACS: Effective Avoidance Combination Strategy

When navigating in crowds, humans are able to move efficiently between people. They look ahead to know which path would reduce the complexity of their interactions with others. Current navigation systems for virtual agents consider long‐term planning to find a path in the static environment and short‐term reactions to avoid collisions with close obstacles. Recently some mid‐term considerations have been added to avoid high density areas. However, there is no mid‐term planning among static and dynamic obstacles that would enable the agent to look ahead and avoid difficult paths or find easy ones as humans do. In this paper, we present a system for such mid‐term planning. This system is added to the navigation process between pathfinding and local avoidance to improve the navigation of virtual agents. We show the capacities of such a system using several case studies. Finally we use an energy criterion to compare trajectories computed with and without the mid‐term planning.     

Virtual view synthesis using stereo vision based on the sum of absolute difference

With the development of multimedia, the technology of image and video is growing from 2D to 3D, thus interactivity is going to become a main character of future multimedia technology. Virtual view synthesis, known as the analogue expression to the real world, is one of the key techniques in interactive 3D video systems. This paper proposes a new algorithm of virtual view synthesis, which is based on disparity estimation. Considering two rectified input reference images that are taken in the same scene simultaneously, the accurate dense disparity maps are gained as the first step. Then image interpolation is used to synthesize certain virtual view image and reverse mapping is adopted to fill up the holes which are formed in previous process. By defining a position parameter, this algorithm can produce results at an arbitrary view between the two original views. Experimental results illustrate the superiority of the proposed method.     

Planning Small Talk behavior with cultural influences for multiagent systems

There are several factors that influence communicative behavior, such as gender, personality or culture. As virtual agents interact in a more and more human-like manner, their behavior should be dependent on social factors as well. Culture is a phenomenon that affects one’s behavior without one realizing it. Behavior is thus sometimes perceived as inappropriate because there is no awareness of the cultural gap. Thus, we think cultural background should also influence the communication behavior of virtual agents. Behavioral differences are sometimes easy to recognize by humans but still hard to describe formally, to enable integration into a system that automatically generates culture-specific behavior. In our work, we focus on culture-related differences in the domain of casual Small Talk. Our model of culture-related differences in Small Talk behavior is based on findings described in the literature as well as on a video corpus that was recorded in Germany and Japan. In a validation study, we provide initial evidence that our simulation of culture-specific Small Talk with virtual agents is perceived differently by human observers. We thus implemented a system that automatically generates culture-specific Small Talk dialogs for virtual agents.     

Online inserting virtual characters into dynamic video scenes

The seamless integration of virtual characters into dynamic scenes captured by video is a challenging problem. In order to achieve consistent composite results, both the virtual and real characters must share the same geometrical constraints and their interactions must follow some common sense. One essential question is how to detect the motion of real objects—such as real characters moving in the video—and how to steer virtual characters accordingly to avoid unrealistic collisions. We propose an online solution. First, by analysis of the input video, the motion states of the real pedestrians are recovered into a common world 3D coordinate system. Meanwhile, a simplified accuracy measurement is defined to represent the confidence of the motion estimate. Then, under the constraints imposed by the real dynamic objects, the motion of virtual characters are accommodated by a uniform steering model. The final step is to merge virtual objects back to the real video scene by taking into account visibility and occlusion constraints between real foreground objects and virtual ones. Several examples demonstrate the efficiency of the proposed algorithm. Copyright © 2011 John Wiley & Sons, Ltd.     

Human action recognition with salient trajectories and multiple kernel learning

Action recognition in videos plays an important role in the field of computer vision and multimedia, and there exist lots of challenges due to the complexity of spatial and temporal information. Trajectory-based approach has shown to be efficient recently, and a new framework and algorithm of trajectory space information based multiple kernel learning (TSI-MKL) is exploited in this paper. First, dense trajectories are extracted as raw features, and three saliency maps are computed corresponding to color, space, and optical flow on frames at the same time. Secondly, a new method combining above saliency maps is proposed to filter the achieved trajectories, by which a set of salient trajectories only containing foreground motion regions is obtained. Afterwards, a novel two-layer clustering is developed to cluster the obtained trajectories into several semantic groups and the ultimate video representation is generated by encoding each group. Finally, representations of different semantic groups are fed into the proposed kernel function of a multiple kernel classifier. Experiments are conducted on three popular video action datasets and the results demonstrate that our presented approach performs competitively compared with the state-of-the-art.     

基于轨迹行为模式特征的视频拷贝检测算法

为了有效地利用视频的时域运动信息来提高视频拷贝检测的精度和鲁棒性,提出一种基于特征点轨迹行为模式的拷贝检测算法.首先从视频连续帧中提取特征点轨迹的行为模式特征,然后采用视觉关键词典技术构造视频的运动特征,最后基于运动特征的相似度进行视频拷贝检测.该算法在TRECVID标准数据集上取得了较高的检测精度.实验分析表明,基于轨迹的运动特征具有较强的描述区分能力,对各种常见的拷贝变化具有鲁棒性.     

基于虚拟窗口的自适应车流量检测应用研究

基于视频交通检测中实时性和准确性要求,研究了三帧差分、背景差分及动态阈值等算法,提出增加虚拟窗口的方法,使整个车流量检测算法只针对有效序列图像信息进行处理.该方法通过排除差异大图像求均值,快速初始化虚拟窗口背景模型,在检测窗口内使用三帧差分和动态阈值准确定位运动车辆目标,实时更新背景后,根据背景差分法得到的窗口图像信息统计车流情况.实验结果表明,该方法可以有效应用于基于视频的实时车流量检测中.     

基于轨迹聚类的公共安全异常检测

公共安全异常检测的需求越来越迫切,监控中基于轨迹聚类的检测方法越来越流行,但是现有方法在处理高维不等长轨迹数据时效果并不理想。提出一个新的轨迹聚类方法,该方法通过组合动态时间弯曲和密度峰算法实现。动态时间弯曲用于度量轨迹间的距离,密度峰算法根据距离进行聚类。前者可直接度量不等长轨迹聚类,后者是近年提出的非球体分布数据聚类算法,以局部密度和最近邻聚类组合实现。实验在PETS2006监控视频数据集上进行,测试结果表明该方法有效地发现了异常的轨迹行为模式。     

The benefit of being physically present: A survey of experimental works comparing copresent robots,telepresent robots and virtual agents

The effects of physical embodiment and physical presence were explored through a survey of 33 experimental works comparing how people interacted with physical robots and virtual agents. A qualitative assessment of the direction of quantitative effects demonstrated that robots were more persuasive and perceived more positively when physically present in a user׳s environment than when digitally-displayed on a screen either as a video feed of the same robot or as a virtual character analog; robots also led to better user performance when they were collocated as opposed to shown via video on a screen. However, participants did not respond differently to physical robots and virtual agents when both were displayed digitally on a screen – suggesting that physical presence, rather than physical embodiment, characterizes people׳s responses to social robots. Implications for understanding psychological response to physical and virtual agents and for methodological design are discussed.     

基于距离特征的自适应阈值视频拼接算法

针对虚拟漫游系统中视频拼接问题,提出一种基于距离的自适应阈值拼接算法。通过提取距离特征,并在待匹配的帧图像中采用自适应阈值序列的序贯相似性检测算法搜索匹配特征,从而确定重叠部分的起始列值。实验结果表明,该算法能更好地实现视频拼接,且减少计算量,提高拼接速度。     

Estimating the focus of expansion in a video sequence using the trajectories of interest points

In this paper, we present a new algorithm for the computation of the focus of expansion in a video sequence. Although several algorithms have been proposed in the literature for its computation, almost all of them are based on the optical flow vectors between a pair of consecutive frames, so being very sensitive to noise, optical flow errors and camera vibrations. Our algorithm is based on the computation of the vanishing point of point trajectories, thus integrating information for more than two consecutive frames. It can improve performance in the presence of erroneous correspondences and occlusions in the field of view of the camera.The algorithm has been tested with virtual sequences generated with Blender, as well as some real sequences from both, the public KITTI benchmark, and a number of challenging video sequences also proposed in this paper. For comparison purposes, some algorithms from the literature have also been implemented. The results show that the algorithm has proven to be very robust, outperforming the compared algorithms, specially in outdoor scenes, where the lack of texture can make optical flow algorithms yield inaccurate results. Timing evaluation proves that the proposed algorithm can reach up to 15fps, showing its suitability for real-time applications.     

Modeling human hand movements,facial expressions,and articulation to synthesize and visualize gesture information

Information and mathematical models are proposed for the animation of sign language communication based on a virtual human model. A model is developed to fix sign language morphemes and is used to create a technology and software to generate, store, and reproduce gestures. Algorithmic solutions are proposed for the computation of human-like trajectories of hand and body movements in passing from a gesture to another and also facial expressions and articulation.     

Mixing virtual and real scenes in the site of ancient Pompeii

This paper presents an innovative 3D reconstruction of ancient fresco paintings through the real‐time revival of their fauna and flora, featuring groups of virtual animated characters with artificial‐life dramaturgical behaviours in an immersive, fully mobile augmented reality (AR) environment. The main goal is to push the limits of current AR and virtual storytelling technologies and to explore the processes of mixed narrative design of fictional spaces (e.g. fresco paintings) where visitors can experience a high degree of realistic immersion. Based on a captured/real‐time video sequence of the real scene in a video‐see‐through HMD set‐up, these scenes are enhanced by the seamless accurate real‐time registration and 3D rendering of realistic complete simulations of virtual flora and fauna (virtual humans and plants) in a real‐time storytelling scenario‐based environment. Thus the visitor of the ancient site is presented with an immersive and innovative multi‐sensory interactive trip to the past. Copyright © 2005 John Wiley & Sons, Ltd.