Action-specific motion prior for efficient Bayesian 3D human body tracking

In this paper, we aim to reconstruct the 3D motion parameters of a human body model from the known 2D positions of a reduced set of joints in the image plane. Towards this end, an action-specific motion model is trained from a database of real motion-captured performances, and used within a particle filtering framework as a priori knowledge on human motion. First, our dynamic model guides the particles according to similar situations previously learnt. Then, the state space is constrained so only feasible human postures are accepted as valid solutions at each time step. As a result, we are able to track the 3D configuration of the full human body from several cycles of walking motion sequences using only the 2D positions of a very reduced set of joints from lateral or frontal viewpoints.     

Kinect动作捕捉技术在三维动画教学中的应用与实践

针对传统三维角色动画制作成本高、时间长的问题,文章介绍了一种应用Kinect动作捕捉技术实现高效制作三维角色动画的方法。该方法借助Kinect体感摄像机捕捉真人的动作生成骨骼关节的关键帧数据并输出bvh动作路径文件,然后把bvh动作路径数据导入C4D软件中,就可以驱动角色模型完成角色动画的制作。把这种方法运用到三维角色动画教学实践中,有利于提高学生学习的兴趣和效率。     

Review of techniques for motion capture data processing

In order to high reality and efficiency,the technique of motion capture (MoCap) has been widely used in the field of computer animation.With the development of motion capture,a large amount of motion capture databases are available and this is significant for the reuse of motion data.But due to the high degree of freedoms and high capture frequency,the dimension of the motion capture data is usually very high and this will lead to a low efficiency in data processing.So how to process the high dimension data and design an efficient and effective retrieval approach has become a challenge which we can’t ignore.In this paper,first we lay out some problems about the key techniques in motion capture data processing.Then the existing approaches are analyzed and summarized.At last,some future work is proposed.     

3D motion estimation of bubbles of gas in fluid glass,using an optical flow gradient technique extended to a third dimension

To solve the problem of estimating velocities of gas bubbles in melted glass, a method based on optical flow constraint (OFC) has been extended to the 3D case. A single camera, whose distance to the fluid is variable in time, is used to capture a sequence of frames at different depths. Since objects are not static, we cannot obtain two frames of different height values at the same time, and to our knowledge, this prevents the use of common 3D motion estimation techni ques. Since the information will be rather sparse, our estimation takes several measures around a given pixel and discards the erroneous ones, using a robust estimator. Along with the exposition of the practical application, the estimation proposed here is first contrasted in the 2D case to common benchmarks and then evaluated for a synthetic problem where velocities are known. Received: 9 July 2001 / Accepted: 5 August 2002 Published online: 3 June 2003 This work has been supported by Saint Gobain Cristaleria S.A., under contract FUO-EM-034-01 with Oviedo University, Spain.     

三维图像运动分析在汽车乘员保护中的应用研究

以安全气袋在不同折叠方式下的展开特性规律为研究对象，通过对静态点爆试验的三维图像运动分析获得了典型安全气袋展开的基本规律，并验证了数值模拟计算结论的可靠性，对安全气袋的折叠方式进行了优化设计。经过三维图像运动分析技术对新型折叠方案的分析，表明新设计的折叠方式的各项展开指标均达到了较高的要求，能够有效提高安全气袋的防护效果，减小对离位乘员的伤害。     

Motion capture and human pose reconstruction from a single-view video sequence

We propose a framework to reconstruct the 3D pose of a human for animation from a sequence of single-view video frames. The framework for pose construction starts with background estimation and the performer?s silhouette is extracted using image subtraction for each frame. Then the body silhouettes are automatically labeled using a model-based approach. Finally, the 3D pose is constructed from the labeled human silhouette by assuming orthographic projection. The proposed approach does not require camera calibration. It assumes that the input video has a static background, it has no significant perspective effects, and the performer is in an upright position. The proposed approach requires minimal user interaction.     

双目视觉下三维人体运动跟踪算法

由于人体运动的复杂性,人体运动轨迹的快速改变和人体自遮挡现象经常发生,这给人体运动跟踪带来了很大的困难。针对此问题提出了一种基于三维Kalman滤波器和人体约束的人体运动跟踪算法。该算法首先利用外极线约束和灰度互相关法对二维标记点进行立体匹配,计算各个标记点的三维位置,从而构建得到三维标记点;然后利用三维Kalman滤波器对三维标记点进行跟踪;最后利用人体约束检验和修正跟踪结果。实验结果表明,该算法能有效地对复杂人体动作进行跟踪并能从跟踪错误中正确恢复。     

Retrieval of logically relevant 3D human motions by Adaptive Feature Selection with Graded Relevance Feedback

A system that can retrieve logically relevant 3D captured motions is useful in game and animation production. We presented a robust logical relevance metric based on the relative distances among the joints. Existing methods select a universal subset of features for all kinds of queries which may not well characterize the variations in different queries. To break through this limitation we proposed an Adaptive Feature Selection (AFS) method that abstracts the characteristics of the query by a Linear Regression Model, and different feature subsets can be selected according to the properties of the specific query. With a Graded Relevance Feedback (GRF) algorithm, we refined the feature subset that enhances the retrieval performance according to the graded relevance of the feedback samples. With an ontology that predefines the logical relevance between motion classes in terms of graded relevance, the performance of the proposed AFS-GRF algorithm is evaluated and shown to outperform other class-specific feature selection and motion retrieval methods.     

Markerless human motion capture by Markov random field and dynamic graph cuts with color constraints

Currently, many vision-based motion capture systems require passive markers attached to key locations on the human body. However, such systems are intrusive with limited application. The algorithm that we use for human motion capture in this paper is based on Markov random field (MRF) and dynamic graph cuts. It takes full account of the impact of 3D reconstruction error and integrates human motion capture and 3D reconstruction into MRF-MAP framework. For more accurate and robust performance, we extend our algorithm by incorporating color constraints into the pose estimation process. The advantages of incorporating color constraints are demonstrated by experimental results on several video sequences. Supported by the National Basic Research Program of China (Grant No. 2006CB303105)     

基于神经网络的运动参数估计

分别就两种约束使用神经网络对三维刚体运动进行参数估计.一是基于三维点匹配,将预测的运动参数作用于运动前的坐标,与运动后坐标进行比较;二是基于二维运动场,将使用预测的运动参数计算得出的二维运动场与图像序列中计算得出的二维运动场进行比较.两个神经网络均使用Newton-Raphson方法更新权值,以达到目标误差最小化.通过实验验证了该神经网络方法.     

Towards human motion capture from a camera mounted on a mobile robot

This article describes a multiple feature data fusion applied to a particle filter for marker-less human motion capture (HMC) by using a single camera devoted to an assistant mobile robot. Particle filters have proved to be well suited to this robotic context. Like numerous approaches, the principle relies on the projection of the model's silhouette of the tracked human limbs and appearance features located on the model surface, to validate the particles (associated configurations) which correspond to the best model-to-image fits. Our particle filter based HMC system is improved and extended in two ways. First, our estimation process is based on the so-called AUXILIARY scheme which has been surprisingly seldom exploited for tracking purpose. This scheme is shown to outperform conventional particle filters as it limits drastically the well-known burst in term of particles when considering high dimensional state-space. The second line of investigation concerns data fusion. Data fusion is considered both in the importance and measurement functions with some degree of adaptability depending on the current human posture and the environmental context encountered by the robot. Implementation and experiments on indoor sequences acquired by an assistant mobile robot highlight the relevance and versatility of our HMC system. Extensions are finally discussed.     

A region-based selective optical flow back-projection for genuine motion vector estimation

Motion vector plays one significant feature in moving object segmentation. However, the motion vector in this application is required to represent the actual motion displacement, rather than regions of visually significant similarity. In this paper, region-based selective optical flow back-projection (RSOFB) which back-projects optical flows in a region to restore the region's motion vector from gradient-based optical flows, is proposed to obtain genuine motion displacement. The back-projection is performed based on minimizing the projection mean square errors of the motion vector on gradient directions. As optical flows of various magnitudes and directions provide various degrees of reliability in the genuine motion restoration, the optical flows to be used in the RSOFB are optimally selected based on their sensitivity to noises and their tendency in causing motion estimation errors. In this paper a deterministic solution is also derived for performing the minimization and obtaining the genuine motion magnitude and motion direction.     

人体运动实时捕捉设备传输数据的滤波与处理

简介人体运动实时捕捉设备的工作原理,对设备探测并传输来的数据进行特性分析,根据分析结论,提出了干扰数据识别与修复、纵向平滑滤波、首帧收敛化处理和值域限定等滤波和处理方法,并详细阐述了这些方法的综合运用。系统使用这些方法,经运行调试后,传输数据稳定,系统对Marker点的探测精度也符合系统设计的要求（≤6mm）,为实现各点坐标计算提供了必要条件。     

Non-rigid 3D shape tracking from multiview video

We present a fast and efficient non-rigid shape tracking method for modeling dynamic 3D objects from multiview video. Starting from an initial mesh representation, the shape of a dynamic object is tracked over time, both in geometry and topology, based on multiview silhouette and 3D scene flow information. The mesh representation of each frame is obtained by deforming the mesh representation of the previous frame towards the optimal surface defined by the time-varying multiview silhouette information with the aid of 3D scene flow vectors. The whole time-varying shape is then represented as a mesh sequence which can efficiently be encoded in terms of restructuring and topological operations, and small-scale vertex displacements along with the initial model. The proposed method has the ability to deal with dynamic objects that may undergo non-rigid transformations and topological changes. The time-varying mesh representations of such non-rigid shapes, which are not necessarily of fixed connectivity, can successfully be tracked thanks to restructuring and topological operations employed in our deformation scheme. We demonstrate the performance of the proposed method both on real and synthetic sequences.     

Animated human agents with motion planning capability for 3D-space postural goals

In this paper, we present a rule-based heuristic method of motion planning for an animated human agent with massively redundant degrees of freedom. It constructs motion plans to achieve 3D-space goals of control points on the body, e.g. a hand, while avoiding collisions. Like the artificial potential field approach, the method performs motion decisions in 3D world space rather than in joint space. To handle the massively redundant degrees of freedom, we use a qualitative kinematic model, which specifies motions of body parts and dependencies among them, without specifying the exact distance parameters. This model helps the body select appropriate primitive motions for given goals of control points more globally than does the gradient vector of an artificial potential field of the body. The method simulates (in imagination) the suggested plan to find whether some body parts hit objects, and how much they penetrate the objects. Based on this simulated collision information, the method suggests intermediate goals of the collision body parts. A subplan to achieve these intermediate goals is again postulated by using the qualitative kinematic model. This explicit reasoning helps alleviate the basic cause of local minima in the potential field approach, namely, conflicts between attractive potential fields due to goal positions of control points and repulsive potential fields due to obstacles.     

一种三维小波视频编码方法

基于三维小波变换的视频编码技术成为当今研究的热点。研究了一种时间轴小波分解的改进结构,给出了时间轴分解后各帧的重要性分析,在此基础上提出了一种结合运动补偿的三维小波视频编码新方法。该算法首先将视频序列按帧序的奇偶性分成两组,对各组分别进行改进的时间轴小波分解,然后对两组中的低频帧进行运动补偿,可以进一步减少数据量。在所要达到的目标码率条件下,对各帧根据其重要性分配最优的压缩码率,并利用SPIHT算法进行压缩。实验结果表明,该方法可以获得较好的压缩效果。     

3D video and free viewpoint video—From capture to display

This paper gives an end-to-end overview of 3D video and free viewpoint video, which can be regarded as advanced functionalities that expand the capabilities of a 2D video. Free viewpoint video can be understood as the functionality to freely navigate within real world visual scenes, as it is known for instance from virtual worlds in computer graphics. 3D video shall be understood as the functionality that provides the user with a 3D depth impression of the observed scene, which is also known as stereo video. In that sense as functionalities, 3D video and free viewpoint video are not mutually exclusive but can very well be combined in a single system. Research in this area combines computer graphics, computer vision and visual communications. It spans the whole media processing chain from capture to display and the design of systems has to take all parts into account, which is outlined in different sections of this paper giving an end-to-end view and mapping of this broad area. The conclusion is that the necessary technology including standard media formats for 3D video and free viewpoint video is available or will be available in the future, and that there is a clear demand from industry and user for such advanced types of visual media. As a consequence we are witnessing these days how such technology enters our everyday life