Fast and high-quality virtual view synthesis from multi-view plus depth videos

Multimedia Tools and Applications - Depth image based rendering (DIBR) is an effective method for virtual view synthesis from Multi-view Plus Depth(MVD) video. Synthetic images, however, often...     

Appearance-based recognition of 3-D objects by cluttered background and occlusions

In this article we present a new appearance-based approach for the classification and the localization of 3-D objects in complex scenes. A main problem for object recognition is that the size and the appearance of the objects in the image vary for 3-D transformations. For this reason, we model the region of the object in the image as well as the object features themselves as functions of these transformations. We integrate the model into a statistical framework, and so we can deal with noise and illumination changes. To handle heterogeneous background and occlusions, we introduce a background model and an assignment function. Thus, the object recognition system becomes robust, and a reliable distinction, which features belong to the object and which to the background, is possible. Experiments on three large data sets that contain rotations orthogonal to the image plane and scaling with together more than 100 000 images show that the approach is well suited for this task.     

Episode detection in videos captured using a head-mounted camera

With the advent of wearable computing, personal imaging, photojournalism and personal video diaries, the need for automated archiving of the videos captured by them has become quite pressing. The principal device used to capture the human-environment interaction with these devices is a wearable camera (usually a head-mounted camera). The videos obtained from such a camera are raw and unedited versions of the visual interaction of the wearer (the user of the camera) with the surroundings. The focus of our research is to develop post-processing techniques that can automatically abstract videos based on episode detection. An episode is defined as a part of the video that was captured when the user was interested in an external event and paid attention to record it. Our research is based on the assumption that head movements have distinguishable patterns during an episode occurrence and these patterns can be exploited to differentiate between an episode and a non-episode. Here we present a novel algorithm exploiting the head and body behaviour for detecting the episodes. The algorithms performance is measured by comparing the ground truth (user-declared episodes) with the detected episodes. The experiments show the high degree of success we achieved with our proposed method on several hours of head-mounted video captured in varying locations.     

Degenerate motions in multicamera cluster SLAM with non-overlapping fields of view

An analysis of the relative motion and point feature model configurations leading to solution degeneracy is presented, for the case of a Simultaneous Localization and Mapping system using multicamera clusters with non-overlapping fields-of-view. The SLAM optimization system seeks to minimize image space reprojection error and is formulated for a cluster containing any number of component cameras, observing any number of point features over two keyframes. The measurement Jacobian is transformed to expose a reduced-dimension representation such that the degeneracy of the system can be determined by the rank of a dense submatrix. A set of relative motions sufficient for degeneracy are identified for certain cluster configurations, independent of target model geometry. Furthermore, it is shown that increasing the number of cameras within the cluster and observing features across different cameras over the two keyframes reduces the size of the degenerate motion sets significantly.     

Automatic mesh generation in 2-D and 3-D objects

The object of the paper is to propose a simple method of mesh generation for objects with complicated boundary configurations and present a program which can generate the element data required for Finite Element Analysis based on these methods. The area to be meshed is considered as a combination of different sub-areas which are further sub-divided into the required number of triangular elements. The main part of the program developed in FORTRAN is included in this paper.     

3-D corridor modeling from a single view under natural lightingconditions

The author discusses the modeling of a 3-D indoor scene from a single view using a generic object model. It is assumed that an image is made by a well-calibrated camera, and that the camera height above the floor is known. The image is assumed to be a projection of a natural corridor scene from which a generic model is known, but the specific model is unknown. This system can model any corridor that satisfies the following conditions: (1) the corridor is a rectangular parallelepiped, (2) there are several lines along each axis of the rectangle parallelepiped, (3) there are not many parallel lines in directions other than along the principal axes, and (4) corridor height is within a known range. The system for 3-D modeling from a single view of such a corridor consists of a robust bottom-up image processing part and a top-down, model-based interpretation part. Experimental results show that specific corridor models can be recovered from a single view with an approximately 1% estimation error     

单目视频中的多视角行为识别方法

针对多视角人体行为识别中,因选取单一特征所造成的识别困难,提出了一种基于分类特征提取的多视角行为识别方法。该方法通过有效利用视频的二维图像序列所包含的三维信息、实现对人体朝向的聚类,解决因观察角度不同造成的行为特征难以选取的问题。实验证明,该方法对单目视频中不同角度的行为动作有较高的识别率。     

Intelligibility of speech in a virtual 3-D environment

In a simulated air traffic control task, improvement in the detection of auditory warnings when using virtual 3-D audio depended on the spatial configuration of the sounds. Performance improved substantially when two of four sources were placed to the left and the remaining two were placed to the right of the participant. Surprisingly, little or no benefits were observed for configurations involving the elevation or transverse (front/back) dimensions of virtual space, suggesting that position on the interaural (left/right) axis is the crucial factor to consider in auditory display design. The relative importance of interaural spacing effects was corroborated in a second, free-field (real space) experiment. Two additional experiments showed that (a) positioning signals to the side of the listener is superior to placing them in front even when two sounds are presented in the same location, and (b) the optimal distance on the interaural axis varies with the amplitude of the sounds. These results are well predicted by the behavior of an ideal observer under the different display conditions. This suggests that guidelines for auditory display design that allow for effective perception of speech information can be developed from an analysis of the physical sound patterns.     

Tiled streaming for layered 3D virtual reality videos with viewport prediction

Multimedia Tools and Applications - In recent years, the demand of 3D video services has gradually increased. More and more bandwidth hungry applications are proposed, such as immersive media...     

Interpolation techniques for 3-D object generation

The technique of interpolation (or blending) and its generalization is considered in this paper. A unifying model for generating 3-D objects using this technique is presented. Various factors which affect the shape of generated objects are identified. Several illustrative examples of 3-D objects generated by this technique, using linear as well as non-linear interpolation, are presented. Although many of the objects have complex shapes, they are represented by simple closed form mathematical equations.     

Towards an integrated view of 3-D computer animation

To automate character animation and extend it to 3-D we need to create and manipulate three-dimensional models of articulated figures as well as the worlds they will inhabit.Abstraction andadaptive motion are key mechanisms for dealing with thedegrees of freedom problem, which refers to the sheer volume of control information necessary for coordinating the motion of an articulated figure when the number of links is large. A three level hierarchy of control modes for animation is proposed:guiding, animator-level, andtask-level systems. Guiding is best suited for specifiying fine details but unsuited for controlling complex motion. Animatorlevel programming is powerful but difficult. Task-level systems give us facile control over complex motions and tasks by trading off explicit control over the details of motion. The integration of the three control levels is discussed.     

Fast generation of 3-D deformable moving surfaces

Dynamic surface modeling is an important subject of geometric modeling due to their extensive applications in engineering design, entertainment and medical visualization. Many deformable objects in the real world are dynamic objects as their shapes change over time. Traditional geometric modeling methods are mainly concerned with static problems, therefore unsuitable for the representation of dynamic objects. Apart from the definition of a dynamic modeling problem, another key issue is how to solve the problem. Because of the complexity of the representations, currently the finite element method or finite difference method is usually used. Their major shortcoming is the excessive computational cost, hence not ideal for applications requiring real-time performance. We propose a representation of dynamic surface modeling with a set of fourth order dynamic partial differential equations (PDEs). To solve these dynamic PDEs accurately and efficiently, we also develop an effective resolution method. This method is further extended to achieve local deformation and produce n-sided patches. It is demonstrated that this new method is almost as fast and accurate as the analytical closed form resolution method and much more efficient and accurate than the numerical methods.     

Automatic camera control in virtual environments augmented using multiple sparse videos

Automated virtual camera control has been widely used in animation and interactive virtual environments. We have developed a multiple sparse camera based free view video system prototype that allows users to control the position and orientation of a virtual camera, enabling the observation of a real scene in three dimensions (3D) from any desired viewpoint. Automatic camera control can be activated to follow selected objects by the user. Our method combines a simple geometric model of the scene composed of planes (virtual environment), augmented with visual information from the cameras and pre-computed tracking information of moving targets to generate novel perspective corrected 3D views of the virtual camera and moving objects. To achieve real-time rendering performance, view-dependent textured mapped billboards are used to render the moving objects at their correct locations and foreground masks are used to remove the moving objects from the projected video streams. The current prototype runs on a PC with a common graphics card and can generate virtual 2D views from three cameras of resolution 768×576 with several moving objects at about 11 fps.     

Semantic concept mining in cricket videos for automated highlight generation

This paper presents a novel approach towards automated highlight generation of broadcast sports video sequences from its extracted events and semantic concepts. A sports video is hierarchically divided into temporal partitions namely, megaslots, slots, and semantic entities, namely concepts, and events. The proposed method extracts event sequence from video and classifies each sequence into a concept by sequential association mining. The extracted concepts and events within the concepts are selected according to their degree of importance to include those in the highlights. A parameter degree of abstraction is proposed, which gives a choice to the user about how concisely the extracted concepts should be produced for a specified highlight duration. We have successfully extracted highlights from recorded video of cricket match and compared our results with the manually-generated highlights by sports television channel.     

Shape-based indexing scheme for camera view invariant 3-D object retrieval

Camera view invariant 3-D object retrieval is an important issue in many traditional and emerging applications such as security, surveillance, computer-aided design (CAD), virtual reality, and place recognition. One straightforward method for camera view invariant 3-D object retrieval is to consider all the possible camera views of 3-D objects. However, capturing and maintaining such views require an enormous amount of time and labor. In addition, all camera views should be indexed for reasonable retrieval performance, which requires extra storage space and maintenance overhead. In the case of shape-based 3-D object retrieval, such overhead could be relieved by considering the symmetric shape feature of most objects. In this paper, we propose a new shape-based indexing and matching scheme of real or rendered 3-D objects for camera view invariant object retrieval. In particular, in order to remove redundant camera views to be indexed, we propose a camera view skimming scheme, which includes: i) mirror shape pairing and ii) camera view pruning according to the symmetrical patterns of object shapes. Since our camera view skimming scheme considerably reduces the number of camera views to be indexed, it could relieve the storage requirement and improve the matching speed without sacrificing retrieval accuracy. Through various experiments, we show that our proposed scheme can achieve excellent performance.     

Study and implementation of the virtual view in unix

The security of information systems is important. protecting them from the viewpoint of methodology has become the critical task of their planners and implementers. This article proposes a new idea about the virtual view based upon unix stemming from the usual view concept,analyzes the characteristics of the virtual view,and finally gives the way of realization of the virtual view.     

一种新的六面体有限元网格算法

在有限元网格产生过程中，吸取弦须编织法中的STC概念，将六面体以节点剖分为基础的思想转变为以单元点为基础，建立了以单元生长为核心的剖分算法，以期解决节点拓扑结构在三维情况下的控制问题，对进一步实现稳定、全自动的六面体剖分具有很大的帮助。     

Human centred design of 3-D interaction devices to control virtual environments

It is commonly acknowledged that user needs should drive design, but often technical influences prevail. Currently, there are no standard interaction devices or interfaces used in 3-D environments, and there is a lack of specific best practice guidelines to develop these. This paper discusses the process of collecting feedback on prototype designs for VR/VE interaction devices from both expert users and non-expert users, and demonstrates how the information gained from human centred evaluation can be used to further the design process. Experiment 1 examined the usability of two magnetically tracked interaction devices with three different types of menus (Sphere, Linear and Fan). Quantitative and qualitative analysis was carried out on the results, and usability problems with the menus and devices are discussed. The findings from this experiment were translated into general design guidance, in addition to specific recommendations. A new device was designed on the basis of some of these recommendations and its usability was evaluated in Experiment 2. Feedback from participants in Experiment 2 demonstrated that the design recommendations emerging from Experiment 1 were successfully applied to develop a more usable and acceptable device.     

Perception of 3-D surfaces from 2-D contours

Inference of 3-D shape from 2-D contours in a single image is an important problem in machine vision. The authors survey classes of techniques proposed in the past and provide a critical analysis. They show that two kinds of symmetries in figures, which are known as parallel and skew symmetries, give significant information about surface shape for a variety of objects. They derive the constraints imposed by these symmetries and show how to use them to infer 3-D shape. They also discuss the zero Gaussian curvature (ZGC) surfaces in depth and show results on the recovery of surface orientation for various ZGC surfaces     

Large displays enhance optical flow cues and narrow the gender gap in 3-D virtual navigation

OBJECTIVE: Existing reports suggest that males significantly outperform females in navigating 3-D virtual environments. Although researchers have recognized that this may be attributable to males and females possessing different spatial abilities, most work has attempted to reduce the gender gap by providing more training for females. In this paper, we explore using large displays to narrow the gender gap within these tasks. BACKGROUND: While evaluating various interaction techniques, we found that large displays affording wider fields of view seemed to improve virtual navigation performance in general and, additionally, to narrow the gender gap that existed on standard desktop displays. METHOD: We conducted two experiments (32 and 22 participants) exploring the individual contributions of display and geometric fields of view to the observed effects as well as isolating factors explaining performance increases seen on the large displays. RESULTS: We show that wider fields of view on large displays not only increase performance of all users on average but also benefit females to such a degree as to allow them to perform as well as males do. We further demonstrate that these benefits can be attributed to better optical flow cues offered by the large displays. CONCLUSION: These findings provide a significant contribution, including recommendations for the improved presentation of 3-D environments, backed by empirical data demonstrating performance benefits during navigation tasks. APPLICATION. Results can be used to design systems that narrow the gender gap in domains such as teleoperation and virtual environments for entertainment, virtual training, or information visualization.