Real-time Anticipation of Occlusions for Automated Camera Control in Toric Space

Efficient visibility computation is a prominent requirement when designing automated camera control techniques for dynamic 3D environments; computer games, interactive storytelling or 3D media applications all need to track 3D entities while ensuring their visibility and delivering a smooth cinematic experience. Addressing this problem requires to sample a large set of potential camera positions and estimate visibility for each of them, which in practice is intractable despite the efficiency of ray-casting techniques on recent platforms. In this work, we introduce a novel GPU-rendering technique to efficiently compute occlusions of tracked targets in Toric Space coordinates – a parametric space designed for cinematic camera control. We then rely on this occlusion evaluation to derive an anticipation map predicting occlusions for a continuous set of cameras over a user-defined time window. We finally design a camera motion strategy exploiting this anticipation map to minimize the occlusions of tracked entities over time. The key features of our approach are demonstrated through comparison with traditionally used ray-casting on benchmark scenes, and through an integration in multiple game-like 3D scenes with heavy, sparse and dense occluders.     

Extending distortion viewing from 2D to 3D

Addresses the visual exploration of 3D information layouts. Several visual exploration techniques have been proposed for 2D information layouts. Many of these try to take advantage of humans' natural visual pattern-recognition abilities to understand global relationships while simultaneously integrating this knowledge with local details. This desire for detail-in-context views (also called fisheye, multiscale and distortion views) has fueled considerable research in the development of distortion viewing tools. Generally, these tools provide space for magnification of local detail by compressing the rest of the image. In considering a possible detail-in-context view for 3D layouts, we first examine 2D distortion techniques, bearing in mind the particular 3D problem of occlusion. Comparing 2D and 3D information layout adjustment tools leads directly to a 3D visual access tool that clears a line of sight to any region of interest. While our technique can extend to any type of 3D information display, we focus on graphs     

Multiview stereo via volumetric Graph-Cuts and occlusion robust photo-consistency

This paper presents a volumetric formulation for the multi-view stereo problem which is amenable to a computationally tractable global optimisation using Graph-cuts. Our approach is to seek the optimal partitioning of 3D space into two regions labelled as "object" and "empty" under a cost functional consisting of the following two terms: (1) A term that forces the boundary between the two regions to pass through photo-consistent locations and (2) a ballooning term that inflates the "object" region. To take account of the effect of occlusion on the first term we use an occlusion robust photo-consistency metric based on Normalised Cross Correlation, which does not assume any geometric knowledge about the reconstructed object. The globally optimal 3D partitioning can be obtained as the minimum cut solution of a weighted graph.     

A Survey of Radial Methods for Information Visualization

Radial visualization, or the practice of displaying data in a circular or elliptical pattern, is an increasingly common technique in information visualization research. In spite of its prevalence, little work has been done to study this visualization paradigm as a methodology in its own right. We provide a historical review of radial visualization, tracing it to its roots in centuries-old statistical graphics. We then identify the types of problem domains to which modern radial visualization techniques have been applied. A taxonomy for radial visualization is proposed in the form of seven design patterns encompassing nearly all recent works in this area. From an analysis of these patterns, we distill a series of design considerations that system builders can use to create new visualizations that address aspects of the design space that have not yet been explored. It is hoped that our taxonomy will provide a framework for facilitating discourse among researchers and stimulate the development of additional theories and systems involving radial visualization as a distinct design metaphor.     

Generating 3D interaction techniques by identifying and breaking assumptions

Researchers have created 3D interaction techniques for immersive virtual worlds, but existing techniques represent just part of the design space. While exploring other parts of the design space might yield more effective techniques, conducting that exploration is difficult and time-consuming. Analyzing the particular task, user, and hardware characteristics for any given problem is straightforward, but only suggests the shape of a potential technique; generating the technique itself still requires a creative breakthrough. We propose extending existing approaches to generating 3D interaction techniques by focusing more explicitly on identifying and breaking assumptions about the real world to inspire potential technique ideas. We describe our approach, suggest an initial list of assumptions to consider, and present a case study of applying the process to create a technique for navigation with visible landmarks and place representations.     

3D human modeling from a single depth image dealing with self-occlusion

This paper presents a 2D to 3D conversion scheme to generate a 3D human model using a single depth image with several color images. In building a complete 3D model, no prior knowledge such as a pre-computed scene structure and photometric and geometric calibrations is required since the depth camera can directly acquire the calibrated geometric and color information in real time. The proposed method deals with a self-occlusion problem which often occurs in images captured by a monocular camera. When an image is obtained from a fixed view, it may not have data for a certain part of an object due to occlusion. The proposed method consists of following steps to resolve this problem. First, the noise in a depth image is reduced by using a series of image processing techniques. Second, a 3D mesh surface is constructed using the proposed depth image-based modeling method. Third, the occlusion problem is resolved by removing the unwanted triangles in the occlusion region and filling the corresponding hole. Finally, textures are extracted and mapped to the 3D surface of the model to provide photo-realistic appearance. Comparison results with the related work demonstrate the efficiency of our method in terms of visual quality and computation time. It can be utilized in creating 3D human models in many 3D applications.     

Indexing hierarchical structures using graph spectra

Hierarchical image structures are abundant in computer vision and have been used to encode part structure, scale spaces, and a variety of multiresolution features. In this paper, we describe a framework for indexing such representations that embeds the topological structure of a directed acyclic graph (DAG) into a low-dimensional vector space. Based on a novel spectral characterization of a DAG, this topological signature allows us to efficiently retrieve a promising set of candidates from a database of models using a simple nearest-neighbor search. We establish the insensitivity of the signature to minor perturbation of graph structure due to noise, occlusion, or node split/merge. To accommodate large-scale occlusion, the DAG rooted at each nonleaf node of the query "votes" for model objects that share that "part," effectively accumulating local evidence in a model DAG's topological subspaces. We demonstrate the approach with a series of indexing experiments in the domain of view-based 3D object recognition using shock graphs.     

全局遮挡图

基于一种有效的全局可见性的紧凑数据表示形式——全局遮挡图,文章给出了一种新的针对视点活动区域的遮挡剔除算法,对一个视点活动区域,全局遮挡图表示了一组位于空间各个方向上的可见性临界面,这些可见性临界面提供了一个不可见性判据——凡是位于该临界面后的物体必是不可见的,全局遮挡图的优点在于：(1)大小与场景复杂度无关,仅取决于空间方向的离散精度;(2)可用深度图像的方式压缩存储;(3)判断遮挡性时计算量小,同时,文章提出一种自适应构建全局遮挡图的方法,该方法综合利用了图像空间和景物空间技术的优势,适用于普遍的三维场景模型,该文在一个城市场景中对算法效率进行了测试,结果显示遮挡剔除算法可有效地提高绘制效率。     

Physical sketching: Reconstruction and analysis of 3D objects from freehand sketches

When designing a 3D object, designers, engineers and teachers often begin investigating potential design tradeoffs by creating informal sketches. Ideally, these sketches-in combination with a variety of engineering analysis tools-would allow prediction of the object’s physical properties, especially those that affect the critical early design process. We introduce a pen-based system that reconstructs 3D spatial geometry from a single 2D freehand-sketch consisting of straight and curved lines in interactive time. Several optimization-based approaches to this problem have been proposed, but these generally have difficulty converging to an acceptable solution because the dimensionality of the search space is large. The primary contribution of this paper is a new reconstruction algorithm for orthographic projections of 3D wireframes. The algorithm reconstructs the depths of each vertex by exploiting geometric regularities among the graph lines in a reduced solution space, then optimizes a cost function over this space to recover the vertex depths. A second optimization algorithm is used to infer the 3D geometry of curved strokes once the vertex depths have been recovered. The proposed approach can recover the geometry of several objects with approximately 50 curved strokes in near interactive time. We also present an iterative, Tablet-PC-based design system that uses the proposed reconstruction algorithm to recover 3D objects from 2D orthographic sketches. The system allows the reconstructed objects to be subjected to two types of physical analysis, the results of which are superimposed directly on the sketch: a fast, kinematic simulation, and a complete finite-element-based static analysis. The object can quickly be modified in place using the pen-based interface according to the results of the analysis to allow for iterative design work. We demonstrate the system in action on a variety of early-stage design analyses.     

Practical performance models of algorithms in evolutionary program induction and other domains

Evolutionary computation techniques have seen a considerable popularity as problem solving and optimisation tools in recent years. Theoreticians have developed a variety of both exact and approximate models for evolutionary program induction algorithms. However, these models are often criticised for being only applicable to simplistic problems or algorithms with unrealistic parameters. In this paper, we start rectifying this situation in relation to what matters the most to practitioners and users of program induction systems: performance. That is, we introduce a simple and practical model for the performance of program-induction algorithms. To test our approach, we consider two important classes of problems — symbolic regression and Boolean function induction — and we model different versions of genetic programming, gene expression programming and stochastic iterated hill climbing in program space. We illustrate the generality of our technique by also accurately modelling the performance of a training algorithm for artificial neural networks and two heuristics for the off-line bin packing problem.We show that our models, besides performing accurate predictions, can help in the analysis and comparison of different algorithms and/or algorithms with different parameters setting. We illustrate this via the automatic construction of a taxonomy for the stochastic program-induction algorithms considered in this study. The taxonomy reveals important features of these algorithms from the performance point of view, which are not detected by ordinary experimentation.     

Particle filters and occlusion handling for rigid 2D–3D pose tracking

In this paper, we address the problem of 2D–3D pose estimation. Specifically, we propose an approach to jointly track a rigid object in a 2D image sequence and to estimate its pose (position and orientation) in 3D space. We revisit a joint 2D segmentation/3D pose estimation technique, and then extend the framework by incorporating a particle filter to robustly track the object in a challenging environment, and by developing an occlusion detection and handling scheme to continuously track the object in the presence of occlusions. In particular, we focus on partial occlusions that prevent the tracker from extracting an exact region properties of the object, which plays a pivotal role for region-based tracking methods in maintaining the track. To this end, a dynamical choice of how to invoke the objective functional is performed online based on the degree of dependencies between predictions and measurements of the system in accordance with the degree of occlusion and the variation of the object’s pose. This scheme provides the robustness to deal with occlusions of an obstacle with different statistical properties from that of the object of interest. Experimental results demonstrate the practical applicability and robustness of the proposed method in several challenging scenarios.     

Stochastic Approximation and Rate-Distortion Analysis for Robust Structure and Motion Estimation

Recent research on structure and motion recovery has focused on issues related to sensitivity and robustness of existing techniques. One possible reason is that in practical applications, the underlying assumptions made by existing algorithms are often violated. In this paper, we propose a framework for 3D reconstruction from short monocular video sequences taking into account the statistical errors in reconstruction algorithms. Detailed error analysis is especially important for this problem because the motion between pairs of frames is small and slight perturbations in its estimates can lead to large errors in 3D reconstruction. We focus on the following issues: physical sources of errors, their experimental and theoretical analysis, robust estimation techniques and measures for characterizing the quality of the final reconstruction. We derive a precise relationship between the error in the reconstruction and the error in the image correspondences. The error analysis is used to design a robust, recursive multi-frame fusion algorithm using stochastic approximation as the framework since it is capable of dealing with incomplete information about errors in observations. Rate-distortion analysis is proposed for evaluating the quality of the final reconstruction as a function of the number of frames and the error in the image correspondences. Finally, to demonstrate the effectiveness of the algorithm, examples of depth reconstruction are shown for different video sequences.     

Quantifying the impact of uncertainty in embedded systems mapping for NoC based architectures

We describe a modeling framework to capture and account for uncertainty in design parameters in embedded systems. We then develop an uncertainty-aware solution to the problem of mapping in embedded systems that uses Network-on-Chip (NoC) based architecture platforms. The problem of mapping is formulated as a multi-objective - reliability, performance, and energy consumption - optimization problem. To solve this problem, we propose a solution based on the NSGA-II genetic algorithm and Monte Carlo simulation techniques. The solution is implemented as a computer-aid design tool that can generate robust 3D Pareto frontiers in the solution space formed by the design objectives of reliability, performance, and energy consumption. Comparison to several state-of-the-art models and solutions for the mapping problem, indicate that significant differences in the actual values of the design attribute of interest exist when one considers uncertainty in design parameters. For example, in the case of mapping with reliability as the only objective, 10% uncertainty in design parameters can lead to a 10.06% difference in MTTF estimation. In the case of mapping with execution time and energy consumption as objectives, the difference in 2D Pareto frontiers due to 10% uncertainty in design parameters can be up to 7.9%. These differences are important because they can mislead the overall optimization process of mapping toward suboptimal solution points. The DESUU-NOC tool that implements the proposed multi-objective mapping algorithm has as a main feature and contribution of this paper the ability to generate 3D Pareto frontiers comprised of robust solution points.     

A taxonomy for and analysis of tangible interfaces

There have been many research efforts devoted to tangible user interfaces (TUIs), but it has proven difficult to create a definition or taxonomy that allows us to compare and contrast disparate research efforts, integrate TUIs with conventional interfaces, or suggest design principles for future efforts. To address this problem, we present a taxonomy, which uses metaphor and embodiment as its two axes. This 2D space treats tangibility as a spectrum rather than a binary quantity. The further from the origin, the more tangible a system is. We show that this spectrum-based taxonomy offers multiple advantages. It unifies previous categorizations and definitions, integrates the notion of calm computing, reveals a previously un-noticed trend in the field, and suggests design principles appropriate for different areas of the spectrum.

---

A Survey on 3D Virtual Object Manipulation: From the Desktop to Immersive Virtual Environments

Interactions within virtual environments often require manipulating 3D virtual objects. To this end, researchers have endeavoured to find efficient solutions using either traditional input devices or focusing on different input modalities, such as touch and mid‐air gestures. Different virtual environments and diverse input modalities present specific issues to control object position, orientation and scaling: traditional mouse input, for example, presents non‐trivial challenges because of the need to map between 2D input and 3D actions. While interactive surfaces enable more natural approaches, they still require smart mappings. Mid‐air gestures can be exploited to offer natural manipulations mimicking interactions with physical objects. However, these approaches often lack precision and control. All these issues and many others have been addressed in a large body of work. In this article, we survey the state‐of‐the‐art in 3D object manipulation, ranging from traditional desktop approaches to touch and mid‐air interfaces, to interact in diverse virtual environments. We propose a new taxonomy to better classify manipulation properties. Using our taxonomy, we discuss the techniques presented in the surveyed literature, highlighting trends, guidelines and open challenges, that can be useful both to future research and to developers of 3D user interfaces.