M2S maps: supporting real-world navigation with mobile VR

Mobile devices are becoming increasingly integrated into our society. In addition to entertaining people with applications like pervasive games, mobile devices can also help to address cognitive challenges people face in the real world. This paper, by drawing on research findings from cognitive psychology and geography, explores a design to use mobile VR to help people overcome one cognitive barrier in navigation, which is to establish the correspondence between 2D spatial information found in maps and 3D entities they perceive from the real world. The design offers users multi-format, multi-scale, and semantic (M²S) maps, ranging from 2D maps to 3D immersive environments, and helps users to connect 2D maps to the real world through 3D environments which are equipped with semantic representation and animation techniques. Consequently, users can apply various kinds of spatial knowledge, 2D or 3D, in understanding the real world as well as assisting in navigation. This research enhances the design repertoire of mobile VR, and suggests a way to integrate virtual environments into people’s real-world life by examining the cognitive implications of 3D models on users’ activities.     

Provably correct and complete transaction rules for updating 3D city models

The shapes of our cities change very frequently. These changes have to be reflected in data sets representing urban objects. However, it must be assured that frequent updates do not affect geometric-topological consistency. This important aspect of spatial data quality guarantees essential assumptions on which users and applications of 3D city models rely: viz. that objects do not intersect, overlap or penetrate mutually, or completely cover one another. This raises the question how to guarantee that geometric-topological consistency is preserved when data sets are updated. Hence, there is a certain risk that plans and decisions which are based on these data sets are erroneous and that the tremendous efforts spent for data acquisition and updates become vain. In this paper, we solve this problem by presenting efficient transaction rules for updating 3D city models. These rules guarantee that geometric-topological consistency is preserved (Safety) and allow for the generation of arbitrary consistent 3D city models (Completeness). Safety as well as completeness is proven with mathematical rigor, guaranteeing the reliability of our method. Our method is applicable to 3D city models, which define—besides the terrain surface—complex spatial objects like buildings with rooms and storeys as interior structures, as well as bridges and tunnels. Those objects are represented as aggregations of solids, and their surfaces are complex from a topology point of view. 3D GIS models like CityGML, which are widely used to represent cities, provide the means to define semantics, geometry and topology, but do not address the problem of maintaining consistency. Hence, our approach complements CityGML.     

Tangibles for learning: a representational analysis of physical manipulation

Manipulatives—physical learning materials such as cubes or tiles—are prevalent in educational settings across cultures and have generated substantial research into how actions with physical objects may support children’s learning. The ability to integrate digital technology into physical objects—so-called ‘digital manipulatives’—has generated excitement over the potential to create new educational materials. However, without a clear understanding of how actions with physical materials lead to learning, it is difficult to evaluate or inform designs in this area. This paper is intended to contribute to the development of effective tangible technologies for children’s learning by summarising key debates about the representational advantages of manipulatives under two key headings: offloading cognition—where manipulatives may help children by freeing up valuable cognitive resources during problem solving, and conceptual metaphors—where perceptual information or actions with objects have a structural correspondence with more symbolic concepts. The review also indicates possible limitations of physical objects—most importantly that their symbolic significance is only granted by the context in which they are used. These arguments are then discussed in light of tangible designs drawing upon the authors’ current research into tangibles and young children’s understanding of number.     

Inter-Image Statistics for 3D Environment Modeling

In this article we present a method for automatically recovering complete and dense depth maps of an indoor environment by fusing incomplete data for the 3D environment modeling problem. The geometry of indoor environments is usually extracted by acquiring a huge amount of range data and registering it. By acquiring a small set of intensity images and a very limited amount of range data, the acquisition process is considerably simplified, saving time and energy consumption. In our method, the intensity and partial range data are registered first by using an image-based registration algorithm. Then, the missing geometric structures are inferred using a statistical learning method that integrates and analyzes the statistical relationships between the visual data and the available depth on terms of small patches. Experiments on real-world data on a variety of sampling strategies demonstrate the feasibility of our method. Electronic Supplementary Material  The online version of this article () contains supplementary material, which is available to authorized users.     

Dual-Finger 3D Interaction Techniques for mobile devices

Three-dimensional capabilities on mobile devices are increasing, and the interactivity is becoming a key feature of these tools. It is expected that users will actively engage with the 3D content, instead of being passive consumers. Because touch-screens provide a direct means of interaction with 3D content by directly touching and manipulating 3D graphical elements, touch-based interaction is a natural and appealing style of input for 3D applications. However, developing 3D interaction techniques for handheld devices using touch-screens is not a straightforward task. One issue is that when interacting with 3D objects, users occlude the object with their fingers. Furthermore, because the user’s finger covers a large area of the screen, the smallest size of the object users can touch is limited. In this paper, we first inspect existing 3D interaction techniques based on their performance with handheld devices. Then, we present a set of precise Dual-Finger 3D Interaction Techniques for a small display. Finally, we present the results of an experimental study, where we evaluate the usability, performance, and error rate of the proposed and existing 3D interaction techniques.     

Strategies of Multi-view and Multi-matching for 3D Object Recognition

A multi-view representation scheme and a multi-matching strategy for 3D object recognition are described; 3D objects are represented in terms of their 2D appearances so that 2D techniques can be applied to 3D recognition. Appearances of objects in the representation scheme are further organized in a hierarchical manner so that the matching process can reduce its search space by examining only the optimal view at every level of the representation scheme. In our multi-matching strategy, the matching module is composed of four components: point matcher, string matcher, vector matcher, and chamfer matcher. Each matcher is associated with a termination rule so that impossible views can be rejected at the early stages of the matching process. Experimental results reveal that the proposed strategies are feasible for 3D object recognition.     

3D mapping with multi-resolution occupied voxel lists

Most current navigation algorithms in mobile robotics produce 2D maps from data provided by 2D sensors. In large part this is due to the availability of suitable 3D sensors and difficulties of managing the large amount of data supplied by 3D sensors. This paper presents a novel, multi-resolution algorithm that aligns 3D range data stored in occupied voxel lists so as to facilitate the construction of 3D maps. Multi-resolution occupied voxel lists (MROL) are voxel based data structures that efficiently represent 3D scan and map information. The process described in this research can align a sequence of scans to produce maps and localise a range sensor within a prior global map. An office environment (200 square metres) is mapped in 3D at 0.02 m resolution, resulting in a 200,000 voxel occupied voxel list. Global localisation within this map, with no prior pose estimate, is completed in 5 seconds on a 2 GHz processor. The MROL based sequential scan matching is compared to a standard iterative closest point (ICP) implementation with an error in the initial pose estimate of plus or minus 1 m and 90 degrees. MROL correctly scan matches 94% of scans to within 0.1 m as opposed to ICP with 30% within 0.1 m.     

New Information Distance Measure and Its Application in Question Answering System

In a question answering (QA) system,the fundamental problem is how to measure tile distance between a question and an answer,hence ranking different answers.We demonstrate that such a distance can he precisely and mathematically defined.Not only such a definition is possible,it is actually provably better than any other feasible definitions. Not only such an ultimate definition is possible,but also it can be conveniently and fruitfully applied to construct a QA system.We have built such a system——QUANTA.Extensive experiments are conducted to justify the new theory.     

Comparison of 2D and 3D representations for visualising telecommunication usage

This paper describes an empirical evaluation of one two-dimensional (2D), and two three-dimensional (3D) representations. These representations were developed to present customer behaviour information on telecommunications usage. The goal of the study was to investigate how the properties of these different but informationally equivalent representations supported information retrieval and problem solving using the database. Thirty-six participants performed a number of information retrieval and problem solving tasks, in one of three experimental conditions: 2D graph, 3D graph and 3D helix plot. Measures included performance time and accuracy and user attitudes concerning the usability of the displays. Despite certain navigational problems associated with the 2D representation, the results indicated a performance advantage for the 2D display compared with both 3D representations. Generally, the analyses revealed that the differences in representational characteristics have a significant effect on the level of cognitive effort required to perform the tasks.     

3D Surface Reconstruction of Noisy Point Clouds Using Growing Neural Gas: 3D Object/Scene Reconstruction

With the advent of low-cost 3D sensors and 3D printers, scene and object 3D surface reconstruction has become an important research topic in the last years. In this work, we propose an automatic (unsupervised) method for 3D surface reconstruction from raw unorganized point clouds acquired using low-cost 3D sensors. We have modified the growing neural gas network, which is a suitable model because of its flexibility, rapid adaptation and excellent quality of representation, to perform 3D surface reconstruction of different real-world objects and scenes. Some improvements have been made on the original algorithm considering colour and surface normal information of input data during the learning stage and creating complete triangular meshes instead of basic wire-frame representations. The proposed method is able to successfully create 3D faces online, whereas existing 3D reconstruction methods based on self-organizing maps required post-processing steps to close gaps and holes produced during the 3D reconstruction process. A set of quantitative and qualitative experiments were carried out to validate the proposed method. The method has been implemented and tested on real data, and has been found to be effective at reconstructing noisy point clouds obtained using low-cost 3D sensors.     

Representation before computation

My main objective is to point out a fundamental weakness in the conventional conception of computation and suggest a promising way out. This weakness is directly related to a gross underestimation of the role of object representation in a computational model, hence confining such models to an unrealistic (input) environment, which, in turn, leads to “unnatural” computational models. This lack of appreciation of the role of structural object representation has been inherited from logic and partly from mathematics, where, in the latter, the centuries-old tradition is to represent objects as unstructured “points”. I also discuss why the appropriate fundamental reorientation in the conception of computational models will bring the resulting study of computation closer to the “natural” computational constrains. An example of the pertinent, class-oriented, representational formalism developed by our group over many years—Evolving Transformation System (ETS)—is briefly outlined here, and several related general lines of research are suggested.     

3D content-based search using sketches

This paper presents a novel interactive framework for 3D content-based search and retrieval using as query model an object that is dynamically sketched by the user. In particular, two approaches are presented for generating the query model. The first approach uses 2D sketching and symbolic representation of the resulting gestures. The second utilizes non-linear least squares minimization to model the 3D point cloud that is generated by the 3D tracking of the user’s hands, using superquadrics. In the context of the proposed framework, three interfaces were integrated to the sketch-based 3D search system including (a) an unobtrusive interface that utilizes pointing gesture recognition to allow the user manipulate objects in 3D, (b) a haptic–VR interface composed by 3D data gloves and a force feedback device, and (c) a simple air–mouse. These interfaces were tested and comparative results were extracted according to usability and efficiency criteria.     

Consistency constraints and 3D building reconstruction

Virtual architectural (indoor) scenes are often modeled in 3D for various types of simulation systems. For instance, some authors propose methods dedicated to lighting, heat transfer, acoustic or radio-wave propagation simulations. These methods rely in most cases on a volumetric representation of the environment, with adjacency and incidence relationships. Unfortunately, many buildings data are only given by 2D plans and the 3D needs varies from one application to another. To face these problems, we propose a formal representation of consistency constraints dedicated to building interiors and associated with a topological model. We show that such a representation can be used for: (i) reconstructing 3D models from 2D architectural plans (ii) detecting automatically geometrical, topological and semantical inconsistencies (iii) designing automatic and semi-automatic operations to correct and enrich a 2D plan. All our constraints are homogeneously defined in 2D and 3D, implemented with generalized maps and used in modeling operations. We explain how this model can be successfully used for lighting and radio-wave propagation simulations.     

Evaluation for Small Visual Difference Between Conforming Meshes on Strain Field

This paper gives a method of quantifying small visual differences between 3D mesh models with conforming topology, based on the theory of strain fields. Strain field is a geometric quantity in elasticity which is used to describe the deformation of elastomer. In this paper we consider the 3D models as objects with elasticity. The further demonstrations are provided: the first is intended to give the reader a visual impression of how our measure works in practice; and the second is to give readers a visua...     

Guest Editors' Introduction: 3D Documents

Digital libraries (DLs) in general and technical or cultural preservation applications in particular offer a rich set of multimedia objects like audio, music, images, videos, and also 3D models. However, instead of handling these objects consistently as regular documents - in the same way we treat textual documents - most applications handle them differently. Considering that textual documents are only one media type among many, it's clear that this type of document is handled quite specially. A full-text search engine lets users retrieve a specific document based on its content - that is, one or more words that appear in it. Content-based retrieval of other media types is an active research area, and in the case of 3D documents, only pilot applications exist.     

An automatic 2D to 3D video conversion approach based on RGB-D images

3D movies/videos have become increasingly popular in the market; however, they are usually produced by professionals. This paper presents a new technique for the automatic conversion of 2D to 3D video based on RGB-D sensors, which can be easily conducted by ordinary users. To generate a 3D image, one approach is to combine the original 2D color image and its corresponding depth map together to perform depth image-based rendering (DIBR). An RGB-D sensor is one of the inexpensive ways to capture an image and its corresponding depth map. The quality of the depth map and the DIBR algorithm are crucial to this process. Our approach is twofold. First, the depth maps captured directly by RGB-D sensors are generally of poor quality because there are many regions missing depth information, especially near the edges of objects. This paper proposes a new RGB-D sensor based depth map inpainting method that divides the regions with missing depths into interior holes and border holes. Different schemes are used to inpaint the different types of holes. Second, an improved hole filling approach for DIBR is proposed to synthesize the 3D images by using the corresponding color images and the inpainted depth maps. Extensive experiments were conducted on different evaluation datasets. The results show the effectiveness of our method.