In-Situ interactive image-based model building for Augmented Reality from a handheld device

Three-dimensional models of objects and their creation process are central for a variety of applications in Augmented Reality. In this article, we present a system that is designed for in-situ modeling using interactive techniques for two generic versions of handheld devices equipped with cameras. The system allows online building of 3D wireframe models through a combination of user interaction and automated methods. In particular, we concentrate in rigorous evaluation of the two devices and interaction methods in the context of 3D feature selection. We present the key components of our system, discuss our findings and results and identify design recommendations.     

Pretreatment of yellow grease for efficient production of fatty acid methyl esters

Biodiesel is a renewable fuel comprised of fatty acid methyl esters (FAME) derived from vegetable oils or animal fats. Comparisons between biodiesel and petroleum-based diesel have shown biodiesel to be effective in reducing exhaust emissions of carbon monoxide, hydrocarbons, particulate matter, and sulfur dioxide. While there are advantages of biodiesel over the traditional petroleum based diesel, biodiesel commercialization is limited by production cost that is dominated by the price of the feedstock (soybean oil). Yellow grease has the potential to be an effective feedstock with lower cost, but the chemical composition of these oils is variable depending on the source of collection and differs from that of virgin oil due to the presence of free fatty acids (FFA). Esterification has been previously demonstrated to reduce the FFA levels of YG; however, large quantities of methanol were required to drive the reaction to high yield. Methanol usage for processing and FFA content are the main factors affecting the economics of FAME production from YG. In this study, the relationship between composition and process variables was systematically studied. The effect of FFA ranging from 2% to 32% (w/w) was studied at three different molar ratios of methanol to FFA (4.5:1, 9:1, 18:1) and was found to have a non-linear relationship. Data obtained from this full factorial screening was used to develop a predictive statistical model to forecast the conversion based on initial FFA level and proportion of alcohol applied for esterification.     

Indoor MAV auto-retrieval using fast 6D relocalisation

This paper develops and evaluates methods for performing auto-retrieval of a micro aerial vehicle (MAV) using fast 6D relocalisation from visual features. Auto-retrieval involves a combination of guided operation to direct the vehicle through obstacles using a human pilot and autonomous operation to navigate the vehicle on its return or during re-exploration. This approach is useful in tasks such as industrial inspection and monitoring, and in particular to operate indoors in GPS-denied environments. Our relocalisation methodology contrasts two sources of information: depth data and feature co-visibility, but in a novel manner that validates matches before a RANSAC procedure. The result is the ability of performing 6D relocalisation at an average of 50 Hz on individual maps containing 120 K features. The use of feature co-visibility reduces memory footprint as well as removes the need to employ depth data as used in previous work. This paper concludes with an example of an industrial application involving visual monitoring from a MAV aided by autonomous navigation.     

Real-time enhancement of sparse 3D maps using a parallel segmentation scheme based on superpixels

In this work, we focus on the problem of feature-based 3D mapping of environments with large textureless regions, which generates sparse 3D maps that may not represent well the mapped scene. To deal with this problem, based on our previous work (Cruz Martinez et al. in 2016 IEEE international symposium on mixed and augmented reality (ISMAR) adjunct proceedings, IEEE, 2016), we propose to enhance sparse 3D maps by using a superpixel-based segmentation with the aim of generating denser 3D maps of the scene in real time. Superpixels are middle-level features, which represent similar regions in an image, which can be connected in order to segment textureless areas. We propose a graphics processor unit architecture for (1) superpixel extraction considering chromatic and depth information, (2) superpixel-based segmentation, generation of connectivity matrix to compute the connected components algorithm and (3) mapping of segmented regions to 3D points. We use the ORB-SLAM system (Mur-Artal et al. in IEEE Trans Robot 31(5):1147–1163, 2015) to generate a sparse 3D map and to project the textureless segments onto it at 27 frames per second. We assessed our approach in terms of segmentation and map quality. Regarding the latter, covered area by the generated map, depth accuracy, and computational performance are reported.     

A topometric system for wide area augmented reality

We describe a system designed to facilitate efficient communication of information relating to the physical world using augmented reality (AR). We bring together a range of technologies to create a system capable of operating in real-time, over wide areas and for both indoor and outdoor operations. The central concept is to integrate localised mapping and tracking based on real-time visual SLAM with global positioning from both GPS and indoor ultra-wide band (UWB) technology. The former allows accurate and repeatable creation and visualisation of AR annotations within local metric maps, whilst the latter provides a coarse global representation of the topology of the maps. We call this a ‘Topometric System’. The key elements are: robust and efficient vision based tracking and mapping using a Kalman filter framework; rapid and reliable vision based relocalisation of users within local maps; user interaction mechanisms for effective annotation insertion; and an integrated framework for managing and fusing mapping and positioning data. We present the results of experiments conducted over a wide area, with indoor and outdoor operations, which demonstrates successful creation and visualisation of large numbers of AR annotations over a range of different locations.     

Interactive 3-D indoor modeler for virtualizing service fields

This paper describes an interactive 3-D indoor modeler that effectively creates photo-realistic 3-D indoor models from multiple photographs. This modeler supports the creation of 3-D models from photographs by implementing interaction techniques that use geometric constraints estimated from photographs and visualization techniques that help to easily understand shapes of 3-D models. We evaluated the availability and usability by applying the modeler to model service fields where actual workers provide services and an experience-based exhibit. Our results confirmed that the modeler enables the creation of large-scale indoor environments such as hot-spring inns and event sites at a relatively modest cost. We also confirmed that school children could learn modeling operations and create 3-D models from a photograph for approximately 20 min because of the easy operations. In addition, we describe additional functions that increase the effectiveness of 3-D modeling based on knowledge from service-field modeling. We present applications for behavior analysis of service workers and for 3-D indoor navigation using augmented virtuality (AV)-based visualization realized by photo-realistic 3-D models.