Specific person tracking using 3D LIDAR and ESPAR antenna for mobile service robots

The ability of detecting and following a specific person is indispensable for mobile service robots. Many image-based methods have been proposed for person detection and identification; however, they are sometimes vulnerable to illumination changes. This paper therefore proposes a novel approach to the problem, namely, using 3D LIDARs for person detection and identification and a directivity-controllable antenna (called ESPAR antenna) for localizing a specific person even under long-term occlusion and/or out-of-view situations. A sensor fusion framework, combined with an adaptive state-based strategy switching, has also been developed for achieving a reliable person following. Experimental results in actual outdoor environments show the effectiveness of the proposed framework.     

GidaBot: a system of heterogeneous robots collaborating as guides in multi-floor environments

Intelligent Service Robotics - GidaBot is an application designed to set up and run a heterogeneous team of robots to act as tour guides in multi-floor buildings. Although the tours can go through...     

Vision-based 3-D trajectory tracking for unknown environments

This paper describes a vision-based system for 3-D localization of a mobile robot in a natural environment. The system includes a mountable head with three on-board charge-coupled device cameras that can be installed on the robot. The main emphasis of this paper is on the ability to estimate the motion of the robot independently from any prior scene knowledge, landmark, or extra sensory devices. Distinctive scene features are identified using a novel algorithm, and their 3-D locations are estimated with high accuracy by a stereo algorithm. Using new two-stage feature tracking and iterative motion estimation in a symbiotic manner, precise motion vectors are obtained. The 3-D positions of scene features and the robot are refined by a Kalman filtering approach with a complete error-propagation modeling scheme. Experimental results show that good tracking and localization can be achieved using the proposed vision system.     

3D SSD tracking with estimated 3D planes

We present a tracking method where full camera position and orientation is tracked from intensity differences in a video sequence. The camera pose is calculated based on 3D planes, and hence does not depend on point correspondences. The plane based formulation also allows additional constraints to be naturally added, e.g., perpendicularity between walls, floor and ceiling surfaces, co-planarity of wall surfaces etc. A particular feature of our method is that the full 3D pose change is directly computed from temporal image differences without making a commitment to a particular intermediate (e.g., 2D feature) representation. We experimentally compared our method with regular 2D SSD tracking and found it more robust and stable. This is due to 3D consistency being enforced even in the low level registration of image regions. This yields better results than first computing (and hence committing to) 2D image features and then from these compute 3D pose.     

Multi-target tracking in dense threat environments

A new approach to multi-target tracking is presented for the mid-course stage of the Strategic Defense Initiative (SDI). This approach is based upon a continuum representation of a cluster of flying objects. The velocities of the flying objects are assumed to be embedded into a smooth velocity field. This assumption is based upon the impossibility of encounters in a high-density cluster between the flying objects. Therefore, the problem is reduced to that of identifying a moving continuum based upon consecutive time frame observations. In contradistinction to the previous approaches, here each target is considered as a center of a small continuous neighborhood subjected to a local-affine transformation, and therefore, the target trajectories do not mix. Obviously, their mixture in plane of sensor view is apparent. The approach is illustrated by an example.     

Visual creation of inhabited 3D environments

The creation of virtual reality applications and 3D environments is a complex task that requires good programming skills and expertise in computer graphics and many other disciplines. The complexity increases when we want to include complex entities such as virtual characters and animate them. In this paper we present a system that assists in the tasks of setting up a 3D scene and configuring several parameters affecting the behavior of virtual entities like objects and autonomous virtual humans. Our application is based on a visual programming paradigm, supported by a semantic representation, an ontology for virtual environments. The ontology allows us to store and organize the components of a 3D scene, together with the knowledge associated with them. It is also used to expose functionalities in the given 3D engine. Based on a formal representation of its components, the proposed architecture provides a scalable VR system. Using this system, non-experts can set up interactive scenarios with minimum effort; no programming skills or advanced knowledge is required.     

Semantic visualization of 3D urban environments

The purpose of this work is the semantic visualization of complex 3D city models containing numerous dynamic entities, as well as performing interactive semantic walkthroughs and flights without predefined paths. This is achieved by using a 3D multilayer scene graph that integrates geometric and semantic information as well as by the performance of efficient geometric and what we call semantic view culling. The proposed semantic-geometric scene graph is a 3D structure composed of several layers which is suitable for visualizing geometric data with semantic meaning while the user is navigating inside the 3D city model. BqR-Tree is the data structure specially developed for the geometric layer for the purpose of speeding up rendering time in urban scenes. It is an improved R-Tree data structure based on a quadtree spatial partitioning which improves the rendering speed of the usual R-trees when view culling is implemented in urban scenes. The BqR-Tree is defined by considering the city block as the basic and logical unit. The advantage of the block as opposed to the traditional unit, the building, is that it is easily identified regardless of the data source format, and allows inclusion of mobile and semantic elements in a natural way. The usefulness of the 3D scene graph has been tested with low structured data, which makes its application appropriate to almost all city data containing not only static but dynamic elements as well.     

Viewpoint invariant exemplar-based 3D human tracking

This paper proposes a clustered exemplar-based model for performing viewpoint invariant tracking of the 3D motion of a human subject from a single camera. Each exemplar is associated with multiple view visual information of a person and the corresponding 3D skeletal pose. The visual information takes the form of contours obtained from different viewpoints around the subject. The inclusion of multi-view information is important for two reasons: viewpoint invariance; and generalisation to novel motions. Visual tracking of human motion is performed using a particle filter coupled to the dynamics of human movement represented by the exemplar-based model. Dynamics are modelled by clustering 3D skeletal motions with similar movement and encoding the flow both within and between clusters. Results of single view tracking demonstrate that the exemplar-based models incorporating dynamics generalise to viewpoint invariant tracking of novel movements.     

3D head tracking under partial occlusion

A new algorithm for 3D head tracking under partial occlusion from 2D monocular image sequences is proposed. The extended superquadric (ESQ) is used to generate a geometric 3D face model in order to reduce the shape ambiguity during tracking. Optical flow is then regularized by this model to estimate the 3D rigid motion. To deal with occlusion, a new motion segmentation algorithm using motion residual error analysis is developed. The occluded areas are successfully detected and discarded as noise. Furthermore, accumulation error is heavily reduced by a new post-regularization process based on edge flow. This makes the algorithm more stable over long image sequences. The algorithm is applied to both synthetic occlusion sequence and real image sequences. Comparisons with the ground truth indicate that our method is effective and is not sensitive to occlusion during head tracking.     

Human attributes from 3D pose tracking

It is well known that biological motion conveys a wealth of socially meaningful information. From even a brief exposure, biological motion cues enable the recognition of familiar people, and the inference of attributes such as gender, age, mental state, actions and intentions. In this paper we show that from the output of a video-based 3D human tracking algorithm we can infer physical attributes (e.g., gender and weight) and aspects of mental state (e.g., happiness or sadness). In particular, with 3D articulated tracking we avoid the need for view-based models, specific camera viewpoints, and constrained domains. The task is useful for man–machine communication, and it provides a natural benchmark for evaluating the performance of 3D pose tracking methods (vs. conventional Euclidean joint error metrics). We show results on a large corpus of motion capture data and on the output of a simple 3D pose tracker applied to videos of people walking.     

Dynamic tracking re-adjustment: a method for automatic tracking recovery in complex visual environments

Detection and analysis of events from video sequences is probably one of the most important research issues in computer vision and pattern analysis society. Before, however, applying methods and tools for analyzing actions, behavior or events, we need to implement robust and reliable tracking algorithms able to automatically monitor the movements of many objects in the scene regardless of the complexity of the background, existence of occlusions and illumination changes. Despite the recent research efforts in the field of object tracking, the main limitation of most of the existing algorithms is that they are not enriched with automatic recovery strategies able to re-initialize tracking whenever its performance severely deteriorates. This is addressed in this paper by proposing an automatic tracking recovery tool which improves the performance of any tracking algorithm whenever the results are not acceptable. For the recovery, non-linear object modeling tools are used which probabilistically label image regions to object classes. The models are also time varying. The first property is implemented in our case using concepts from functional analysis which allow parametrization of any arbitrary non-linear function (with some restrictions on its continuity) as a finite series of known functional components but of unknown coefficients. The second property is addressed by proposing an innovative algorithm that optimally estimates the non-linear model at an upcoming time instance based on the current non-linear models that have been already approximated. The architecture is enhanced by a decision mechanism which permits verification of the time instances in which tracking recovery should take place. Experimental results on a set of different video sequences that present complex visual phenomena (full and partial occlusions, illumination variations, complex background, etc) are depicted to demonstrate the efficiency of the proposed scheme in proving tracking in very difficult visual content conditions. Additionally, criteria are proposed to objectively evaluate the tracking performance and compare it with other strategies.     

Grid_strain and grid_strain3: Software packages for strain field computation in 2D and 3D environments

Two Matlab™ software packages for strain field computation, starting from displacements of experimental points (EPs), are here presented. In particular, grid_strain estimates the strain on the nodes of a regular planar grid, whereas grid_strain3 operates on the points of a digital terrain model (DTM). In both cases, the computations are performed in a modified least-square approach, emphasizing the effects of nearest points. This approach allows users to operate at different scales of analysis by introducing a scale factor to reduce or also exclude points too far from grid nodes. The input data are displacements (or velocities) that can be provided by several techniques (e.g. GPS, total topographical station, terrestrial laser scanner). The analysis can be applied to both regional- and local-scale phenomena, to study tectonic crustal deformations (strain ≈10⁻⁸−10⁻⁶) or rapid landslide collapses (10⁻⁴−10²), and to characterize the kinematics of the studied system. Errors on strains and geometric significance of the results are also provided.     

A 2D/3D model-based object tracking framework

This paper presents a robust framework for tracking complex objects in video sequences. Multiple hypothesis tracking (MHT) algorithm reported in (IEEE Trans. Pattern Anal. Mach. Intell. 18(2) (1996)) is modified to accommodate a high level representations (2D edge map, 3D models) of objects for tracking. The framework exploits the advantages of MHT algorithm which is capable of resolving data association/uncertainty and integrates it with object matching techniques to provide a robust behavior while tracking complex objects. To track objects in 2D, a 4D feature is used to represent edge/line segments and are tracked using MHT. In many practical applications 3D models provide more information about the object's pose (i.e., rotation information in the transformation space) which cannot be recovered using 2D edge information. Hence, a 3D model-based object tracking algorithm is also presented. A probabilistic Hausdorff image matching algorithm is incorporated into the framework in order to determine the geometric transformation that best maps the model features onto their corresponding ones in the image plane. 3D model of the object is used to constrain the tracker to operate in a consistent manner. Experimental results on real and synthetic image sequences are presented to demonstrate the efficacy of the proposed framework.     

Improving 3D navigation techniques in multiscale environments: a cubemap-based approach

Navigation in virtual 3D environments, especially those with multiscale features, is still a problem for many users. In this regard, a good design of the navigation interfaces is critical to ensure that the users navigate with the best possible efficiency and comfort. In this paper, we present improvements for two well-known interfaces: fly, including support to collision treatment and automatic navigation speed adjustment in relation to scale, and examine, with automatic pivot point. Such techniques are based on the cubemap structure, which provides information about the surrounding environment at each instant. Usability tests with both beginner and advanced users revealed that the proposed techniques created a significant improvement in the execution of navigation tasks and a reduction in navigation errors.     

Snake terrestrial locomotion synthesis in 3D virtual environments

We present a method for a 3D snake model construction and terrestrial snake locomotion synthesis in 3D virtual environments using image sequences. The snake skeleton is extracted and partitioned into equal segments using a new iterative algorithm for solving the equipartition problem. This method is applied to 3D model construction and at the motion analysis stage. Concerning the snake motion, the snake orientation is controlled by a path planning method. An animation synthesis algorithm, based on a physical motion model and tracking data from image sequences, describes the snake’s velocity and skeleton shape transitions. Moreover, the proposed motion planning algorithm allows a large number of skeleton shapes, providing a general method for aperiodic motion sequences synthesis in any motion graph. Finally, the snake locomotion is adapted to the 3D local ground, while its behavior can be easily controlled by the model parameters yielding the appropriate realistic animations.     

On optimal constrained trajectory planning in 3D environments

A novel approach to generating acceleration-based optimal smooth piecewise trajectories is proposed. Given two configurations (position and orientation) in 3D, we search for the minimal energy trajectory that minimizes the integral of the squared acceleration, opposed to curvature, which is widely investigated. The variation in both components of acceleration: tangential (forces on gas pedal or brakes) and normal (forces that tend to drive a car on the road while making a turn) controls the smoothness of generated trajectories. In the optimization process, our objective is to search for the trajectory along which a free moving robot is able to accelerate (decelerate) to a safe speed in an optimal way. A numerical iterative procedure is devised for computing the optimal piecewise trajectory as a solution of a constrained boundary value problem. The resulting trajectories are not only smooth but also safe with optimal velocity (acceleration) profiles and therefore suitable for robot motion planning applications. Experimental results demonstrate this fact.     

Gamified learning through unity 3D in visualizing environments

Gamified learning has been proven to have the potential for promoting the quality of learning by better engaging students in the learning process. The goal of this study is to create an immersive 3D learning environment where a game is integrated into virtual exhibitions so as to inspire high-school students’ motivation and raise learning performance in humanities courses. In this paper, we developed a framework of virtual exhibitions by combining Unity game engine with a novel 3D JanusVR browser. The survey results from a group of students, whose ages range between 15 and 16, indicate that the proposed virtual exhibitions efficiently capture the attention of students and improve interests to learn history in comparison with traditional teaching and online webpages.

     

Glasses-type wearable computer displays: usability considerations examined with a 3D glasses case study

This study presents usability considerations and solutions for the design of glasses-type wearable computer displays and examines their effectiveness in a case study. Design countermeasures were investigated by a four-step design process: (1) preliminary design analysis; (2) design idea generation; (3) final design selection; and (4) virtual fitting trial. Three design interventions were devised from the design process: (1) weight balance to reduce pressure concentrated on the nose, (2) compliant temples to accommodate diverse head sizes and (3) a hanger mechanism to help spectacle users hang their wearable display on their eye glasses. To investigate their effectiveness, in the case study, the novel 3D glasses adopting the three interventions were compared with two existing 3D glasses in terms of neck muscle fatigue and subjective discomfort rating. While neck muscle fatigue was not significantly different among the three glasses (p = 0.467), the novel glasses had significantly smaller discomfort ratings (p = 0.009).

Relevance to Industry: A four-step design process identified usability considerations and solutions for the design of glasses-type wearable computer displays. A novel 3D glasses was proposed through the process and its effectiveness was validated. The results identify design considerations and opportunities relevant to the emerging wearable display industry.     

Smartphone-based pedestrian tracking in indoor corridor environments

As the use of smartphones spreads rapidly, user localization becomes an important issue for providing diverse location-based services (LBS). While tracking users in outdoor environments is easily done with GPS, the solution for indoor tracking is not trivial. One common technique for indoor user tracking is to employ inertial sensors, but such a system needs to be capable of handling noisy sensors that would normally lead to cumulative locating errors. To reduce such error, additional infrastructure has often been deployed to adjust for these cumulative location errors. As well, previous work has used highly accurate sensors or sensors that are strapped to the body. This paper presents a stand-alone pedestrian tracking system, using only a magnetometer and an accelerometer in a smartphone in indoor corridor environments that are normally laid out in a perpendicular design. Our system provides reasonably accurate pedestrian locations without additional infrastructure or sensors. The experiment results show that the location error is less than approximately 7 m, which is considered adequate for indoor LBS applications.