The virtual landing pad: facilitating rotary-wing landing operations in degraded visual environments

The safety of rotary-wing operations is significantly affected by the local weather conditions, especially during key phases of flight including hover and landing. Despite the operational flexibility of rotary-wing craft, such craft accounts for a significantly greater proportion of accidents than their fixed-wing counterparts. A key period of risk when operating rotary-wing aircraft is during operations that occur in degraded visual environments, for example as a result of thick fog. During such conditions, pilots’ workload significantly increases and their situation awareness can be greatly impeded. The current study examines the extent to which providing information to pilots via the use of a head-up display (HUD) influenced perceived workload and situation awareness, when operating in both clear and degraded visual environments. Results suggest that whilst the HUD did not benefit pilots during clear conditions, workload was reduced when operating in degraded visual conditions. Overall results demonstrate that access to the HUD reduces the difficulties associated with flying in degraded visual environments.     

RTTES: Real-time search in dynamic environments

In this paper we propose a real-time search algorithm called Real-Time Target Evaluation Search (RTTES) for the problem of searching a route in grid worlds from a starting point to a static or dynamic target point in real-time. The algorithm makes use of a new effective heuristic method which utilizes environmental information to successfully find solution paths to the target in dynamic and partially observable environments. The method requires analysis of nearby obstacles to determine closed directions and estimate the goal relevance of open directions in order to identify the most beneficial move. We compared RTTES with other competing real-time search algorithms and observed a significant improvement on solution quality.     

Real-time hierarchical stereo Visual SLAM in large-scale environments

In this paper we present a new real-time hierarchical (topological/metric) Visual SLAM system focusing on the localization of a vehicle in large-scale outdoor urban environments. It is exclusively based on the visual information provided by a cheap wide-angle stereo camera. Our approach divides the whole map into local sub-maps identified by the so-called fingerprints (vehicle poses). At the sub-map level (low level SLAM), 3D sequential mapping of natural landmarks and the robot location/orientation are obtained using a top-down Bayesian method to model the dynamic behavior. A higher topological level (high level SLAM) based on fingerprints has been added to reduce the global accumulated drift, keeping real-time constraints. Using this hierarchical strategy, we keep the local consistency of the metric sub-maps, by mean of the EKF, and global consistency by using the topological map and the MultiLevel Relaxation (MLR) algorithm. Some experimental results for different large-scale outdoor environments are presented, showing an almost constant processing time.     

Real-time scheduling of divisible loads in cluster computing environments

Cluster computing has become an important paradigm for solving large-scale problems. To enhance the quality of service (QoS) and provide performance guarantees in a cluster computing environment, various real-time scheduling algorithms and workload models have been investigated. Computational loads that can be arbitrarily divided into independent tasks represent many real-world applications. However, the problem of providing performance guarantees to divisible load applications has only recently been studied systematically. In this work, three important and necessary design decisions, (1) workload partitioning, (2) node assignment, and (3) task execution order, are identified for real-time divisible load scheduling. A scheduling framework that can configure different policies for each of the three design decisions is proposed and used to generate various algorithms. This paper systematically studies these algorithms and identifies scenarios where the choices of design parameters have significant effects.     

Construction of the initial deorbit region for landing at the given place

The algorithm of the two-point boundary problem is developed for choice the nominal deorbit point, from which a descent vehicle is enabled to land at a given place without a cross-range maneuver. The method is suggested and the example is given of constructing the initial deorbit region for landing at the Vostochny cosmodrome.     

Analyzing place metaphors in 3D educational collaborative virtual environments

In the recent years, the usage of three dimensional (3D) collaborative virtual environments (CVEs) for educational purposes has increased. The metaphors behind the design of virtual places are quite diverse, from replication of real universities to art museums and scientific labs. This paper reports the results of a case study where the students of our university, as a part of their course assignment, analyzed place metaphors used in a range of 3D educational CVEs vs. the corresponding educational goals. The students suggested a design for a virtual campus representing the Norwegian University of Science and Technology (NTNU). The results of this study provide some suggestions concerning the characterization of different design features in educational CVEs and the suitability of such features for different educational goals. Also, a preliminary set of design guidelines for an ideal virtual campus representing a real university is presented. Finally, some challenges associated with using 3D CVEs in various educational situations are discussed.     

Real-time fault detection in manufacturing environments using face recognition techniques

New image processing techniques as well digital image capture equipment provide an opportunity for fast detection and diagnosis of quality problems in manufacturing environments compared with traditional dimensional measurement techniques. This paper proposes a new use of image processing to detect in real-time quality faults using images traditionally obtained to guide manufacturing processes. The proposed method utilizes face recognition tools to eliminate the need of specific feature detection on determining out-of-specification parts. The focus of the proposed methodology is on computational efficiency to ensure that the algorithm runs in real time in high volume manufacturing environments. The algorithm is trained with previously classified images. New images are then classified into two groups, healthy and unhealthy. This paper proposes a method that combines Discrete Cosine Transform with Fisher’s Linear Discriminant Analysis to detect faults, such as cracks, directly from aluminum stamped parts.     

Real-time 6-DOF multi-session visual SLAM over large-scale environments

This paper describes a system for performing real-time multi-session visual mapping in large-scale environments. Multi-session mapping considers the problem of combining the results of multiple simultaneous localisation and mapping (SLAM) missions performed repeatedly over time in the same environment. The goal is to robustly combine multiple maps in a common metrical coordinate system, with consistent estimates of uncertainty. Our work employs incremental smoothing and mapping (iSAM) as the underlying SLAM state estimator and uses an improved appearance-based method for detecting loop closures within single mapping sessions and across multiple sessions. To stitch together pose graph maps from multiple visual mapping sessions, we employ spatial separator variables, called anchor nodes, to link together multiple relative pose graphs.The system architecture consists of a separate front-end for computing visual odometry and windowed bundle adjustment on individual sessions, in conjunction with a back-end for performing the place recognition and multi-session mapping. We provide experimental results for real-time multi-session visual mapping on wheeled and handheld datasets in the MIT Stata Center. These results demonstrate key capabilities that will serve as a foundation for future work in large-scale persistent visual mapping.     

Real-time exploration and photorealistic reconstruction of large natural environments

This paper presents a hybrid (geometry- and image-based) framework suitable for providing photorealistic walkthroughs of large, complex outdoor scenes, based only on a small set of real images from the scene. To this end, a novel data representation of a 3D scene is proposed, which is called morphable 3D panoramas. Motion is assumed to be taking place along a predefined path of the 3D environment and the input to the system is a sparse set of stereoscopic views at certain positions (key positions) along that path (one view per position). An approximate local 3D model is constructed from each view, capable of capturing the photometric and geometric properties of the scene only locally. Then, during the rendering process, a continuous morphing (both photometric as well as geometric) takes place between successive local 3D models, using what we call a ‘morphable 3D model’. For the estimation of the photometric morphing, a robust algorithm capable of extracting a dense field of 2D correspondences between wide-baseline images is used, whereas, for the geometric morphing, a novel method of computing 3D correspondences between local models is proposed. In this way, a physically valid morphing is always produced, which is thus kept transparent from the user. Moreover, a highly optimized rendering path is used during morphing. Thanks to the use of appropriate pixel and vertex shaders, this rendering path can be run fully in 3D graphics hardware and thus allows for high frame rates. Our system can be extended to handle multiple stereoscopic views (and therefore multiple local models) per key position of the path (related by a camera rotation). In this case, one local 3D panorama (per key position) is constructed, comprising all local 3D models therein, and so a ‘morphable 3D panorama’ is now used during the rendering process. For handling the geometric consistency of each 3D panorama, a technique which is based on solving a partial differential equation is adopted. The effectiveness of our framework is demonstrated by using it for the 3D visual reconstruction of the Samaria Gorge in Crete.     

Real-time path planning in dynamic virtual environments using multiagent navigation graphs

We present a novel approach for efficient path planning and navigation of multiple virtual agents in complex dynamic scenes. We introduce a new data structure, Multi-agent Navigation Graph (MaNG), which is constructed using first- and second-order Voronoi diagrams. The MaNG is used to perform route planning and proximity computations for each agent in real time. Moreover, we use the path information and proximity relationships for local dynamics computation of each agent by extending a social force model [Helbing05]. We compute the MaNG using graphics hardware and present culling techniques to accelerate the computation. We also address undersampling issues and present techniques to improve the accuracy of our algorithm. Our algorithm is used for real-time multi-agent planning in pursuit-evasion, terrain exploration and crowd simulation scenarios consisting of hundreds of moving agents, each with a distinct goal.     

Resource failures risk assessment modelling in distributed environments

Service providers offer access to resources and services in distributed environments such as Grids and Clouds through formal Service level Agreements (SLA), and need well-balanced infrastructures so that they can maximise the Quality of Service (QoS) they offer and minimise the number of SLA violations. We propose a mathematical model to predict the risk of failure of resources in such environments using a discrete-time analytical model driven by reliability functions fitted to observed data. The model relies on the resource historical data so as to predict the risk of failure for a given time interval. The model is evaluated by comparing the predicted risk of failure with the observed risk of failure, and is shown to accurately predict the resources risk of failure, allowing a service provider to selectively choose which SLA request to accept.     

Real-time stress assessment using thermal imaging

Thermal imaging is one of the most promising methods of probing the psychological status of human beings because of its non-invasiveness. This study develops a new method of assessing the stress status in real time. The differential energy between philtrum (located in the maxillary area) and forehead (DEFP) algorithm is developed to amplify and extract stress-induced thermal imprints. The algorithm is then validated against the clinical standard in a controlled lab experiment in which neurophysiologic responses are invoked from subjects via Trier Social Stress Test. The correlation between the extracted thermal imprints and established stress markers (heart beat rate and cortisol level) is significant. Experimental result demonstrates that DEFP has the capacity to classify stress and baseline status. An accuracy rate of over 90 % suggests the feasibility of the real-time assessment of stress, disregarding personal factors.     

Real-time deep satellite image quality assessment

A method for deep satellite image quality assessment based on no-reference satellite images is proposed. We design suitable deep convolutional neural networks, which are named satellite image quality assessment of deep convolutional neural networks (SIQA-DCNN) and SIQA-DCNN++. These sophisticated methods can remove various distorted satellite images in real-time remote sensing. The novelty of this method lies in the objective assessment and restoration of the deep model which understands various distorted satellite images in high- and low-resolution problems. The activation function has a lower computational time and ensures the deactivation of noise by making the mean activators close to zero. Our methods are also effective for transfer learning, which can be used to adequately investigate satellite image classification in deep satellite image quality assessment. Using Spearman’s rank order correlation coefficient (SROCC) and linear correlation coefficient (LCC) evaluations, we demonstrated that our methods show better performance than other algorithms, with more than 0.90 of SROCC and LCC values compared to the full-reference and no-reference satellite image in MODIS/Terra and USGS datasets. Regarding computational complexity, we obtained better performance in operational function times. As compared to other methods, SIQA-DCNN and SIQA-DCNN++ also reduced computational time by more than 40 and 56%, respectively, when applied to the USGS dataset, and by more than 46 and 60% respectively, when applied to the MODIS/Terra dataset.     

Real-time scheduling algorithm for safety-critical systems on faulty multicore environments

An algorithm (called FTM) for scheduling of real-time sporadic tasks on a multicore platform is proposed. Each task has a deadline by which it must complete its non-erroneous execution. The FTM algorithm executes backups in order to recover from errors caused by non-permanent and permanent hardware faults. The worst-case schedulability analysis of FTM algorithm is presented considering an application-level error model, which is independent of the stochastic behavior of the underlying hardware-level fault model. Then, the stochastic behavior of hardware-level fault model is plugged in to the analysis to derive the probability of meeting all the deadlines. Such probabilistic guarantee is the level of assurance (i.e., reliability) regarding the correct functional and timing behaviors of the system. One of the salient features of FTM algorithm is that it executes some backups in active redundancy to exploit the parallel multicore architecture while other backups passively to avoid unnecessary execution of too many active backups. This paper also proposes a scheme to determine for each task the number of backups that should run in active redundancy in order to increase the probability of meeting all the deadlines. The effectiveness of the proposed approach is demonstrated using an example application.     

Real-time control architecture using Xenomai for intelligent service robots in USN environments

This paper describes the implementation of a dual-kernel software architecture, based on standard Linux and real-time embedded Linux, for real-time control of service robots in ubiquitous sensor network environments. Mobile robots are used in active service for the assisted living of elderly people, monitoring their mental and physiological data with wireless sensor nodes. The data collected from sensor nodes are routed back to a sink node through multi-hop communication. The moving sink node installed on the main controller of the robot collects data and transmits it to the main controller. To be able to handle emergency situations, the robot needs to satisfy real-time requirements when processing the data collected, and invoking tasks to execute. This paper realizes a multi-hop sensor network and proposes real-time software architecture based on Xenomai. The real-time tasks were implemented, with priority, to rapidly respond to urgent sensor data. In order to validate the deterministic response of the proposed system, the performance measurements for the delay in handling the sensed data transmission and the trajectory control with a feedback loop were evaluated on the non real-time standard Linux.     

Real-time tracking of humans and visualization of their future footsteps in public indoor environments

In this work, an interactive entertainment system which employs multiple-human tracking from a single camera is presented. The proposed system robustly tracks people in an indoor environment and displays their predicted future footsteps in front of them in real-time. The system is composed of a video camera, a computer and a projector. There are three main modules: tracking, analysis and visualization. The tracking module extracts people as moving blobs by using an adaptive background subtraction algorithm. Then, the location and orientation of their next footsteps are predicted. The future footsteps are visualized by a high-paced continuous display of foot images in the predicted location to simulate the natural stepping of a person. To evaluate the performance, the proposed system was exhibited during a public art exhibition in an airport. People showed surprise, excitement, curiosity. They tried to control the display of the footsteps by making various movements.     

Subjective and objective quality assessment of videos in error-prone network environments

Compression and transmission are two fundamental stages involved in wireless video communications, each of which may cause degradation of the quality of experience (QoE) of end users by producing compression artifacts and packet loss artifacts, respectively. They have their own unique perceptual influences. To provide insight for designing QoE-aware content delivery applications, this paper studies subjective and objective quality of videos containing both types of artifacts. First, subjective quality assessment is conducted, from which interaction between the two types of artifacts during quality perception is investigated. Second, using the subjective data, the performance of the state-of-the-art objective quality metrics is evaluated, with the aim of examining suitability of the existing metrics for their use in error-prone video communication applications. Finally, the developed data set is made publicly available for the community.     

Real-time assessment of bone structure positions via ultrasound imaging

Computer-assisted orthopedic surgery allows clinicians to have better results and decreases the number of early prosthetic replacements. Nevertheless, the patient follow-up from pre-operative diagnosis to post-operative control cannot be assessed in a constant referential. In this paper, a real-time algorithm that extracts bone edges from images and, then, derives bony landmarks from these edges is proposed. Indeed, we assess in real-time the bone structure positions via ultrasound imaging to create a useful referential for pre-operative, intra-operative and post-operative measurements. To assist the clinician while acquiring bony anatomical landmarks, the extraction of the bone–soft tissue interface and bony landmarks from ultrasound images is done automatically. The experimentations were performed on a database of images from healthy volunteers, and the obtained results showed the efficiency and the stability of the performance of the proposed method.     

A simplified approach to assessment of mission success for helicopter landing on a ship

International Journal of Control, Automation and Systems - In this paper, operation of a helicopter near a ship is statistically investigated to provide an estimate of the probability of...     

Afforded actions as a behavioral assessment of physical presence in virtual environments

A particular affordance was used as a potential candidate for behavioral assessment of physical presence in virtual environments. The subjects’ task was to walk through a virtual aperture of variable widths. In the case of presence, the subjects’ body orientation, while walking, was hypothesized to be adapted to the width of the aperture and to their own shoulder width. Results show that most subjects adapted their behavior to both their body architecture and the virtual width constraints. These subjects exhibited a behavioral transition from frontal walking to body rotation while walking through broad to narrow apertures. The same behavioral transition has already been documented in real environments (Warren and Whang in J Exp Psychol Human Percept Perform 13(3):371–383, 1987). This behavioral adjustment is thus assumed to be an objective indication of presence. Beyond these results, the present study suggests that every afforded action could be a potential tool for sensorimotor assessment of physical presence. Parts of this research have been first presented at the 11th Annual International Workshop of Presence, Padova, Italy, 16-18 October 2008.