Accurate 3D-vision-based obstacle detection for an autonomous train

In this paper we present a 3D-vision based obstacle detection system for an autonomously operating train in open terrain environments. The system produces dense depth data in real-time from a stereo camera system with a baseline of 1.4 m to fulfill accuracy requirements for reliable obstacle detection 80 m ahead. On an existing high speed stereo engine, several modifications have been applied to significantly improve the overall performance of the system. Hierarchical stereo matching and slanted correlation masks increased the quality of the depth data in a way that the obstacle detection rate increased from 89.4% to 97.75% while the false positive detection rate could be kept as low as 0.25%. The evaluation results have been obtained from extensive real-world test data. An additional stereo matching speed-up of factor 2.15 was achieved and the overall latency of obstacle detection is considerably faster than 300 ms.     

Assessment of a simple obstacle detection device for the visually impaired

A simple obstacle detection device, based upon an automobile parking sensor, was assessed as a mobility aid for the visually impaired. A questionnaire survey for mobility needs was performed at the start of this study. After the detector was developed, five blindfolded sighted and 15 visually impaired participants were invited to conduct travel experiments under three test conditions: (1) using a white cane only, (2) using the obstacle detector only and (3) using both devices. A post-experiment interview regarding the usefulness of the obstacle detector for the visually impaired participants was performed. The results showed that the obstacle detector could augment mobility performance with the white cane. The obstacle detection device should be used in conjunction with the white cane to achieve the best mobility speed and body protection.     

Proximal object and hazard detection for autonomous underwater vehicle with optical fibre sensors

This paper describes short range and tactile optical fibre sensors for marine applications. The sensors are designed for obstacle avoidance on unmanned underwater vehicles (UUVs) operating in confined spaces, but have other possible applications. The fibre sensors augment the sensory abilities derived from ultrasonic and other sensors employed for marine proximity measurement. Of particular interest is proximity detection in the “near” (less than 1 m) and tactile areas. The paper describes the basic principle of operation and alternative sensor configurations. Results are given based on laboratory tests and deployment on a mini autonomous submersible in a test pool.     

Uncalibrated obstacle detection using normal flow

This paper addresses the problem of obstacle detection for mobile robots. The visual information provided by a single on-board camera is used as input. We assume that the robot is moving on a planar pavement, and any point lying outside this plane is treated as an obstacle. We address the problem of obstacle detection by exploiting the geometric arrangement between the robot, the camera, and the scene. During an initialization stage, we estimate an inverse perspective transformation that maps the image plane onto the horizontal plane. During normal operation, the normal flow is computed and inversely projected onto the horizontal plane. This simplifies the resultant flow pattern, and fast tests can be used to detect obstacles. A salient feature of our method is that only the normal flow information, or first order time-and-space image derivatives, is used, and thus we cope with the aperture problem. Another important issue is that, contrasting with other methods, the vehicle motion and intrinsic and extrinsic parameters of the camera need not be known or calibrated. Both translational and rotational motion can be dealt with. We present motion estimation results on synthetic and real-image data. A real-time version implemented on a mobile robot, is described.     

Navigation of an AUV for investigation of underwater structures

There is a great demand for autonomous underwater vehicles (AUVs) to investigate artificial underwater structures such as piles and caissons in harbours, and risers and jackets of deep-sea oilfields. This paper proposes an autonomous investigation method of underwater structures using AUVs that is implemented by initially detecting the target objects, localizing them, then approaching them by taking video images while closely tracing their shape. A laser ranging system and a navigation method based on the relative position with respect to the target objects are introduced to realize this behaviour.     

Multimodal obstacle detection in unstructured environments with conditional random fields

Reliable obstacle detection and classification in rough and unstructured terrain such as agricultural fields or orchards remains a challenging problem. These environments involve large variations in both geometry and appearance, challenging perception systems that rely on only a single sensor modality. Geometrically, tall grass, fallen leaves, or terrain roughness can mistakenly be perceived as nontraversable or might even obscure actual obstacles. Likewise, traversable grass or dirt roads and obstacles such as trees and bushes might be visually ambiguous. In this paper, we combine appearance‐ and geometry‐based detection methods by probabilistically fusing lidar and camera sensing with semantic segmentation using a conditional random field. We apply a state‐of‐the‐art multimodal fusion algorithm from the scene analysis domain and adjust it for obstacle detection in agriculture with moving ground vehicles. This involves explicitly handling sparse point cloud data and exploiting both spatial, temporal, and multimodal links between corresponding 2D and 3D regions. The proposed method was evaluated on a diverse data set, comprising a dairy paddock and different orchards gathered with a perception research robot in Australia. Results showed that for a two‐class classification problem (ground and nonground), only the camera leveraged from information provided by the other modality with an increase in the mean classification score of 0.5%. However, as more classes were introduced (ground, sky, vegetation, and object), both modalities complemented each other with improvements of 1.4% in 2D and 7.9% in 3D. Finally, introducing temporal links between successive frames resulted in improvements of 0.2% in 2D and 1.5% in 3D.     

Hierarchical adaptive stereo matching algorithm for obstacle detection with dynamic programming

An adaptive weighted stereo matching algorithm with multilevel and bidirectional dynamic programming based on ground control points （GCPs） is presented. To decrease time complexity without losing matching precision, using a multilevel search scheme, the coarse matching is processed in typical disparity space image, while the fine matching is processed in disparity-offset space image. In the upper level, GCPs are obtained by enhanced volumetric iterative algorithm enforcing the mutual constraint and the threshold constraint. Under the supervision of the highly reliable GCPs, bidirectional dynamic programming framework is employed to solve the inconsistency in the optimization path. In the lower level, to reduce running time, disparity-offset space is proposed to efficiently achieve the dense disparity image. In addition, an adaptive dual support-weight strategy is presented to aggregate matching cost, which considers photometric and geometric information. Further, post-processing algorithm can ameliorate disparity results in areas with depth discontinuities and related by occlusions using dual threshold algorithm, where missing stereo information is substituted from surrounding regions. To demonstrate the effectiveness of the algorithm, we present the two groups of experimental results for four widely used standard stereo data sets, including discussion on performance and comparison with other methods, which show that the algorithm has not only a fast speed, but also significantly improves the efficiency of holistic optimization.     

Analysis of natural images processing for the extraction of agricultural elements

This work presents several developed computer-vision-based methods for the estimation of percentages of weed, crop and soil present in an image showing a region of interest of the crop field. The visual detection of weed, crop and soil is an arduous task due to physical similarities between weeds and crop and to the natural and therefore complex environments (with non-controlled illumination) encountered. The image processing was divided in three different stages at which each different agricultural element is extracted: (1) segmentation of vegetation against non-vegetation (soil), (2) crop row elimination (crop) and (3) weed extraction (weed). For each stage, different and interchangeable methods are proposed, each one using a series of input parameters which value can be changed for further refining the processing. A genetic algorithm was then used to find the best value of parameters and method combination for different sets of images. The whole system was tested on several images from different years and fields, resulting in an average correlation coefficient with real data (bio-mass) of 84%, with up to 96% correlation using the best methods on winter cereal images and of up to 84% on maize images. Moreover, the method’s low computational complexity leads to the possibility, as future work, of adapting them to real-time processing.     

A sonar approach to obstacle detection for a vision-based autonomous wheelchair

An advanced prototype Computer Controlled Power Wheelchair Navigation System or CCPWNS has been developed to provide autonomy for highly disabled users, whose mix of disabilities makes it difficult or impossible to control their own power chairs in their homes. The working paradigm is “teach and repeat” a mode of control for typical industrial holonomic robots. Ultrasound sensors, which during subsequent autonomous tracking will be used to detect obstacles, also are active during teaching. Based upon post-processed data collected during this teaching event, elaborate trajectories–which may involve multiple direction changes, pivoting and so on, depending upon the requirements of the typically restricted spaces within which the chair must operate–will later be called upon by the disabled rider. An off-line postprocessor assigns an ultrasound profile to the sequence of poses of any taught trajectory. Use of this profile during tracking obviates most of the inherent problems of using ultrasound to avoid obstacles while retaining the ability to near solid objects, such as when passing through a narrow doorway, where required by the environment and trajectory objectives. The work in this article describes a procedure to obtain consistent maps of sonar boundaries during the teaching process, and a preliminary approach to use this information during the tracking phase. The approach is illustrated by results obtained by using the CCPWNS prototype.     

一种障碍物模型检测算法

针对观测场景结构复杂、建模困难的问题,提出了一种基于运动目标跟踪的场景障碍物模型检测方法.结合对目标的检测与跟踪过程,利用空间投影关系确定场景中障碍物的位置,并将其映射到场景模型上,即可得到场景观测模型.该方法只需要对目标运动进行简单地跟踪和统计,不需要进行三维结构恢复以及图像拼接处理,能够有效解决场景的在线自适应建模问题.     

Mission design for a group of autonomous guided vehicles

In this paper, a generic approach for the integration of vehicle routing and scheduling and motion planning for a group of autonomous guided vehicles (AGVs) is proposed. The AGVs are requested to serve all the work stations in a two-dimensional environment while taking into account kinematics constraints and the environment’s geometry during their motion. The problem objective is the simultaneous determination of time-optimum and collision-free paths for all AGVs. The proposed method is investigated and discussed through a number of simulated experiments using a variety of environments and different initial conditions.     

一种快速新闻视频标题字幕探测与定位方法*

新闻视频字幕包含有丰富的语义信息,尤其是标题字幕,对新闻视频高层语义内容的分析和理解具有 重要作用。利用标题字幕的时空分布特征,提出了一个新闻视频标题字幕的快速探测与定位方法。首先利用标 题字幕持续多帧出现的特点降低所需处理的帧数,然后基于标题字幕的边缘特征和位置特征,标记帧图像的候 选字幕块,对帧序列中的图像进行统计分析,探测出视频中标题字幕的位置及出现消失时间。实验结果表明所 提方法简单有效,能够快速、鲁棒地探测并定位新闻视频中的标题字幕。     

Using obstacle analysis to identify contingency requirements on an unpiloted aerial vehicle

This paper describes the use of Obstacle Analysis to identify anomaly handling requirements for a safety-critical, autonomous system. The software requirements for the system evolved during operations due to an on-going effort to increase the autonomous system’s robustness. The resulting increase in autonomy also increased system complexity. This investigation used Obstacle Analysis to identify and to reason incrementally about new requirements for handling failures and other anomalous events. Results reported in the paper show that Obstacle Analysis complemented standard safety-analysis techniques in identifying undesirable behaviors and ways to resolve them. The step-by-step use of Obstacle Analysis identified potential side effects and missing monitoring and control requirements. Adding an Availability Indicator and feature-interaction patterns proved useful for the analysis of obstacle resolutions. The paper discusses the consequences of these results in terms of the adoption of Obstacle Analysis to analyze anomaly handling requirements in evolving systems.     

Combining experts for anchorperson shot detection in news videos

Automatic classification of shots extracted by news videos plays an important role in the context of news video segmentation, which is an essential step towards effective indexing of broadcasters digital databases. In spite of the efforts reported by the researchers involved in this field, no techniques providing fully satisfactory performance have been presented until now. In this paper, we propose a multi-expert approach for unsupervised shot classification. The proposed multi-expert system (MES) combines three algorithms that are model-free and do not require a specific training phase. In order to assess the performance of the MES, we built up a database significantly wider than those typically used in the field. Experimental results demonstrate the effectiveness of the proposed approach both in terms of shot classification and of news story detection capability.     

Highlights modeling and detection in sports videos

Automatic annotation of semantic events allows effective retrieval of video content. In this work, we present solutions for highlights detection in sports videos. This application is particularly interesting for broadcasters, since they extensively use manual annotation to select interesting highlights that are edited to create new programmes. The proposed approach exploits the typical structure of a wide class of sports videos, namely, those related to sports which are played in delimited venues with playfields of well known geometry, like soccer, basketball, swimming, track and field disciplines, and so on. For this class of sports, a modeling scheme based on a limited set of visual cues and on finite state machines (FSM) that encode the temporal evolution of highlights is presented. Algorithms for model checking and for visual cues estimation are discussed, as well as applications of the representation to different sport domains.     

iSurveillance: Intelligent framework for multiple events detection in surveillance videos

Research in the video surveillance is gaining more popularity due to its widespread applications as well as social impact. In this paper, we present an intelligent framework for detection of multiple events in surveillance videos. Based on the principle of compositionality, we modularize the surveillance problems into a set of variables comprising regions-of-interest, classes (i.e. human, vehicle), attributes (i.e. speed, locality) and a set of notions (i.e. rules) associated to each of the attributes to construct a knowledge-based understanding of the environment. The final output from the reasoning process, which combines the definition domains of the various variables, allows a broader and integrated understanding of complex pattern of activities in the scene. This is in contrast to the state-of-the-art solutions that are only able to perform only a singular task, at a time. Experimental results on both the public and real-time datasets have demonstrated the effectiveness and robustness of the proposed framework in detecting multiple events in surveillance videos.     

Combining appearance and motion for face and gender recognition from videos

While many works consider moving faces only as collections of frames and apply still image-based methods, recent developments indicate that excellent results can be obtained using texture-based spatiotemporal representations for describing and analyzing faces in videos. Inspired by the psychophysical findings which state that facial movements can provide valuable information to face analysis, and also by our recent success in using LBP (local binary patterns) for combining appearance and motion for dynamic texture analysis, this paper investigates the combination of facial appearance (the shape of the face) and motion (the way a person is talking and moving his/her facial features) for face analysis in videos. We propose and study an approach for spatiotemporal face and gender recognition from videos using an extended set of volume LBP features and a boosting scheme. We experiment with several publicly available video face databases and consider different benchmark methods for comparison. Our extensive experimental analysis clearly assesses the promising performance of the LBP-based spatiotemporal representations for describing and analyzing faces in videos.     

双足机器人实时障碍检测视觉系统

提出一种双足步行机器人的实时障碍检测视觉系统．基于图像平面与机器人行走地面之间的映射变换矩阵的唯一性准则，判别图像中的像点是否位于地面上，高于或低于地面的点被认为是障碍点，为减少实际行走过程中系统外部和内部参数变化对映射变换矩阵的影响，系统加入了在线校正映射变换矩阵模块．在提取出障碍物体边缘后，通过对其三维信息的简单恢复，建立了机器人行走空间的障碍投影图．该系统计算量小，可靠性强，能基本满足双足步行机器人实时避障的要求。     

Episode detection in videos captured using a head-mounted camera

With the advent of wearable computing, personal imaging, photojournalism and personal video diaries, the need for automated archiving of the videos captured by them has become quite pressing. The principal device used to capture the human-environment interaction with these devices is a wearable camera (usually a head-mounted camera). The videos obtained from such a camera are raw and unedited versions of the visual interaction of the wearer (the user of the camera) with the surroundings. The focus of our research is to develop post-processing techniques that can automatically abstract videos based on episode detection. An episode is defined as a part of the video that was captured when the user was interested in an external event and paid attention to record it. Our research is based on the assumption that head movements have distinguishable patterns during an episode occurrence and these patterns can be exploited to differentiate between an episode and a non-episode. Here we present a novel algorithm exploiting the head and body behaviour for detecting the episodes. The algorithms performance is measured by comparing the ground truth (user-declared episodes) with the detected episodes. The experiments show the high degree of success we achieved with our proposed method on several hours of head-mounted video captured in varying locations.     

视频中人脸位置的定量检测

现有的人脸检测评判标准通常情况下仅仅只是对人脸有无的定性检测,对于视频中人脸位置的定量描述并没有严格的规范;另外,现在的一些研究如视频人脸替换等对视频流中人脸位置的连续性有较高的要求。为了解决上述两个问题,相比之前的人脸检测以及人脸跟踪评估标准,提出了一种视频中人脸位置的定量检测评估标准,并且提出了一种视频中人脸位置的检测方法。该方法首先通过改进的Haar-Like级联分类器在目标区域中检测到人脸初始位置;然后采用金字塔光流法对人脸位置进行预测,同时引入正反向误差检测机制实现对结果的自检测,最终确定人脸位置。实验结果表明,检测标准能够对测试算法在视频人脸检测的定量描述结果给出评判,提出的检测算法在人脸位置的时间一致性上有所提升。