New Visual Invariants for Terrain Navigation Without 3D Reconstruction

For autonomous vehicles to achieve terrain navigation, obstacles must be discriminated from terrain before any path planning and obstacle avoidance activity is undertaken. In this paper, a novel approach to obstacle detection has been developed. The method finds obstacles in the 2D image space, as opposed to 3D reconstructed space, using optical flow. Our method assumes that both nonobstacle terrain regions, as well as regions with obstacles, will be visible in the imagery. Therefore, our goal is to discriminate between terrain regions with obstacles and terrain regions without obstacles. Our method uses new visual linear invariants based on optical flow. Employing the linear invariance property, obstacles can be directly detected by using reference flow lines obtained from measured optical flow. The main features of this approach are: (1) 2D visual information (i.e., optical flow) is directly used to detect obstacles; no range, 3D motion, or 3D scene geometry is recovered; (2) knowledge about the camera-to-ground coordinate transformation is not required; (3) knowledge about vehicle (or camera) motion is not required; (4) the method is valid for the vehicle (or camera) undergoing general six-degree-of-freedom motion; (5) the error sources involved are reduced to a minimum, because the only information required is one component of optical flow. Numerous experiments using both synthetic and real image data are presented. Our methods are demonstrated in both ground and air vehicle scenarios.     

Applications of dynamic monocular machine vision

The 4-D approach to real-time machine vision presented in the companion paper (Dickmanns and Graefe 1988, this volume) is applied here to two problem areas of widespread interest in robotics. Following a discussion of the vision hardware sed, first, the precise position control for planar docking between 3-D vehicles is discussed; second, the application to high speed road vehicle guidance is demonstrated. With the 5 ton test vehicle VaMoRs, speeds up to 96 km/h (limited by the speed capability of the basic vehicle) have been reached. The test run available, of more than 20 km length, has been driven autonomously several times under various weather conditions.This research has been partially supported by the German Federal Ministry of Research and Technology (BMFT), the Deutsche Forschungsgemeinschaft (DFG), the Daimler-Benz AG, and by Messerschmitt-Bölkow-Blohm GmbH (MBB).     

In‐water visual ship hull inspection using a hover‐capable underwater vehicle with stereo vision

Underwater visual inspection is an important task for checking the structural integrity and biofouling of the ship hull surface to improve the operational safety and efficiency of ships and floating vessels. This paper describes the development of an autonomous in‐water visual inspection system and its application to visual hull inspection of a full‐scale ship. The developed system includes a hardware vehicle platform and software algorithms for autonomous operation of the vehicle. The algorithms for vehicle autonomy consist of the guidance, navigation, and control algorithms for real‐time and onboard operation of the vehicle around the hull surface. The environmental perception of the developed system is mainly based on optical camera images, and various computer vision and optimization algorithms are used for vision‐based navigation and visual mapping. In particular, a stereo camera is installed on the underwater vehicle to estimate instantaneous surface normal vectors, which enables high‐precision navigation and robust visual mapping, not only on flat areas but also over moderately curved hull surface areas. The development process of the vehicle platform and the implemented algorithms are described. The results of the field experiment with a full‐scale ship in a real sea environment are presented to demonstrate the feasibility and practical performance of the developed system.     

Three-dimensional Route Planning for Unmanned Aerial Vehicles in a Risk Environment

This paper introduces a new approach for three-dimensional flight path optimization for unmanned aerial vehicles. It considers the performance of the air vehicle as well as mission specific requirements including the avoidance of no-fly areas, risk reduction in threat environments by terrain following flight or terrain masking low-level flight, and other regulations such as fixed release and approach vectors at the start and destination locations. The focus of the approach is on a proper discretization of the airspace by a network which allows the application of standard algorithms of combinatorial optimization. In contrast to conventional discretizations by grids or grid-like graphs, our network is non-regular since created by some random process. Moreover, each path in the network corresponds to a twice continuously differentiable trajectory which obeys the kinematic restrictions of the air vehicle and which is feasible with respect to the operational requirements of the mission. With suitable costs defined on the edges of the network, a minimum-cost path calculation allows to identify a trajectory of shortest length, shortest flight time, minimum flight height, or minimum visibility from the ground. The latter objectives aim to minimize the probability of being detected by hostile forces, hence increasing the survivability of the air vehicle.     

基于有限时间系统同步的自治水下航行器回收控制

基于主-从系统状态同步的思想,提出了母艇在平面运动中回收自治水下航行器（Autonomous underwater vehicle,AUV）的一种控制方法. 在给出母艇和自治水下航行器的动力学模型基础上,建立了自治水下航行器（从系统）接收母艇（主系统）的状态信息并控制自身接近母艇的主从控制方案,使母艇自主回收水下航行器的问题转化为两者的运动状态同步问题. 利用有限时间稳定性理论,设计了一种在常值海流扰动影响下,自治水下航行器能够在有限时间内被母艇回收的滑模控制器,理论证明和仿真实例证实了该控制器的有效性.     

基于多目标协同演化算法的大规模自动驾驶策略

目前无人驾驶技术领域的研究重点主要集中在单车层面的感知、决策与控制,而缺少对多车 之间交互及博弈的研究,因此无法有效降低交通系统整体事故率并提升通行效率。该文提出一种基于 合作博弈理论的大规模自动驾驶策略涌现方法。通过建立面向网联汽车、多目标优化决策的合作博弈 演化平台,并构造了一种网格道路模型和车辆运动学模型,使得系统中各车辆之间以近邻博弈的方式 进行交互;同时系统采用分布式算法并具有间接交互的特点,最终模型计算复杂度与模拟车辆规模呈 线性关系。实验结果表明,最佳策略涌现后,事故率和平均速度均取得明显改善,其中事故率降低了 90%,模型计算速度提升了 30%。该方法可应用于包含数百万辆自动驾驶汽车的城市级智能交通规划 系统中。     

The ANFIS approach applied to AUV autopilot design

This paper describes the application of neurofuzzy techniques in the design of autopilots for controlling the yaw dynamics of an autonomous underwater vehicle. Autopilots are designed using an Adaptive-Network-based Fuzzy Inference System (ANFIS), a chemotaxis tuning methodology and a fixed fuzzy rule-based approach. To describe the yaw dynamic characteristics of an autonomous underwater vehicle, a realistic simulation model is employed. Results are presented which demonstrate the superiority of the ANFIS approach. It is concluded that the approach offers a viable alternative method for designing such autopilots.     

Robot guidance using computer vision

This paper describes a procedure for locating an autonomous robot vehicle in a three-dimensional space. The method is an extension of a two-dimensional technique described by Fukui. The method is simple, requires little computation and provides the unique three-dimensional position of the robot relative to a standard mark.     

A Rotary-wing Unmanned Air Vehicle for Aquatic Weed Surveillance and Management

This paper addresses the novel application of an autonomous rotary-wing unmanned air vehicle (RUAV) as a cost-effective tool for the surveillance and management of aquatic weeds. A conservative estimate of the annual loss of agricultural revenue to the Australian economy due to weeds is in the order of A$4 billion, hence the reason why weed control is of national significance. The presented system locates and identifies weeds in inaccessible locations. The RUAV is equipped with low-cost sensor suites and various weed detection algorithms. In order to provide the weed control operators with the capability of autonomous or remote control spraying and treatment of the aquatic weeds the RUAV is also fitted with a spray mechanism. The system has been demonstrated over inaccessible weed infested aquatic habitats.     

Multisensor Fusion: An Autonomous Mobile Robot

A conventional autonomous mobile robot is introduced. The main idea is the integration of many conventional and sophisticated sensor fusion techniques, introduced by several authors in recent years. We show the actual possibility of integrating all these techniques together, rather than analyzing implementation details. The topics of multisensor fusion, observation integration and sensor coordination are widely used throuhout the article. The final goal is to demonstrate the validity of both mathematical and artificial intelligence techniques in guaranteeing vehicle survival in a dynamic environment, while the robot carries out a specific task. We review conventional techniques for the management of uncertainty while we describe an implementation of a mobile robot which combines on-line heterogeneous sensors in its navigation and localisation tasks.     

Route Planning for Autonomous Transmission of Large Sport Utility Vehicle

The autonomous driving aims at ensuring the vehicle to effectively sense the environment and use proper strategies to navigate the vehicle without the interventions of humans. Hence, there exist a prediction of the background scenes and that leads to discontinuity between the predicted and planned outputs. An optimal prediction engine is required that suitably reads the background objects and make optimal decisions. In this paper, the author(s) develop an autonomous model for vehicle driving using ensemble model for large Sport Utility Vehicles (SUVs) that uses three different modules involving (a) recognition model, (b) planning model and (c) prediction model. The study develops a direct realization method for an autonomous vehicle driving. The direct realization method is designed as a behavioral model that incorporates three different modules to ensure optimal autonomous driving. The behavioral model includes recognition, planning and prediction modules that regulates the input trajectory processing of input video datasets. A deep learning algorithm is used in the proposed approach that helps in the classification of known or unknown objects along the line of sight. This model is compared with conventional deep learning classifiers in terms of recall rate and root mean square error (RMSE) to estimate its efficacy. Simulation results on different traffic environment shows that the Ensemble Convolutional Network Reinforcement Learning (E-CNN-RL) offers increased accuracy of 95.45%, reduced RMSE and increased recall rate than existing Ensemble Convolutional Neural Networks (CNN) and Ensemble Stacked CNN.     

A requirements-based programming approach to developing a NASA autonomous ground control system

A new requirements-based programming approach to the engineering of computer-based systems offers not only an underlying formalism, but also full formal development from requirements capture through to the automatic generation of provably-correct code. The method, Requirements-to-Design-to-Code (R2D2C), is directly applicable to the development of autonomous systems and systems having autonomic properties. We describe both the R2D2C method and a prototype tool that embodies the method, and illustrate the applicability of the method by describing how the prototype tool could be used in the development of LOGOS, a NASA autonomous ground control system that exhibits autonomic behavior. Finally, we briefly discuss other possible areas of application of the approach.     

基于彩色立体视觉的障碍物快速检测方法

Real-time obstacle detection method is a key technique for machine vision based mobile robot and au-tonomous land vehicle navigation in unstructured environments. In this paper o considering the real-time requirement for stereo matching algorithm, an adaptive color segmentation method for possible obstacle region detection is first developed based on the color feature, and a simple region based stereo matching algorithm of binocular vision for realobstacle recognition is also introduced. Obstacle detection is implemented by combining the road color adaptive seg-mentation method and region based stereovision method. Lots of experiment results show that the proposed approachcan detect obstacle quickly and effectively, and this algorithm is particularly suited for road environments in which the road is relatively flat and of roughly the same color.     

Parametric ego-motion estimation for vehicle surround analysis using an omnidirectional camera

Omnidirectional cameras that give a 360° panoramic view of the surroundings have recently been used in many applications such as robotics, navigation, and surveillance. This paper describes the application of parametric ego-motion estimation for vehicle detection to perform surround analysis using an automobile-mounted camera. For this purpose, the parametric planar motion model is integrated with the transformations to compensate distortion in omnidirectional images. The framework is used to detect objects with independent motion or height above the road. Camera calibration as well as the approximate vehicle speed obtained from a CAN bus are integrated with the motion information from spatial and temporal gradients using a Bayesian approach. The approach is tested for various configurations of an automobile-mounted omni camera as well as a rectilinear camera. Successful detection and tracking of moving vehicles and generation of a surround map are demonstrated for application to intelligent driver support.Received: 1 August 2003, Accepted: 8 July 2004, Published online: 3 February 2005     

Vessel Inspection: A Micro-Aerial Vehicle-based Approach

Vessel maintenance entails periodic visual inspections of the internal and external parts of the hull in order to detect the typical defective situations affecting metallic structures, such as coating breakdown, corrosion, cracks, etc. The main goal of project MINOAS is the automation of the inspection process, currently undertaken by human surveyors, by means of a fleet of robotic agents. This paper overviews an approach to the inspection problem based on an autonomous Micro Aerial Vehicle (MAV) which, as part of this fleet, is in charge of regularly supplying images that can teleport the surveyor from a base station to the areas of the hull to be inspected. The control software approach adopted for the MAV is fully described, with a special emphasis on the self-localization capabilities of the vehicle. Experimental results showing the suitability of the platform to the application are as well reported and discussed.     

一种基于机器视觉的跑偏角估计算法

在移动机器人、汽车等自主行进过程中,由于受到道路情况等因素的影响,极易出现方向跑偏,导致其无法按照预定路径前进。为了防止出现跑偏等危险情况,研究了利用机器视觉估计预定运动方向和实际运动方向夹角即跑偏角的方法。介绍了利用机器视觉测量跑偏角的基本原理;研究了基于特征点跟踪的跑偏角估计算法。实验研究表明:该方法可以有效地在行进过程中计算跑偏角。     

Autonomous driving at the handling limit using residual reinforcement learning

While driving a vehicle safely at its handling limit is essential in autonomous vehicles in Level 5 autonomy, it is a very challenging task for current conventional methods. Therefore, this study proposes a novel controller of trajectory planning and motion control for autonomous driving through manifold corners at the handling limit to improve the speed and shorten the lap time of the vehicle. The proposed controller innovatively combines the advantages of conventional model-based control algorithm, model-free reinforcement learning algorithm, and prior expert knowledge, to improve the training efficiency for autonomous driving in extreme conditions. The reward shaping of this algorithm refers to the procedure and experience of race training of professional drivers in real time. After training on track maps that exhibit different levels of difficulty, the proposed controller implemented a superior strategy compared to the original reference trajectory, and can to other tougher maps based on the basic driving knowledge learned from the simpler map, which verifies its superiority and extensibility. We believe this technology can be further applied to daily life to expand the application scenarios and maneuvering envelopes of autonomous vehicles.     

Dynamical analysis in real time: detecting perturbations to team communication

Dynamical systems methods characterise patterns of change over time. Typically, such methods are applied only after data collection is complete. However, brief disturbances - perturbations - can occur as a process unfolds and can result in undesirable outcomes if not acted on. The application of dynamics in real time would be useful for detecting these sudden changes. Real-time analysis was accomplished by updating dynamical estimates simultaneously across different window sizes. We calculated the largest Lyapunov exponent, a measure of dynamical stability, to detect a perturbation to team communication in a simulated uninhabited air vehicle (UAV) reconnaissance mission. The perturbation consisted of information demands from a confederate that occurred unexpectedly during performance of a UAV mission. We demonstrate the use of real-time methods in detecting that perturbation as it occurred. In application, this technique would have enabled real-time intervention. Extensions of the real-time dynamical method to other domains of psychological inquiry are discussed. PRACTITIONER SUMMARY: A real-time dynamical analysis method that was developed to detect unexpected perturbations in team communication is described. The use of the method is demonstrated on perturbed communication from a three-person uninhabited air vehicle command-and-control team. The generalisability of the method is considered with respect to physiological and motor coordination dynamics.     

Autonomous Vehicles and Avoiding the Trolley (Dilemma): Vehicle Perception,Classification, and the Challenges of Framing Decision Ethics

Abstract

This article aims to introduce a degree of technological and ethical realism to the framing of autonomous vehicle perception and decisionality. The objective is to move the socioethical dialog surrounding autonomous vehicle decisionality from the dominance of “trolley framings” to more pressing ethical issues. The article argues that more realistic ethical framings of autonomous vehicle technologies should focus on the matters of HMI, machine perception, classification, and data privacy, which are some distance from the decisionality framing premise of the MIT Moral Machine experiment. To support this claim the article appeals to state-of-the-art technologies and emerging technologies concerning autonomous vehicle perception and decisionality, as a means to inform and frame ethical contexts. This is further supported by considering a context specific ethical framing for each time phase we anticipate regarding emerging autonomous vehicle technology.     

Autonomous landing of airplanes by dynamic machine vision

The 4D approach to dynamic machine vision has been validated for the application area of on-board autonomous landing approaches in the visual flight regime with computing technology available today; sensors are a video-camera, inertial gyros and an air velocity meter. The key feature of the method is the reconstruction and servo-maintained adjustment by prediction error feedback of an internal spatiotemporal model about the process to be controlled. This encompasses both the egomotion state of the aircraft carrying the sensors and the relevant geometric properties of the runway and its spatial environment. The efficiency of the approach is proved both in a hardware-in-the-loop simulation and in real test flights with a twin turbo-prop aircraft. For accuracy evaluation of the data gathered, the results of differential GPS and radiometric altitude measurements have been recorded simultaneously.