Stereo vision for planetary rovers: Stochastic modeling to near real-time implementation

NASA scenarios for lunar and planetary missions include robotic vehicles that function in both teleoperated and semi-autonomous modes. Under teleoperation, on-board stereo cameras may provide 3-D scene information to human operators via stereographic displays; likewise, under semi-autonomy, machine stereo vision may provide 3-D information for obstacle avoidance. In the past, the slow speed of machine stereo vision systems has posed a hurdle to the semi-autonomous scenario; however, recent work at JPL and other laboratories has produced stereo systems with high reliability and near real-time performance for low-resolution image pairs. In particular, JPL has taken a significant step by achieving the first autonomous, cross-country robotic traverses (of up to 100 meters) to use stereo vision, with all computing on-board the vehicle. Here, we describe the stereo vision system, including the underlying statistical model and the details of the implementation. The statistical and algorithmic aspects employ random field models of the disparity map, Bayesian formulations of single-scale matching, and area-based image comparisons. The implementation builds bandpass image pyramids and produces disparity maps from the 60×64 level of the pyramids at rates of up to two seconds per image pair. All vision processing is done in one 68020 augmented with Datacube image processing boards. We argue that the overall approach provides a unifying paradigm for practical, domain-independent stereo ranging. We close with a discussion of practical and theoretical issues involved in evaluating and extending the performance of the stereo system.     

Vision‐based navigation through urban canyons

We address the problem of navigating unmanned vehicles safely through urban canyons in two dimensions using only vision‐based techniques. Two commonly used vision‐based obstacle avoidance techniques (namely stereo vision and optic flow) are implemented on an aerial and a ground‐based robotic platform and evaluated for urban canyon navigation. Optic flow is evaluated for its ability to produce a centering response between obstacles, and stereo vision is evaluated for detecting obstacles to the front. We also evaluate a combination of these two techniques, which allows a vehicle to detect obstacles to the front while remaining centered between obstacles to the side. Through experiments on an unmanned ground vehicle and in simulation, this combination is shown to be beneficial for navigating urban canyons, including T‐junctions and 90‐deg bends. Experiments on a rotorcraft unmanned aerial vehicle, which was constrained to two‐dimensional flight, demonstrate that stereo vision allowed it to detect an obstacle to the front, and optic flow allowed it to turn away from obstacles to the side. We discuss the theory behind these techniques, our experience in implementing them on the robotic platforms, and their suitability to the urban canyon navigation problem. © 2009 Wiley Periodicals, Inc.     

Collision avoidance and map construction using synthetic vision

In this paper, we present a method to implement a navigation system for an intelligent agent that exists in a virtual world to generate collision-free motion. To observe the world, the agent uses virtual sensors consisting of the depth buffer information of a rendered image of the scene. This information is used to generate low-level collision avoidance, obstacle avoidance when moving to intermediate goals, and to create an accessibility graph and an obstacle map of the environment. The agent does not require access to the internal representation of the virtual world, which is similar to perception in mobile robots in the real world. Furthermore, the algorithm used is fast enough to work in real time.     

人眼立体视觉空间与虚拟三维空间的几何映射关系理论分析

虚拟三维空间是现实世界的数字化三维空间,而人眼立体视觉空间则是人眼视觉系统对于现实世界或虚拟世界所形成的三维立体构象。传统上人眼直接观察现实世界,确立了人眼立体视觉空间与现实世界之间的几何对应关系。而人眼立体视觉空间与虚拟三维空间是否也存在对应的几何关系？以人眼立体视觉和虚拟三维场景为研究对象,根据双目视差原理,论述了人眼立体视觉的几何模型及视觉三维模型的表示形式。通过分析虚拟空间三维点元、屏幕视差及网膜视差等三者之间的内在几何关系,利用矩阵代数建立了虚拟空间与视觉空间之间的几何映射关系。这一映射关系表明视觉三维模型与虚拟三维模型之间存在一一对应关系,也反映了人眼视觉系统在虚拟空间中的可测量性质。本文创新性之处在于得到了视觉三维模型的完整表示,突破了传统上立体显示的定性感知,提供了定量分析的基础。这一工作对于在虚拟空间中的立体体验、虚拟交互以及立体测量等实践活动具有一定理论参考价值。     

危险环境消防侦察机器人视轴规划算法研究

针对自主研发的危险环境消防侦察机器人展开研究。此机器人为履带式车身,装载一3自由度机械,高清侦察相机固定在机械臂末端。研究内容为在已知环境下自主规划机械臂规避障碍物,并调节相机视轴方向,实现对环境重点区域的锁定。首先对机械臂建立指数积（POE）正运动学模型,使用Minkowski和与三维凸包检测算法作为碰撞检测算法,使用蒙特卡洛方法在构型空间采样。再通过采样数据匹配末端视角方向,在末端建立虚拟连杆,使用碰撞检测算法保证实现视线无遮挡。仿真实验证明,使用此算法能够快速将相机视轴锁定至重点区域并保证无遮挡,且机械臂末端相机处于最佳位姿。     

Fly-inspired visual steering of an ultralight indoor aircraft

We aim at developing autonomous microflyers capable of navigating within houses or small indoor environments using vision as the principal source of information. Due to severe weight and energy constraints, inspiration is taken from the fly for the selection of sensors, for signal processing, and for the control strategy. The current 30-g prototype is capable of autonomous steering in a 16/spl times/16 m textured environment. This paper describes models and algorithms which allow for efficient course stabilization and collision avoidance using optic flow and inertial information.     

A fuzzy controller with supervised learning assisted reinforcement learning algorithm for obstacle avoidance

Fuzzy logic systems are promising for efficient obstacle avoidance. However, it is difficult to maintain the correctness, consistency, and completeness of a fuzzy rule base constructed and tuned by a human expert. A reinforcement learning method is capable of learning the fuzzy rules automatically. However, it incurs a heavy learning phase and may result in an insufficiently learned rule base due to the curse of dimensionality. In this paper, we propose a neural fuzzy system with mixed coarse learning and fine learning phases. In the first phase, a supervised learning method is used to determine the membership functions for input and output variables simultaneously. After sufficient training, fine learning is applied which employs reinforcement learning algorithm to fine-tune the membership functions for output variables. For sufficient learning, a new learning method using a modification of Sutton and Barto's model is proposed to strengthen the exploration. Through this two-step tuning approach, the mobile robot is able to perform collision-free navigation. To deal with the difficulty of acquiring a large amount of training data with high consistency for supervised learning, we develop a virtual environment (VE) simulator, which is able to provide desktop virtual environment (DVE) and immersive virtual environment (IVE) visualization. Through operating a mobile robot in the virtual environment (DVE/IVE) by a skilled human operator, training data are readily obtained and used to train the neural fuzzy system.     

自组织映射神经网络量化机器人强化学习方法研究

强化学习一词来自于行为心理学，这门学科把行为学习看成反复试验的过程，从而把环境状态映射成相应的动作。在设计智能机器人过程中，如何来实现行为主义的思想，在与环境的交互中学习行为动作？文中把机器人在未知环境中为躲避障碍所采取的动作看作一种行为，采用强化学习方法来实现智能机器人避碰行为学习。为了提高机器人学习速度，在机器人局部路径规划中的状态空量化就显得十分重要。本文采用自组织映射网络的方法来进行空间的量化。由于自组织映射网络本身所具有的自组织特性，使得它在进行空间量化时就能够较好地解决适应性灵活性问题，本文在对状态空间进行自组织量化的基础方法上，采用强化学习。解决了机器人避碰行为的学习问题，取得了满意的学习结果。     

Directing crowd simulations using navigation fields

We present a novel approach to direct and control virtual crowds using navigation fields. Our method guides one or more agents toward desired goals based on guidance fields. The system allows the user to specify these fields by either sketching paths directly in the scene via an intuitive authoring interface or by importing motion flow fields extracted from crowd video footage. We propose a novel formulation to blend input guidance fields to create singularity-free, goal-directed navigation fields. Our method can be easily combined with the most current local collision avoidance methods and we use two such methods as examples to highlight the potential of our approach. We illustrate its performance on several simulation scenarios.     

The Responsive Workbench [virtual work environment]

The Responsive Workbench concept was developed as an alternative to the multimedia and virtual reality systems of the past decade. It was recognized that almost nobody wants simulations of their working worlds in a desktop environment. Generally. users want to focus on their tasks rather than on operating the computer. The Responsive Workbench is a virtual working environment that locates virtual objects and control tools on a real “workbench”. The objects-computer-generated stereo images-are projected onto the surface of a workbench. This setting corresponds to the actual work situation in an architect's office, in surgery, and so forth. A human guide uses the virtual working environment while several collaborators watch events through stereo shutter glasses. The participants operate within a nonimmersive virtual environment. Depending on the application, the virtual workbench can integrate various input and output modules, such as motion, gesture, and voice recognition systems. This characterizes the general trend away from the classical human-machine interface. Several guides can work together in similar environments either locally or by using broadband communication networks. A responsive environment, consisting of powerful graphics workstations, tracking systems, cameras, projectors, and microphones, replaces the traditional multimedia desktop workstation     

Online learning of virtual impedance parameters in non-contact impedance control using neural networks

Impedance control is one of the most effective methods for controlling the interaction between a manipulator and a task environment. In conventional impedance control methods, however, the manipulator cannot be controlled until the end-effector contacts task environments. A noncontact impedance control method has been proposed to resolve such a problem. This method on only can regulate the end-point impedance, but also the virtual impedance that works between the manipulator and the environment by using visual information. This paper proposes a learning method using neural networks to regulate the virtual impedance parameters according to a given task. The validity of the proposed method was verified through computer simulations and experiments with a multijoint robotic manipulator.     

A PSO-based algorithm designed for a swarm of mobile robots

This paper presents an algorithm called augmented Lagrangian particle swarm optimization with velocity limits (VL-ALPSO). It uses a particle swarm optimization (PSO) based algorithm to optimize the motion planning for swarm mobile robots. Considering problems with engineering constraints and obstacles in the environment, the algorithm combines the method of augmented Lagrangian multipliers and strategies of velocity limits and virtual detectors so as to ensure enforcement of constraints, obstacle avoidance and mutual avoidance. All the strategies together with basic PSO are corresponding to real situations of swarm mobile robots in coordinated movements. This work also builds a swarm motion model based on Euler forward time integration that involves some mechanical properties such as masses, inertias or external forces to the swarm robotic system. Simulations show that the robots moving in the environment display the desired behavior. Each robot has the ability to do target searching, obstacle avoidance, random wonder, acceleration or deceleration and escape entrapment. So, in summary due to the characteristic features of the VL-ALPSO algorithm, after some engineering adaptation, it can work well for the planning of coordinated movements of swarm robotic systems.     

基于双层模糊逻辑的多机器人路径规划与避碰

针对无通信情况下的多机器人系统在未知动态环境下的路径规划问题,设计了基于双层模糊逻辑的多机器人路径规划与动态避碰系统。方向模糊控制器充分考虑了障碍物的距离信息和目标的角度信息,转化为机器人与障碍物的碰撞可能性,从而输出转向角度实现机器人的动态避障;速度模糊控制器将障碍物的距离信息作为输入,将速度因子作为输出,提高了多机器人路径规划与动态避碰系统的效率和鲁棒性。在Pioneer3-DX机器人实体上验证了该系统的可行性。     

动态环境下多移动机器人路径规划的一种新方法

本文提出一种多个移动机器人在动态环境下进行路径规划的新方法．基于栅格类的环 境表示和障碍地图,协调同一工作环境中多移动机器人的运动状态,以避免相互之间发生碰 撞．该方法的复杂性较低,具有很好的实时性能．结合机器人足球领域,进行了实验,验证 了这个方法的正确性和有效性．     

虚拟行人仿真最低能耗动态避障

虚拟行人仿真动态避障算法设计直接影响了仿真效果的真实性与科学性.大部分基于速度障碍的动态避障方法假定场景中的所有个体按照相同的避碰策略调整速度大小和方向.为提升虚拟行人仿真中局部动态避碰的真实性,本文对经典的底层分布式动态障算法ORCA中使用调节系数以区分不同行人的避障策略,同时引入行人瞬间能耗的概念,讨论行人在局部避碰过程中的能量消耗与速度变化之间的关系,在改进ORCA算法得到的可行速度域基础上使用线性规划的高效解法得到虚拟行人仿真的最低能耗避障速度.实验结果表明本文方法可以提升动态避障的仿真效率,计算性能也满足实时仿真的要求.     

基于深度学习的立体影像视差估计方法综述

三维重建技术常用于自动驾驶、机器人、无人机和增强现实等领域。视差估计是三维重建的关键步骤,随着数据集的增加、硬件和网络模型的发展,深度学习视差估计模型被广泛使用并取得良好效果。然而,这些方法常用室外场景的物体,很少使用在室内场景的数据集中。回顾了双目视差估计的深度学习方法,选用5种深度学习网络：PSMNet(pyramid stereo matching network)、GA-Net(guided aggregation network)、LEAStereo(hierarchical neural architecture search for deep stereo matching)、DeepPruner(learning efficient stereo matching via differentiable patchmatch)、BGNet(bilateral grid learning for stereo matching networks),将其运用在一套真实世界的街景数据集（KITTI2015）和两套室内场景数据集（Middlebury2014、Instereo2K...     

分布式虚拟环境中基于扫描体的碰撞检测研究

在分布式虚拟环境(DVE)中,由于网络传输的不确定性,物体的状态信息无法准确及时地传输到其他节点上,从而使得DVE中传统的基于状态序列的碰撞检测算法存在漏检和错检现象。在分析传统方法不足的基础上,提出了DVE中基于扫描体的碰撞检测算法,并给出了算法步骤,通过试验证明了该方法能够有效地避免网络传输所带来的不确定性,同时能为DVE提供具有较高精度的检测结果。     

融合神经网络的布料碰撞检测算法

目的 针对当前在虚拟环境中布料柔体碰撞检测效率慢和准确性低的问题,提出一种根节点双层包围盒树结构和融合OpenNN （open neural networks library）神经网络加速预测碰撞检测的算法。方法 首先改进了碰撞检测常用的包围盒技术,提出根节点双层包围盒算法,减少包围盒的构造时间。其次使用神经网络优化碰撞检测技术,利用神经网络可以处理大量数据的优势,每次可以检测大量基本图元是否发生碰撞,解决了碰撞检测计算复杂性高的问题。最后准确地找到碰撞粒子并做出碰撞响应。结果 在相同的复杂布料模型情况下,根节点双层包围盒算法在运行速度上比传统混合包围盒算法快,耗时缩减了5.51%~11.32%。基于OpenNN算法的总耗时比根节点双层包围盒缩减了11.70%,比融合DNN （deep neural network）的自碰撞检测算法减少了6.62%。随着碰撞检测难度的增大,当布料模型的精度增加84%时,传统物理碰撞检测方法用时增加96%,融合DNN的自碰撞检测算法用时增加90.11%,而本文基于神经网络的算法用时仅增加了68.37%,同时表现出更高的稳定性,满足使用者对实时性的要求。结论 对于模拟场景中简单模型的碰撞,本文提出的根节点双层包围盒算法比传统的包围盒方法耗时短。对于复杂模型,基于OpenNN神经网络的碰撞检测算法在效率上优于传统的包围盒算法和融合DNN的自碰撞检查算法,而且模拟效果的准确性也得以保证,是一种高效的碰撞检测方法。     

A stereo matching approach based on particle filters and scattered control landmarks

In robot localization, particle filtering can estimate the position of a robot in a known environment with the help of sensor data. In this paper, we present an approach based on particle filtering, for accurate stereo matching. The proposed method consists of three parts. First, we utilize multiple disparity maps in order to acquire a very distinctive set of features called landmarks, and then we use segmentation as a grouping technique. Secondly, we apply scan line particle filtering using the corresponding landmarks as a virtual sensor data to estimate the best disparity value. Lastly, we reduce the computational redundancy of particle filtering in our stereo correspondence with a Markov chain model, given the previous scan line values. More precisely, we assist particle filtering convergence by adding a proportional weight in the predicted disparity value estimated by Markov chains. In addition to this, we optimize our results by applying a plane fitting algorithm along with a histogram technique to refine any outliers. This work provides new insights into stereo matching methodologies by taking advantage of global geometrical and spatial information from distinctive landmarks. Experimental results show that our approach is capable of providing high-quality disparity maps comparable to other well-known contemporary techniques.