Development of an autonomous bridge deck inspection robotic system

The threat to safety of aging bridges has been recognized as a critical concern to the general public due to the poor condition of many bridges in the United States. Currently, the bridge inspection is conducted manually, and it is not efficient to identify bridge condition deterioration in order to facilitate implementation of appropriate maintenance or rehabilitation procedures. In this paper, we report a new development of the autonomous mobile robotic system for bridge deck inspection and evaluation. The robot is integrated with several nondestructive evaluation (NDE) sensors and a navigation control algorithm to allow it to accurately and autonomously maneuver on the bridge deck to collect visual images and conduct NDE measurements. The developed robotic system can reduce the cost and time of the bridge deck data collection and inspection. For efficient bridge deck monitoring, the crack detection algorithm to build the deck crack map is presented in detail. The impact‐echo (IE), ultrasonic surface waves (USW), and electrical resistivity (ER) data collected by the robot are analyzed to generate the delamination, concrete elastic modulus, corrosion maps of the bridge deck, respectively. The presented robotic system has been successfully deployed to inspect numerous bridges in more than ten different states in the United States.     

In‐water visual ship hull inspection using a hover‐capable underwater vehicle with stereo vision

Underwater visual inspection is an important task for checking the structural integrity and biofouling of the ship hull surface to improve the operational safety and efficiency of ships and floating vessels. This paper describes the development of an autonomous in‐water visual inspection system and its application to visual hull inspection of a full‐scale ship. The developed system includes a hardware vehicle platform and software algorithms for autonomous operation of the vehicle. The algorithms for vehicle autonomy consist of the guidance, navigation, and control algorithms for real‐time and onboard operation of the vehicle around the hull surface. The environmental perception of the developed system is mainly based on optical camera images, and various computer vision and optimization algorithms are used for vision‐based navigation and visual mapping. In particular, a stereo camera is installed on the underwater vehicle to estimate instantaneous surface normal vectors, which enables high‐precision navigation and robust visual mapping, not only on flat areas but also over moderately curved hull surface areas. The development process of the vehicle platform and the implemented algorithms are described. The results of the field experiment with a full‐scale ship in a real sea environment are presented to demonstrate the feasibility and practical performance of the developed system.     

Realization of a Pheromone Potential Field for Autonomous Navigation by Radio Frequency Identification

This paper presents the results of our research that aims to implement autonomous navigation with an artificial pheromone system. As social insects, a group of ants show advanced performance in their activity by using a chemical substance called a pheromone. By introducing an artificial pheromone system composed of data carriers and autonomous robots, the robotic system creates a potential field to navigate their group. Using this approach we have developed a pheromone density model to realize the function of pheromones with the help of data carriers. We intend to show the effectiveness of the proposed system by performing computer simulations and real experiments for one and two dimensions. The proposed system can be used for an autonomous navigation system. Results are discussed and future works are proposed.     

High‐precision control of tracked field robots in the presence of unknown traction coefficients

Accurate steering through crop rows that avoids crop damage is one of the most important tasks for agricultural robots utilized in various field operations, such as monitoring, mechanical weeding, or spraying. In practice, varying soil conditions can result in off‐track navigation due to unknown traction coefficients so that it can cause crop damage. To address this problem, this paper presents the development, application, and experimental results of a real‐time receding horizon estimation and control (RHEC) framework applied to a fully autonomous mobile robotic platform to increase its steering accuracy. Recent advances in cheap and fast microprocessors, as well as advances in solution methods for nonlinear optimization problems, have made nonlinear receding horizon control (RHC) and receding horizon estimation (RHE) methods suitable for field robots that require high‐frequency (milliseconds) updates. A real‐time RHEC framework is developed and applied to a fully autonomous mobile robotic platform designed by the authors for in‐field phenotyping applications in sorghum fields. Nonlinear RHE is used to estimate constrained states and parameters, and nonlinear RHC is designed based on an adaptive system model that contains time‐varying parameters. The capabilities of the real‐time RHEC framework are verified experimentally, and the results show an accurate tracking performance on a bumpy and wet soil field. The mean values of the Euclidean error and required computation time of the RHEC framework are equal to 0.0423 m and 0.88 ms, respectively.     

一种输电线路巡检机器人越障规划方法

针对输电线路巡检机器人的局部自主与远程控制方式,提出了一种基于有限状态机的越障规划方法.将巡检机器人的越障运动过程分解成若干离散化的关键状态,建立了运动规划的有限状态机模型.采用一种基于模糊方法的产生式系统用于推理越障模式,并产生运动序列.仿真与现场实验验证了越障规划方法的正确性和有效性,该方法可应用于输电线路巡检机器人运动控制.     

A multirobot system for autonomous deployment and recovery of a blade crawler for operations and maintenance of offshore wind turbine blades

Offshore wind farms will play a vital role in the global ambition of net zero energy generation. Future offshore wind farms will be larger and further from the coast, meaning that traditional human-based operations and maintenance approaches will become infeasible due to safety, cost, and skills shortages. The use of remotely operated or autonomous robotic assistants to undertake these activities provides an attractive alternative solution. This paper presents an autonomous multirobot system which is able to transport, deploy and retrieve a wind turbine blade inspection robot using an unmanned aerial vehicle (UAV). The proposed solution is a fully autonomous system including a robot deployment interface for deployment, a mechatronic link-hook module (LHM) for retrieval, both installed on the underside of a UAV, a mechatronic on-load attaching module installed on the robotic payload and an intelligent global mission planner. The LHM is integrated with a 2-DOF hinge that can operate either passively or actively to reduce the swing motion of a slung load by approximately 30%. The mechatronic modules can be coupled and decoupled by special maneuvers of the UAV, and the intelligent global mission planner coordinates the operations of the UAV and the mechatronic modules for synchronous and seamless actions. For navigation in the vicinity of wind turbine blades, a visual-based localization merged with the location knowledge from Global Navigation Satellite System has been developed. A proof-of-concept system was field tested on a full-size decommissioned wind-turbine blade. The results show that the experimental system is able to deploy and retrieve a robotic payload onto and from a wind turbine blade safely and robustly without the need for human intervention. The vicinity localization and navigation system have shown an accuracy of 0.65 and 0.44 m in the horizontal and vertical directions, respectively. Furthermore, this study shows the feasibility of systems toward autonomous inspection and maintenance of offshore windfarms.     

Inspection of an underwater structure using point‐cloud SLAM with an AUV and a laser scanner

This paper presents experimental results using a newly developed 3D underwater laser scanner mounted on an autonomous underwater vehicle (AUV) for real‐time simultaneous localization and mapping (SLAM). The algorithm consists of registering point clouds using a dual step procedure. First, a feature‐based coarse alignment is performed, which is then refined using iterative closest point. The robot position is estimated using an extended Kalman filter (EKF) that fuses the data coming from navigation sensors of the AUV. Moreover, the pose from where each point cloud was collected is also stored in the pose‐based EKF‐SLAM state vector. The results of the registration algorithm are used as constraint observations among the different poses within the state vector, solving the full‐SLAM problem. The method is demonstrated using the Girona 500 AUV, equipped with a laser scanner and inspecting a 3D sub‐sea infrastructure inside a water tank. Our results prove that it is possible to limit the navigation drift and deliver a consistent high‐accuracy 3D map of the inspected object.     

A 3D simulator for autonomous robotic fish

This paper presents a 3D simulator used for studying the motion control and autonomous navigation of robotic fish. The simulator's system structure and computation flow are presented. Simplified kinematics and hydrodynamics models for a virtual robotic fish are proposed. Many other object models are created for water, obstacles, sonar sensors and a swimming pool. Experimental results show that the simulator provides a, realistic and convenient way to develop autonomous navigation algorithms for robotic fish.     

一种输电线路巡检机器人控制系统的设计与实现

介绍了一种超高压输电线路巡检机器人控制系统的设计与实现方法．根据巡检作业任务的要求，采用遥控与局部自主相结合的控制模式实现巡检机器人沿线行走及跨越障碍．设计了巡检机器人有限状态机模型，实现了机器人遥控与局部自主控制的有机结合．采用基于激光传感器定位的方法实现了巡检机器人的自主越障控制．实验结果表明，该机器人可实现沿线行走及自主跨越障碍，从而验证了控制系统设计的有效性与合理性．     

An open‐source system for vision‐based micro‐aerial vehicle mapping,planning, and flight in cluttered environments

We present an open‐source system for Micro‐Aerial Vehicle (MAV) autonomous navigation from vision‐based sensing. Our system focuses on dense mapping, safe local planning, and global trajectory generation, especially when using narrow field‐of‐view sensors in very cluttered environments. In addition, details about other necessary parts of the system and special considerations for applications in real‐world scenarios are presented. We focus our experiments on evaluating global planning, path smoothing, and local planning methods on real maps made on MAVs in realistic search‐and‐rescue and industrial inspection scenarios. We also perform thousands of simulations in cluttered synthetic environments, and finally validate the complete system in real‐world experiments.     

Instance invariant visual servoing framework for part‐aware autonomous vehicle inspection using MAVs

Visual servoing approaches navigate a robot to the desired pose with respect to a given object using image measurements. As a result, these approaches have several applications in manipulation, navigation and inspection. However, existing visual servoing approaches are instance specific, that is, they control camera motion between two views of the same object. In this paper, we present a framework for visual servoing to a novel object instance. We further employ our framework for the autonomous inspection of vehicles using Micro Aerial Vehicles (MAVs), which is vital for day‐to‐day maintenance, damage assessment, and merchandising a vehicle. This visual inspection task comprises the MAV visiting the essential parts of the vehicle, for example, wheels, lights, and so forth, to get a closer look at the damages incurred. Existing methods for autonomous inspection could not be extended for vehicles due to the following reasons: First, several existing methods require a 3D model of the structure, which is not available for every vehicle. Second, existing methods require expensive depth sensor for localization and path planning. Third, current approaches do not account for the semantic understanding of the vehicle, which is essential for identifying parts. Our instance invariant visual servoing framework is capable of autonomously navigating to every essential part of a vehicle for inspection and can be initialized from any random pose. To the best our knowledge, this is the first approach demonstrating fully autonomous visual inspection of vehicles using MAVs. We have validated the efficacy of our approach through a series of experiments in simulation and outdoor scenarios.     

高压巡线机器人穿越耐张杆塔的控制算法

通过对高压输电线路中耐张杆塔的电磁场进行理论和试验分析,根据电磁场的分布规律对导航电磁传感器的布置进行试验研究,提出了采用传感器补偿算法的控制对策。试验表明:采用传感器补偿算法的控制对策,巡线机器人自主穿越耐张杆塔障碍是可行的。     

Into the dirt: Datasets of sewer networks with aerial and ground platforms

This paper presents an unprecedented set of data in a challenging underground environment: the visitable sewers of Barcelona. To the best of our knowledge, this is the first data set involving ground and aerial robots in such scenario: the sewer inspection autonomous robot (SIAR) ground robot and the autonomous robot for sewer inspection aerial platform. These platforms captured data from a great variety of sensors, including sequences of red green blue‐depth (RGB‐D) images with their onboard cameras. The set consists of 14 logs of experiments that were obtained in more than 10 different days and in four different locations. The complete length of the experiments in the data set exceeds 5 km. In addition, we provide the users with a partial ground‐truth and baselines of the localization of the platforms, which can be used for testing their localization and simultaneous localization and mapping (SLAM) algorithms. We also provide details on the setup and execution of each mission and a partial labeling of the elements found in the sewers. All the data were recorded by using the rosbag tool from robot operating system framework. Our goal is to make the data available to the scientific community as a benchmark to test localization, SLAM and classification algorithms in underground environments. The data set are available at https://robotics.upo.es/datasets/echord .     

基于任务分解的机器人路径规划导航

介绍一种基于目标分解的机器人导航系统,机器人通过激光或声纳传感器获得地图信息,并以此为基础,系统控制机器人按照事先规划路径的分解任务导航,最终利用视觉传感器发现目标球并加以捕获。分析仿真和实际环境实验数据存在差异的原因,证明系统实现了预期目标。     

<Emphasis Type="Italic">HumanPT</Emphasis>: An Open-source,HumanPT Architecture-based,Robotic Application for Low Cost Robotic Tasks

In this paper, first HumanPT architecture for low cost robotic applications is presented. HumanPT architecture differs than other architectures because it is implemented on existing robotic systems (robot  robotic controller) and exploits the minimum communication facilities for real-time control that these systems provide. It is based on well-known communication methods like serial communication (USB, RS232, IEEE-1394) and windows sockets (server–client model) and permits an important number of different type of components like actuators, sensors and particularly vision systems to be connected in a robotic system. The operating system (OS) used is Microsoft Windows, the most widely spread OS. The proposed architecture exploits features of this OS that is not a real-time one, to ensure – in case that the robotic system provide such a facility – control and real time communication with the robotic system controller and to integrate by means of sensors and actuators an important number of robotic tasks and procedures. As implementation of this architecture, HumanPT robotic application and experimental results concerning its performance and its implementation in real tasks are provided. HumanPT robotic application, developed in Visual C++, is an integrated, but simultaneously an open-source software that can be adapted in different types of robotic systems. An important number of robotic tasks or procedures including sensors and particularly vision systems can be generated and executed. Small enterprises by means of the proposed architecture and the open source software can be automated at low cost enhancing in this way their production.     

用于避障研究的微型仿生机器鱼3维仿真系统

基于VC++6.0开发环境和OpenGL（open graphics library）国际图形标准,在Windows系统下开发了微型仿生机器鱼3维仿真系统。该系统可以降低用实体机器鱼进行机器鱼避障能力研究的成本和减少在研究过程中对实体机器鱼造成的损害。采用多边形建模的方法构建了虚拟微型仿生机器鱼模型,模拟了鱼类尾鳍的摆动。提出了一种模拟红外传感器探测障碍物的虚拟射线方法。并采用实时模糊决策算法设计了基于多传感器信息的复合模糊控制器,决策微型仿生机器鱼的避障行为。仿真实验表明,复合模糊控制器实时性好、效率高;无论是单个任意形状的障碍物还是多个连续障碍物,复合模糊控制器都能有效地引导仿生机器鱼避开障碍物,到达目标点。微型仿生机器鱼3维仿真系统为研究仿生机器鱼的自主避障能力提供了可靠、逼真、便利的平台。     

Automatic concrete tunnel inspection robot system

This paper describes the development of an automatic concrete tunnel inspection system by an autonomous mobile robot. This work was motivated by the accidents in recent years that were caused by falling parts of the inner wall of concrete tunnels. This brought about serious damage to national property and led to the great concern throughout the whole society. Here, we proposed a non-destructive automatic tunnel inspection method. In this method, we aim to inspect the tunnel automatically and completely at high speed by using non-destructive sensors. For the non-destructive sensors, we employ 24 ultrasonic range sensors and six video cameras. These sensors are mounted on the same plane in the shape of a semi-ring. This ultrasonic range sensor video camera semi-ring is called a URS/VC semi-ring. This URS/VC semi-ring is mounted on an autonomous mobile robot to inspect the concrete tunnel. Experiment results show that this system can detect deformed inner walls at divisions of 14 mm when the robot moves at 20 mm/s.     

Autonomous robotic inspection and manipulation using multisensorfeedback

A six-degree-of-freedom industrial robot to which was added a number of sensors-vision, range, sound, proximity, force/torque, and touch-to enhance its inspection and manipulation capabilities is described. The work falls under the scope of partial autonomy. In teleoperation mode, the human operator prepares the robotic system to perform the desired task. Using its sensory cues, the system maps the workspace and performs its operations in a fully autonomous mode. Finally, the system reports back to the human operator on the success or failure of the task and resumes its teleoperation mode. The feasibility of realistic autonomous robotic inspection and manipulation tasks using multisensory information cues is demonstrated. The focus is on the estimation of the three-dimensional position and orientation of the task panel and the use of other nonvision sensors for valve manipulation. The experiment illustrates the need for multisensory information to accomplish complex, autonomous robotic inspection and manipulation tasks     

全电压等级无人机高空巡检自主导航线路切换技术研究

针对无人机巡检电压等级较高的输电线路时,可能会因杆塔整体结构过于复杂导致导航线路混乱无法有效完成巡检工作的问题,研究全电压等级无人机高空巡检自主导航线路切换技术。将无人机导航线路规划问题转换为蚁群确定最优或次优路径问题,利用信息素挥发因子动态自适应调整规则,改善蚁群算法易出现局部最优以及较慢收敛的情况,完成导航线路规划。基于规划结果,采用自抗扰控制算法,设计包含微分器、扰动估计补偿等设备的自抗扰控制器,通过控制器调整航向角方向,实现导航线路切换控制。实验结果表明,所研究技术能够通过不同航向角度切换,完成输电线路高空巡检目的,且航线规划效率满足无人机高空巡检自主导航线路切换的实时性要求。