Nondestructive evaluation sensor fusion with autonomous robotic system for civil infrastructure inspection

Civil infrastructure inspection is crucial to maintaining the quality of that infrastructure, which has a great impact on the economy. Performing this inspection is costly work that requires workers to be trained on how to use varying technologies, which can be error prone when performed manually and can result in damage to the infrastructure in some cases. For this reason, nondestructive evaluation (NDE) sensors are preferred for civil infrastructure inspection as they can perform the necessary inspection without damaging the infrastructure. In this paper, we develop a fully autonomous robotic system capable of real‐time data collection and quasi‐real‐time data processing. The robotic system is equipped with several NDE sensors that allow for a sensor fusion method to be developed that successfully minimizes inspection time while performing adequate inspection of areas that require more in‐depth data to be collected. A detailed discussion of the inspection framework developed for this robotic system, and the dual navigation modes for both indoor and outdoor autonomous navigation is presented. The developed robotic system is deployed to inspect several infrastructures (e.g., parking garages, bridges) at and near by the University of Nevada, Reno campus.     

一种基于桥梁横向裂缝的病害识别方法

桥梁安全是交通安全中最重要的部分之一,桥梁定检是对桥梁安全最有效的评价方法,而在桥梁定检中对病害的识别尤其重要.据相关统计,我国混凝土桥梁的病害中有80％以上是由裂缝造成的,因此对桥梁的健康检测主要是对桥梁表观裂缝的检测与测量.为了提高检测的准确性和效率,统计桥梁裂缝病害图片的识别精度与收敛性,本文结合卷积神经网络(C...     

改进的桥梁三维重构及裂缝检测系统

针对桥梁病害检测问题, 尤其是损害程度较高的裂缝检测, 结合已有的桥梁检测系统, 本文提出一种改进的桥梁检测系统, 改进后的系统硬件是大疆M210-RTK无人机, 软件由图像数据获取模块、裂缝检测模块、3D模型构建模块构成. 其中, 裂缝检测模块增加了裂缝长宽计算功能, 对裂缝分段迭代后进行曲线拟合求取长度, 骨架法计算宽度. 在实验中设置无人机的飞行轨迹、扫描间距, 拍摄距离以及对待检测桥梁桥墩分区域编号, 最终拍摄了200张桥墩桥面图片和采集了桥梁视频数据. 通过对桥墩桥面裂缝种类的识别和裂缝长宽计算, 更全面的了解了裂缝信息及危害程度, 减少了后期人工测量, 并结合Ubuntu 16.04系统, 使用直接稀疏里程计法(DSO)进行桥梁3D建模, 3D模型能够方便直观的展示桥梁概况. 改进后的系统稳定, 方法省时省力, 适用性广, 特别是对一些跨海大桥及周边环境复杂的桥梁检测具有重要意义.     

全功能无线同步桥梁结构监测系统的研究及应用

大型桥梁在改善城市交通状况及促进社会经济发展等方面起着重要作用.长期监测运营中的桥梁,并及时评估其安全状况对保障人们生命财产安全方面具有重要意义.开发了可以综合监测桥梁结构的全功能无线传感系统,提出了精度满足系统要求、能量开销更低的时间同步算法LP-TPSN.并对厦门某悬索桥进行了桥面振动测试.结果表明,该无线监测系统测试桥面振动准确度高,同时具有操作方便、实时性好、同步开销低等特点,在桥梁结构监测及状态评估领域中有良好的应用前景.     

Condition evaluation of existing reinforced concrete bridges using fuzzy based analytic hierarchy approach

There is an urgent need for a systematic methodology for condition assessment of the bridges because the old bridges of most of the countries are inadequate to carry current-day traffic and satisfy the present codal provisions and even in newer bridges, deterioration caused by unforeseen service condition and deferred maintenance is of great concern. In view of this, an attempt has been made in this study to develop a systematic procedure and formulations for condition evaluation of existing bridges using Analytic Hierarchy Process in a fuzzy environment. Analytic Hierarchy Process (AHP) is an efficient decision making tool for complicated problems with multiple evaluation criteria and uncertainty. Fuzzy logic approach has been used to take care of the uncertainties and imprecision in the bridge inspector’s observations. In this paper, first, a methodology has been proposed for condition ranking of number of reinforced concrete (RC) bridges. Then, based on the results obtained from prioritization, rating of the most deserved bridge has been carried out using MADM based fuzzy logic. Computer programs have been developed based on the formulations presented in this paper for evaluating condition of existing bridges and the details are presented in the paper. The methodology and its application are demonstrated through a case study. This methodology would certainly help the engineers and policy makers concerned with bridge management to overcome the problem related to prioritization and decision on funding related to rehabilitation of bridges.     

约束条件下的巡线机器人逆运动学求解

高压输电线路巡检机器人是一种多关节悬挂运动机构,要实现其运动控制就需要根据机器人的本身机构特点和悬挂系统的运动约束条件进行运动学分析.本文利用微分扭转法对巡线机器人的正向运动学进行了求解,并通过对机器人悬挂系统的力学分析,得到了机器人运动学的约束条件,并在这种约束条件下,采用了一种可用来进行实时控制的迭代循环坐标下降(CCD)算法,来进行机器人的逆运动学求解.这种迭代算法对于有运动约束系统的逆运动学求解具有较强的适用性,而且它具有较好的收敛性和有效性,适合于在线计算,便于巡线机器人的实时运动控制.     

飞机表面检查机器人系统研究

研究了飞机表面检查机器人系统结构及相关关键技术，提出了针对非对称异型飞机表面爬行机器人的总体结构和关键部件，对机器人作业过程中的吸附、运动及检测功能进行了分析，并给出了解决方案。研究了基于轨迹线性化理论的机器人路径跟踪控制算法，并对基于视觉的飞机铆钉裂纹缺陷检测与识别方法。试验结果验证了算法的有效性，为飞机蒙皮铆钉缺陷的自动在线检测提供了可行的途径。     

基于人工势场的机器人遥操作安全预警域动态生成方法

为了避免机器人遥操作过程中,机器人与环境物体不必要的碰撞,以及解决"运动-等待"现象影响机器人系统安全性和效率的问题,提出了基于人工势场生成动态安全预警域的算法.该算法根据等势面原理生成的预警域既可以实时地检测机器人与环境物体的距离,又可以根据机器人的速度和加速度划定一个相对安全的区域,避免机器人在下一时刻因为速度过快与环境物体发生碰撞.最后搭建遥操作实验平台,进行了机器人在预警算法的辅助下快速抓取目标的实验.实验证明,该算法可有效提高远程机器人遥操作的安全性和效率.     

Visual and nondestructive evaluation inspection of live gas mains using the Explorer™ family of pipe robots

Visual inspection and nondestructive evaluation (NDE) of natural gas distribution mains is an important future maintenance cost‐planning step for the nation's gas utilities. These data need to be gathered at an affordable cost with the fewest excavations and maximum linear feet inspected for each deployment, with minimal to no disruption in service. Current methods (sniffing, direct assessment) are either postleak reactive or too unreliable to offer a viable and Department of Transportation–acceptable approach as a whole. Toward achieving the above goal, a consortium of federal and commercial sponsors funded the development of Explorer?. Explorer? is a long‐range, untethered, self‐powered, and wirelessly controlled modular inspection robot for the visual inspection and NDE of 6‐ and 8‐in. natural gas distribution pipelines/mains. The robot is launched into the pipeline under live (pressurized flow) conditions and can negotiate diameter changes, 45‐ and 90‐deg bends and tees, as well as inclined and vertical sections of the piping network. The modular design of the system allows it to be expanded to include additional inspection and/or repair tools. The range of the robot is an order of magnitude higher (thousands of feet) than present state‐of‐the‐art inspection systems and will improve the way gas utilities maintain and manage their systems. Two prototypes, Explorer‐I and ‐II (X‐I and X‐II), were developed and field‐tested over a 3‐year period. X‐I is capable of visual inspection only and was field‐tested in 2004 and 2005. The next‐generation X‐II, capable of visual and NDE inspection [remote field eddy current (RFEC) and magnetic flux leakage (MFL)] was developed thereafter and had field trials in 2006 and late 2007. It was successfully deployed into low‐pressure (<125 psig) and high‐pressure (>500 psig) distribution and transmission natural gas mains, with multi‐1,000‐ft inspection runs under live conditions from a single excavation. This paper will describe the overall engineering design and functionality of the Explorer? family of robots, as well as the results of the field trials for both platforms. It will highlight the importance of the various design and safety features of the in‐pipe crawler and showcase the value of data types and position‐tagged visual～NDE data collected in working pipelines under live flow conditions. © 2010 Wiley Periodicals, Inc.     

Dynamic response of trussed bridges for moving loads

A continuum approach is presented to obtain the transverse vibration of trussed bridges traversed by a single moving load. The efficiency and the accuracy of the method are determined by comparing its results with those obtained by the dynamic analysis of the bridge as a discrete lumped mass system, which can account for both truss action and flexural action of the deck in the response. Using the proposed method, a parametric study is performed to show the influence of some important parameters on the dynamic response of the bridge. The parameters include relative stiffness of the bridge deck and truss, number of panels, type of truss and speed parameter.     

BDES: A bridge design expert system

The expert system described here designs superstructures for small to medium span highway bridges. The system addresses a domain within the engineering design consultation paradigm. It is installed in a personal microcomputer environment. This paper examines the nature of the bridge design domain as practiced in the United States and its suitability for expert system application. The architecture, the context structure, the knowledge base, the control module, and the implementation of the expert system are presented.     

Computer simulation of buffeting actions of suspension bridges under turbulent wind

Suspension bridges are long-span flexible structures susceptible to various types of wind induced vibrations such as buffeting actions. In this paper, a three dimensional finite-element model formulated to deal with suspension bridges under turbulent wind is presented. In this model, all sources of geometric nonlinearity such as cable sag, force-bending moment interaction in the bridge deck and towers, and changes of bridge geometry due to large displacements, are fully considered. The wind loads, composed of steady-state wind loads, buffeting loads and self-excited loads, are converted into time domain by using the computer simulation technique. The Newmark-β step by step numerical integration algorithm is used to calculate the buffeting responses of bridges. Compared with the results obtained by classical buffeting theory, the validity of the simulation is proved.     

基于蛇形机器人桥梁缆索无损检测系统的研究

基于桥梁缆索无损检测的特殊性,且针对目前桥梁缆索检测智能化的需要,介绍了一种新型的检测系统,即基于桥梁缆索检测的蛇形机器人.此蛇形机器人具有较大的自由度,有非常好的环境适应能力.在分析蛇形机器人的本体结构特点和运动机理的基础上,并根据桥梁缆索的特性,研究了基于蛇形的器人桥梁缆索的无损检测系统.通过实验证明:此无损检测系...     

Automated wall-climbing robot for concrete construction inspection

Human-made concrete structures require cutting-edge inspection tools to ensure the quality of the construction to meet the applicable building codes and to maintain the sustainability of the aging infrastructure. This paper introduces a wall-climbing robot for metric concrete inspection that can reach difficult-to-access locations with a close-up view for visual data collection and real-time flaws detection and localization. The wall-climbing robot is able to detect concrete surface flaws (i.e., cracks and spalls) and produce a defect-highlighted 3D model with extracted location clues and metric measurements. The system encompasses four modules, including a data collection module to capture RGB-D frames and inertial measurement unit data, a visual–inertial navigation system module to generate pose-coupled keyframes, a deep neural network module (namely InspectionNet) to classify each pixel into three classes (background, crack, and spall), and a semantic reconstruction module to integrate per-frame measurement into a global volumetric model with defects highlighted. We found that commercial RGB-D camera output depth is noisy with holes, and a Gussian-Bilateral filter for depth completion is introduced to inpaint the depth image. The method achieves the state-of-the-art depth completion accuracy even with large holes. Based on the semantic mesh, we introduce a coherent defect metric evaluation approach to compute the metric measurement of crack and spall area (e.g., length, width, area, and depth). Field experiments on a concrete bridge demonstrate that our wall-climbing robot is able to operate on a rough surface and can cross over shallow gaps. The robot is capable to detect and measure surface flaws under low illuminated environments and texture-less environments. Besides the robot system, we create the first publicly accessible concrete structure spalls and cracks data set that includes 820 labeled images and over 10,000 field-collected images and release it to the research community.     

A Pseudo-Distance Algorithm for Collision Detection of Manipulators Using Convex-Plane-Polygons-based Representation

Robots have become indispensable parts for the industrial automated workshop. The distance calculation between the industrial robot and obstacles is a fundamental problem for the robotic collision-free motion control. But existing methods for this problem have the disadvantage that the accuracy and execution efficiency cannot be guaranteed at the same time. In this paper, a pseudo-distance algorithm is presented based on the convex-plane-polygons-based representation to solve this problem. In this algorithm, convex plane polygons (CPPs) and cylinders are utilized to represent obstacles and the manipulator, respectively. The spatial relationship between each CPP and each cylinder is discussed through defining condition parameters, and is divided into six conditions including three special conditions when the contact happens, and the other three conditions when there exists a certain distance between the CPP and the cylinder. The distance is modelled under different relations based on the theory of mathematics and geometry. The pseudo distance is determined through selecting the smallest value from distances between CPPs and cylinders. The numerical experiment is performed based on the proposed and two previous algorithms to estimate the shortest distance between an industrial robot and a groove-shape obstacle. And an application example is performed to show the significance of this work on the robotic motion control. Results indicate that the proposed algorithm is more efficient than previous algorithms under a certain precision requirement, and can be effectively applied in the robotic motion control.     

Close visual bridge inspection using a UAV with a passive rotating spherical shell

Today, robots are expected to be used for the inspection of infrastructures such as bridges. One important task in bridge inspection is to acquire clear images of its various parts for further damage evaluation. However, this task has not yet been completely realized because of the complexity of bridge structures. For this situation, a UAV with a passive rotating spherical shell (PRSS) that can easily maneuver while protecting itself is introduced. In this paper, we explain the development of the PRSS unmanned aerial vehicle (UAV) based on the design strategy. We then analyze its performance through simulated bridge inspection. Finally, we discuss the results of actual bridge experiments. The spherical shell has a size ( 0.95 m) and structure (fullerene) that is well‐suited for bridge inspection applications. It can also handle an impact equivalent to 2 m/s, which is more than the UAV's maximum flight speed. Test flight experiment also validated the characteristic of PRSS that shows a stable flight during disturbances. The image test also shows that the visual system can provide a full overhead view of the bridge. Likewise, it can detect a 0.1‐mm wide line that replicates the damage (e.g., a crack) from a position 0.5 m away and while the camera moves at 0.3 m/s. These characteristics have yielded a system that can acquire inspection images from critical parts of the bridge. An evaluation by third parties confirmed the effectiveness of the system. Further, the system satisfies the mandatory requirements of the Next Generation Robots for Social Infrastructure (NGRSI) program.     

Concrete Defects Inspection and 3D Mapping Using CityFlyer Quadrotor Robot

The concrete aging problem has gained more attention in recent years as more bridges and tunnels in the United States lack proper maintenance. Though the Federal Highway Administration requires these public concrete structures to be inspected regularly, on-site manual inspection by human operators is time-consuming and labor-intensive. Conventional inspection approaches for concrete inspection, using RGB image-based thresholding methods, are not able to determine metric information as well as accurate location information for assessed defects for conditions. To address this challenge, we propose a deep neural network (DNN) based concrete inspection system using a quadrotor flying robot (referred to as CityFlyer) mounted with an RGB-D camera. The inspection system introduces several novel modules. Firstly, a visual-inertial fusion approach is introduced to perform camera and robot positioning and structure 3D metric reconstruction. The reconstructed map is used to retrieve the location and metric information of the defects. Secondly, we introduce a DNN model, namely AdaNet, to detect concrete spalling and cracking, with the capability of maintaining robustness under various distances between the camera and concrete surface. In order to train the model, we craft a new dataset, i.e., the concrete structure spalling and cracking (CSSC) dataset, which is released publicly to the research community. Finally, we introduce a 3D semantic mapping method using the annotated framework to reconstruct the concrete structure for visualization. We performed comparative studies and demonstrated that our AdaNet can achieve 8.41% higher detection accuracy than ResNets and VGGs. Moreover, we conducted five field tests, of which three are manual hand-held tests and two are drone-based field tests. These results indicate that our system is capable of performing metric field inspection, and can serve as an effective tool for civil engineers.      

Xgboost application on bridge management systems for proactive damage estimation

Bridge inspection is one of the most fundamental tasks in bridge management practices. Because of limited professional manpower and budget constraints, providing prior information about possible damage can reduce inspection errors and time. The purpose of this study was to estimate the condition of bridges at a damage level, considering various influencing factors for seven different damage types by six different main structure types, using data from the Korean Bridge Management System. The extreme gradient boosting (XGBoost) method was used because it has the advantage of not assuming determinacy and independence, and it clearly can handle the numerous variables that affect damage to bridges. As a result, out of the 38 decision trees that were generated, 36 trees were derived with significant performance measures. The influence of the variables was calculated by the Shapley Additive Explanation (SHAP) value. Age, average daily truck traffic, vehicle weight limit, total length, and effective width were found to be the major factors that influenced damage to bridges. This study confirmed that more detailed structural factors were significant contributors to severe damage to complex structural designs and the use of multiple kinds of materials, such as the cross-sectional properties of girders for the concrete deck of bridges with steel girders compared to the properties of the decks for bridges made of a simple slab of reinforced concrete. The research findings emphasized the benefits of artificial intelligence in the analysis of the conditions of bridges and showed its potential for use in network-level decision making for preventive maintenance.