Image‐based post‐disaster inspection of reinforced concrete bridge systems using deep learning with Bayesian optimization

Many bridge structures, one of the most critical components in transportation infrastructure systems, exhibit signs of deteriorations and are approaching or beyond the initial design service life. Therefore, structural health inspections of these bridges are becoming critically important, especially after extreme events. To enhance the efficiency of such an inspection, in recent years, autonomous damage detection based on computer vision has become a research hotspot. This article proposes a three‐level image‐based approach for post‐disaster inspection of the reinforced concrete bridge using deep learning with novel training strategies. The convolutional neural network for image classification, object detection, and semantic segmentation are, respectively, proposed to conduct system‐level failure classification, component‐level bridge column detection, and local damage‐level damage localization. To enable efficient training and prediction using a small data set, the model robustness is a crucial aspect to be taken into account, generally through its hyperparameters’ selection. This article, based on Bayesian optimization, proposes a principled manner of such selection, with which very promising results (well over 90% accuracies) and robustness are observed on all three‐level deep learning models.     

Autonomous UAVs for Structural Health Monitoring Using Deep Learning and an Ultrasonic Beacon System with Geo‐Tagging

Visual inspection has traditionally been used for structural health monitoring. However, assessments conducted by trained inspectors or using contact sensors on structures for monitoring are costly and inefficient because of the number of inspectors and sensors required. To date, data acquisition using unmanned aerial vehicles (UAVs) equipped with cameras has become popular, but UAVs require skilled pilots or a global positioning system (GPS) for autonomous flight. Unfortunately, GPS cannot be used by a UAV for autonomous flight near some parts of certain structures (e.g., beneath a bridge), but these are the critical locations that should be inspected to monitor and maintain structural health. To address this difficulty, this article proposes an autonomous UAV method using ultrasonic beacons to replace the role of GPS, a deep convolutional neural network (CNN) for damage detection, and a geo‐tagging method for the localization of damage. Concrete cracks, as an example of structural damage, were successfully detected with 97.7% specificity and 91.9% sensitivity, by processing video data collected from an autonomous UAV.     

Pruning deep convolutional neural networks for efficient edge computing in condition assessment of infrastructures

Health monitoring of civil infrastructures is a key application of Internet of things (IoT), while edge computing is an important component of IoT. In this context, swarms of autonomous inspection robots, which can replace current manual inspections, are examples of edge devices. Incorporation of pretrained deep learning algorithms into these robots for autonomous damage detection is a challenging problem since these devices are typically limited in computing and memory resources. This study introduces a solution based on network pruning using Taylor expansion to utilize pretrained deep convolutional neural networks for efficient edge computing and incorporation into inspection robots. Results from comprehensive experiments on two pretrained networks (i.e., VGG16 and ResNet18) and two types of prevalent surface defects (i.e., crack and corrosion) are presented and discussed in detail with respect to performance, memory demands, and the inference time for damage detection. It is shown that the proposed approach significantly enhances resource efficiency without decreasing damage detection performance.     

Concrete bridge surface damage detection using a single‐stage detector

Early and timely detection of surface damages is important for maintaining the functionality, reliability, and safety of concrete bridges. Recent advancement in convolution neural network has enabled the development of deep learning‐based visual inspection techniques for detecting multiple structural damages. However, most deep learning‐based techniques are built on two‐stage, proposal‐driven detectors using less complex image data, which could be restricted for practical applications and possible integration within intelligent autonomous inspection systems. In this study, a faster, simpler single‐stage detector is proposed based on a real‐time object detection technique, You Only Look Once (YOLOv3), for detecting multiple concrete bridge damages. A field inspection images dataset labeled with four types of concrete damages (crack, pop‐out, spalling, and exposed rebar) is used for training and testing of YOLOv3. To enhance the detection accuracy, the original YOLOv3 is further improved by introducing a novel transfer learning method with fully pretrained weights from a geometrically similar dataset. Batch renormalization and focal loss are also incorporated to increase the accuracy. Testing results show that the improved YOLOv3 has a detection accuracy of up to 80% and 47% at the Intersection‐over‐Union (IoU) metrics of 0.5 and 0.75, respectively. It outperforms the original YOLOv3 and the two‐stage detector Faster Region‐based Convolutional Neural Network (Faster R‐CNN) with ResNet‐101, especially for the IoU metric of 0.75.     

Image‐based crack assessment of bridge piers using unmanned aerial vehicles and three‐dimensional scene reconstruction

Crack assessment of bridge piers using unmanned aerial vehicles (UAVs) eliminates unsafe factors of manual inspection and provides a potential way for the maintenance of transportation infrastructures. However, the implementation of UAV‐based crack assessment for real bridge piers is hindered by several key issues, including the following: (a) both perspective distortion and the geometry distortion by nonflat structural surfaces usually appear on crack images taken by the UAV system from the pier surface; however, these two kinds of distortions are difficult to correct at the same time; and (b) the crack image taken by a close‐range inspection flight UAV system is partially imaged, containing only a small part of the entire surface of the pier, and thereby hinders crack localization. In this paper, a new image‐based crack assessment methodology for bridge piers using UAV and three‐dimensional (3D) scene reconstruction is proposed. First, the data acquisition of UAV‐based crack assessment is discussed, and the UAV flight path and photography strategy for bridge pier assessment are proposed. Second, image‐based crack detection and 3D reconstruction are conducted to obtain crack width feature pair sequences and 3D surface models, respectively. Third, a new method of projecting cracks onto a meshed 3D surface triangular model is proposed, which can correct both the perspective distortion and geometry distortion by nonflat structural surfaces, and realize the crack localization. Field test investigations of crack assessment of a real bridge pier using a UAV are carried out for illustration, validation, and error analysis of the proposed methodology.     

Structural Displacement Measurement Using an Unmanned Aerial System

Vibration‐based Structural Health Monitoring (SHM) is one of the most popular solutions to assess the safety of civil infrastructure. SHM applications all begin with measuring the dynamic response of structures, but displacement measurement has been limited by the difficulty in requiring a fixed reference point, high cost, and/or low accuracy. Recently, researchers have conducted studies on vision‐based structural health monitoring, which provides noncontact and efficient measurement. However, these approaches have been limited to stationary cameras, which have the challenge of finding a location to deploy the cameras with appropriate line‐of‐sight, especially to monitor critical civil infrastructures such as bridges. The Unmanned Aerial System (UAS) can potentially overcome the limitation of finding optimal locations to deploy the camera, but existing vision‐based displacement measurement methods rely on the assumption that the camera is stationary. The displacements obtained by such methods will be a relative displacement of a structure to the camera motion, not an absolute displacement. Therefore, this article presents a framework to achieve absolute displacement of a structure from a video taken from an UAS using the following phased approach. First, a target‐free method is implemented to extract the relative structural displacement from the video. Next, the 6 degree‐of‐freedom camera motion (three translations and three rotations) is estimated by tracking the background feature points. Finally, the absolute structural displacement is recovered by combining the relative structural displacement and the camera motion. The performance of the proposed system has been validated in the laboratory using a commercial UAS. Displacement of a pinned‐connected railroad truss bridge in Rockford, IL subjected to revenue‐service traffic loading was reproduced on a hydraulic simulator, while the UAS was flown from a distance of 4.6 m (simulating the track clearance required by the Federal Railroad Administration), resulting in estimated displacements with an RMS error of 2.14 mm.     

Bridge inspection robot system with machine vision

A robotic system for inspecting the safety status of bridges is proposed in this paper. Currently, most bridge inspections have been done manually by counting the number of cracks, measuring their lengths and widths and taking pictures of them. Thus the quality and reliability of diagnosis reports on bridges are greatly dependent upon the diligence and education of inspection workers. The robotic inspection system to be proposed consists of three parts: a specially designed car, a robot mechanism and control system for mobility and a machine vision system for automatic detection of cracks. Ultimately, this robot system has been developed for gathering accurate data in order to record the biennial changes of the bridge's safety circumstances as well as check the safety status of bridges. We also demonstrate the effectiveness of the suggested crack detecting and tracing algorithms through experiments on a real bridge crack inspection.     

A Condition‐Based Deterioration Model for the Stochastic Dependency of Corrosion Rate and Crack Propagation in Corroded Concrete Structures

Physics‐based models are intensively studied in mechanical and civil engineering but their constant increase in complexity makes them harder to use in a maintenance context, especially when degradation model can/should be updated from new inspection data. On the other hand, Markovian cumulative damage approaches such as Gamma processes seem promising; however, they suffer from lack of acceptability by the civil engineering community due to poor physics considerations. In this article, we want to promote an approach for modeling the degradation of structures and infrastructures for maintenance purposes which can be seen as an intermediate approach between physical models and probabilistic models. A new statistical, data‐driven state‐dependent model is proposed. The construction of the degradation model will be discussed within an application to the cracking of concrete due to chloride‐induced corrosion. Numerical experiments will later be conducted to identify preliminary properties of the model in terms of statistical inferences. An estimation algorithm is proposed to estimate the parameters of the model in cases where databases suffer from irregularities.     

A sematic and prior‐knowledge‐aided monocular localization method for construction‐related entities

The real‐time location of construction‐related entities is some of the most useful basic information for automated construction management. However, the implementation of most existing localization methods is limited due to the weak adaptability to construction sites. In this paper, we enhance the monocular vision technique for the localization of construction‐related entities by a sematic and prior knowledge‐based method. A deep learning algorithm is employed to segment the sematic instance in the images, and then the prior knowledge model specifies projection strategies for entities corresponding to various scenarios. Results show that the proposed method achieves satisfying positioning accuracy, is robust in low‐ratio occlusions, and can help facilitate safety early warning, activity recognition, and productivity analysis.     

红外热像技术检测建筑外墙饰面砖粘结缺陷

红外热像技术检测建筑物外墙饰面砖粘结缺陷是近年来新发展的检测手段。结合某高层教学大楼的实际检测评估案例,叙述红外热像技术的基本原理和外墙缺陷识别方法,详细介绍红外热像技术的实践运用。     

Applications of computer vision on tile alignment inspection

Subjective visual inspection is the main quality measure for tile alignment acceptance because of a lack of standard operation procedures in Taiwan. Without quantitative specifications for tile alignment, inspectors can easily manipulate the outcome; therefore, fine craftsmanship is not valued, resulting in considerable quality variation in tile installation works. This paper proposes an automated tile installation quality assurance prototype system that uses computer vision technologies. The system receives digital images of finished tile installation and has the images processed and analyzed to capture the geometric characteristics of the finished tile surface. The geometric characteristics are subsequently evaluated to determine the quality level of the tiling work. Application of the proposed automated system can effectively improve the tile alignment inspection practice and simultaneously reduce improper manipulation during acceptance procedures.     

白蚁探测技术发展概况

依靠肉眼观察法探测白蚁具有明显的缺陷，为了更为方便和精确的探测白蚁，近年来，出现了多种使用仪器探测白蚁的技术，本文对这些技术及相关仪器进行了简要介绍。     

Zernike‐moment measurement of thin‐crack width in images enabled by dual‐scale deep learning

Although crack inspection is a routine practice in civil infrastructure management (especially for highway bridge structures), it is time‐consuming and safety‐concerning to trained engineers and costly to the stakeholders. To automate this in the near future, the algorithmic challenge at the onset is to detect and localize cracks in imagery data with complex scenes. The rise of deep learning (DL) sheds light on overcoming this challenge through learning from imagery big data. However, how to exploit DL techniques is yet to be fully explored. One primary component of practical crack inspection is that it is not merely detection via visual recognition. To evaluate the potential risk of structural failure, it entails quantitative characterization, which usually includes crack width measurement. To further facilitate the automation of machine‐vision‐based concrete crack inspection, this article proposes a DL‐enabled quantitative crack width measurement method. In the detection and mapping phase, dual‐scale convolutional neural networks are designed to detect cracks in complex scene images with validated high accuracy. Subsequently, a novel crack width estimation method based on the use of Zernike moment operator is further developed for thin cracks. The experimental results based on a laboratory loading test agree well with the direct measurements, which substantiates the effectiveness of the proposed method for quantitative crack detection.     

Forward‐Backward Approach for 3D Event Localization Using Commodity Smartphones for Ubiquitous Context‐Aware Applications in Civil and Infrastructure Engineering

An inexpensive and robust 3D localization system for tracking the position of a user in GPS‐ or WLAN‐denied environments offers significant potential for improving decision‐making tasks for civil and infrastructure engineering applications. To this end, an infrastructure‐free approach for 3D event localization on commodity smartphones is presented. In the proposed method, the position of the user is continuously tracked based on the smartphone sensory data (the Forward approach) until the user reaches a certain event. Here, an event location refers to the 3D location of a user conducting value‐added activities such as tasks involved in emergency response and field reporting of operational issues. Once an event is observed, the motion trajectory of the user is backtracked from the postevent landmark to reestimate the location of the event (the Backward approach). By integrating probability distributions of the Forward and Backward approaches together, the proposed method derives the most‐likely location of the event. To validate the proposed approach, seven case studies are conducted in a multistory parking garage. The experimental results show that the probabilistic integration of the localization results from the Forward and Backward dead reckonings can produce more accurate 3D localization results when compared to a single best estimate from a one‐way dead reckoning process. Lessons learned from several real‐world case studies and open research challenges in improving localization accuracy are discussed in detail.     

Mobile Active-Vision Traffic Surveillance System for Urban Networks

Abstract:   This article discusses the development of a mobile bus-mounted machine vision system for transit and traffic monitoring in urban corridors, as required by intelligent transportation systems. In contrast to earlier machine vision technologies used for traffic management, which rely mainly on fixed-point detection and simpler algorithms to detect certain traffic characteristics, the new proposed approach makes use of a recent trend in computer vision research; namely, the active vision paradigm. Active vision systems have mechanisms that can actively control camera parameters such as orientation, focus, zoom, and vergence in response to the requirements of the task and external stimuli. Mounting active vision systems on buses will have the advantage of providing real-time feedback of the current traffic conditions, while possessing the intelligence and visual skills that allow them to interact with a rapidly changing dynamic environment, such as moving traffic and continuously changing image background.     

RELIABILITY BASED OPTIMAL INSPECTION AND MAINTENANCE FOR PIPELINE UNDER CORROSION

Nondestructive inspection tools used for pipeline inspection are uncertain in detecting corrosion pits and in sizing detected defects. Probability-based optimal inspection schedule analysis must taken these uncertainties into account. In this paper, the probability of time to failure is formulated as integral equations with domain of integration expressed as unions and intersections of domains of failure, defect detection, defect nondetection and maintenance criterion. The rate of defect detection as a function of defect size and the maintenance criterion are used as filters to eliminate the defects that are not fit for service in an expected remaining service life after inspection. Simulation procedure is given to estimate the probability distribution of time to failure by using the integral equations. To facilitate the probabilistic analysis, a standard uniformly distributed variate is introduce and used in defining the domain of detected defect and the domain of nondetected defect. The advantages of using the proposed simulation procedure are discussed. Optimal inspection schedules are selected based on the minimum value of the maximum probability of time to failure before inspections and before the time at the end of service life. Effect of inspection quality and maintenance criterion on probability of time to failure and on selecting optimal inspection schedule is presented through an illustrative application study.     

Attention-guided multiscale neural network for defect detection in sewer pipelines

Sanitary sewer systems are major infrastructures in every modern city, which are essential in protecting water pollution and preventing urban waterlogging. Since the conditions of sewer systems continuously deteriorate over time due to various defects and extrinsic factors, early intervention in the defects is necessary to prolong the service life of the pipelines. However, prior works for defect inspection are limited by accuracy, efficiency, and economic cost. In addition, the current loss functions in object detection approaches are unable to handle the imbalanced data well. To address the above drawbacks, this paper proposes an automatic defect detection framework that accurately identifies and localizes eight types of defects in closed-circuit television videos based on a deep neural network. First, an effective attention module is introduced and used in the backbone of the detector for better feature extraction. Then, a novel feature fusion mechanism is presented in the neck to alleviate the problem of feature dilution. After that, an efficient loss function that can reasonably adjust the weight of training samples is proposed to tackle the imbalanced data problem. Also, a publicly available dataset is provided for defect detection tasks. The proposed detection framework is robust against the imbalanced data and achieves a state-of-the-art mean average precision of 73.4%, which is potentially applied in realistic sewer defect inspections.     

GPS动态测量精度检测及分析

GPS动态测量技术的应用日益广泛使得其精度检测显得越发重要。本文介绍了GPS动态测量技术概况,设计了基于旋转平台的检测系统并分析了该系统用于GPS动态检测的位置和速度精度。基于检测系统对GPS单、双动态相对测量精度进行了检测实验,结果表明,静态与低动态条件下（0．314m／s以下）,水平和高程两方向上测量结果差异均很小,水平方向的标准差在5mm以内,高程方向上的误差接近1cm;不同动态条件下的双动态相对测量中,不仅出现了2—3mm的系统误差,还存在7～10mm的标准差。在实验条件得到改善基础上,该系统还可以进一步用于RTK测量的检测。     

Automated detection of sewer pipe defects in closed-circuit television images using deep learning techniques

Sanitary sewer systems are designed to collect and transport sanitary wastewater and stormwater. Pipe inspection is important in identifying both the type and location of pipe defects to maintain the normal sewer operations. Closed-circuit television (CCTV) has been commonly utilized for sewer pipe inspection. Currently, interpretation of the CCTV images is mostly conducted manually to identify the defect type and location, which is time-consuming, labor-intensive and inaccurate. Conventional computer vision techniques are explored for automated interpretation of CCTV images, but such process requires large amount of image pre-processing and the design of complex feature extractor for certain cases. In this study, an automated approach is developed for detecting sewer pipe defects based on a deep learning technique namely faster region-based convolutional neural network (faster R-CNN). The detection model is trained using 3000 images collected from CCTV inspection videos of sewer pipes. After training, the model is evaluated in terms of detection accuracy and computation cost using mean average precision (mAP), missing rate, detection speed and training time. The proposed approach is demonstrated to be applicable for detecting sewer pipe defects accurately with high accuracy and fast speed. In addition, a new model is constructed and several hyper-parameters are adjusted to study the influential factors of the proposed approach. The experiment results demonstrate that dataset size, initialization network type and training mode, and network hyper-parameters have influence on model performance. Specifically, the increase of dataset size and convolutional layers can improve the model accuracy. The adjustment of hyper-parameters such as filter dimensions or stride values contributes to higher detection accuracy, achieving an mAP of 83%. The study lays the foundation for applying deep learning techniques in sewer pipe defect detection as well as addressing similar issues for construction and facility management.